CSV Parsing in .net core

tvavrda · May 31, 2021, 2:08pm

The currently used library FileHelpers is not supporting dynamic structure creation we have been using (as of May 2021).

The related code uses DelimitedClassBuilder

github.com

origam/origam-source/blob/03cf7cac9cc29844748a097a222c1d4b564cbd3a/Origam.Workflow.FileService/FileServiceAgent.cs#L47

    
      
          public class FileServiceAgent  : AbstractServiceAgent
          {
              private object _result;
              private static readonly log4net.ILog log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
          
          
    private IXmlContainer ReadDelimitedFile(TextReader reader, XmlContainer optionsXml, string entity)
              {
                  DataSet data = CreateEmptyOutputData();
                  DataTable dt = data.Tables[entity];
                  TextReaderOptions options = TextReaderOptions.Deserialize(optionsXml.Xml);
                  DelimitedClassBuilder cb = new DelimitedClassBuilder(entity);
                  cb.IgnoreFirstLines = options.IgnoreFirst;
                  cb.IgnoreLastLines = options.IgnoreLast;
                  cb.Delimiter = options.Separator;
          
          
        foreach (TextReaderOptionsField trof in options.FieldOptions)
                  {
                      DataColumn col = null;
                      Type dataType = typeof(string);
                      if (!trof.IsIgnored)
                      {

There are different possibilities:

Replace FileHelpers altogether
CON: Lots of work
Help FileHelpers and implement the missing dynamic functionality there
PRO: Would help everyone
CON: there are lots of parts we do not need that we would have to implement (save/load class to/from code/xml/binary).
Work around and dynamically create the classes we need for FileHelpers to use
PRO: There is a very simple library that creates classes (and decorates them with attributes) at runtime. Maybe that would be sufficient.

washi · June 1, 2021, 3:55pm

I’m in favour of exloring the third option.

jsusen · August 13, 2021, 1:12pm

I have modified the TypeExtender package. Do we fork it and create our own NuGet package or do we reference it as a dll?

jsusen · August 14, 2021, 10:37am

Well. Looks like the TypeExtender problem will solve itself. The author is more responsive then I expected.

jsusen · August 17, 2021, 6:26am

jsusen · August 17, 2021, 6:47am

jsusen · August 27, 2021, 11:18am

tvavrda · August 29, 2021, 11:18am

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.