Trouble using ArrayList

Hi,

I posted previously that I was trying to import C code that I previously wrote into a C# utility that I recently wrote. I was advised to create a dll, but after some research, I realized it was way too difficult for me. Since my code is realitively short, I decided to attempt to convert syntax to C#. Here is where I am having difficulty.

My C code contained these structs:

Code:

---

typedef struct{                                                /* This struct defines the sub-units of time */
        char sec[10];
        char nsec[15];
        char msec[15];
} timeType;

typedef struct{                                                /* This struct defines the fields in each line of raw data */
        char count[4];
        timeType time;                                        /* time for the event */
        char desc[21];                                        /* something descriptive about this event */
        char val[11];
} eventType;

typedef struct{                                                /* This struct will hold the path of an event */
        char dest1 [20];                                /* each member of the struct represents a filename */
        char dest2 [20];                                /*                in the path of the form: _____________.csv */
        char dest3 [20];
        char dest4 [20];
        char dest5 [20];
        char dest6 [20];
} mapType;

---

I would create pointers to pointers of the mapType struct:

Code:

---

  mapType onePath[1];                                /* One instance of the map type to hold path for current operation */
  mapType *pathPtr = onePath;

  eventType **ptrArray1;                        /* Pointers to dynamic arrays of pointers: each will hold a pointer to an event struct */
  eventType **ptrArray2;
  eventType **ptrArray3;
  eventType **ptrArray4;
  eventType **ptrArray5;
  eventType **ptrArray6;

---

Finally, I would allocate space to fill each array with raw data from a csv file. The ptrArrays are sent to a dump function where each is written out to a file:
( the dest. is created in another function. It is an array of filenames (.csv) )

Code:

---

/*
                        The number of lines will be calculated for each file in the path.
                        A dynamic array for structs will be allocated for each file and then
                        filled with the raw data of the file using the function CsvParse.  Finally,
                        the space used for the dynamic array will be freed. 
                        */

                        //calloc(numberoflines+2...) so that the last set of the array of pointers
                        //are always NULL.  calloc will initialize all to 0 (NULL).  This will be useful
                        //when testing when we've moved beyond the scope of valid data in the array.
                        mfcCount = CountLines(inputFilename);
                        ptrArray1 = calloc(mfcCount+2, sizeof(eventType *));
                        CsvParse(inputFilename, ptrArray1);

                        numOfLines = CountLines( pathPtr->dest2 );
                        ptrArray2 = calloc(numOfLines+2, sizeof(eventType *));
                        CsvParse(pathPtr->dest2, ptrArray2);

                        numOfLines = CountLines( pathPtr->dest3 );
                        ptrArray3 = calloc(numOfLines+2, sizeof(eventType *));
                        CsvParse(pathPtr->dest3, ptrArray3);

                        numOfLines = CountLines( pathPtr->dest4 );
                        ptrArray4 = calloc(numOfLines+2, sizeof(eventType *));
                        CsvParse(pathPtr->dest4, ptrArray4);

                        numOfLines = CountLines( pathPtr->dest5 );
                        ptrArray5 = calloc(numOfLines+2, sizeof(eventType *));
                        CsvParse(pathPtr->dest5, ptrArray5);

                        numOfLines = CountLines( pathPtr->dest6 );
                        ptrArray6 = calloc(numOfLines+2, sizeof(eventType *));
                        CsvParse(pathPtr->dest6, ptrArray6);
                       
                        sprintf(summaryFilename, "RawSummary_mfcDevice%d.csv", mfcOutputFileNumber);
                        mfcOutputFileNumber++;

                        DumpTimeline_mfc(summaryFilename, mfcCount, ptrArray1, ptrArray2, ptrArray3,
                                                                                                        ptrArray4, ptrArray5, ptrArray6);
                        free(ptrArray1);
                        free(ptrArray2);
                        free(ptrArray3);
                        free(ptrArray4);
                        free(ptrArray5);
                        free(ptrArray6);

---

Now, my problem is that I want the number of ptrArrays to be data driven. I am thinking that if I read an external file that has the sequence of the files, I will know how many ptrArrays to allocate. How can I use the ArrayList in C# to perform this? I want to be able to store raw data in eventType structs into a data-driven number of ArrayList elements (appologies for my wording, I'm new to programming) and eventually dump them to an external file.

My program will have a database of many raw .csv files. It will access certain ones based on a sequence (in the form of filenames) given in an external file. It will search the first file for a start timestamp, then the next for a corresponding timestamp, and so on. Each completed timeline will be dumped to an external summary file.

So far, on the C# side I have a function that opens and external file and creates the filenames path:

Code:

---

public void FillTemplates(string inputFilename, ArrayList fileTemplates_MFC, ArrayList fileTemplates_SW)
        {
            string theSourcePath;
           
            //Create Path
            theSourcePath = "C:\\Documents and Settings\\Mobile2\\Desktop\\PROJECT SOURCE FILES\\";
            theSourcePath = String.Concat(theSourcePath, inputfilename);

            using (CSVReader csv = new CSVReader(@theSourcePath))
            {
                string[] fields;

                //MFC templates
                fields = csv.GetCSVLine();
                foreach (string field in fields)
                {
                    fileTemplates_MFC.Add(field);
                }

                //SW templates
                fields = csv.GetCSVLine();
                foreach (string field in fields)
                {
                    fileTemplates_SW.Add(field);
                }

            }

        }

 public void FilePath(string pathStart, ArrayList pathArray, ArrayList fileTemplates)
        {

                int  found = 0;                                                        /* Flag to indicate path found */
            int i = 0;
            string filename = "paths.csv";                                /* name of file that contains filemap */
            string theSourcePath;
            string[] fields;
            string tempString;

            //Create Path
            theSourcePath = "C:\\Documents and Settings\\Mobile2\\Desktop\\PROJECT SOURCE FILES\\";
            theSourcePath = String.Concat(theSourcePath, filename);

            using (CSVReader csv = new CSVReader(@theSourcePath)) /* read each line in file until correct one found */
            {
                while((fields = csv.GetCSVLine()) != null)
                {
                    if (pathStart.CompareTo(fields[0]) == 0)      /* compare */
                    {
                        found = 1;
                        break;                                    /* break out of loop when found */
                    }
                }
            }
           

                /* if found, parse the line that contains correct path */
                if ( found )
                {
                foreach (string field in fields)
                {
                    tempString = fileTemplates[i];
                    tempString = String.Concat(field);
                    tempString = String.Concat(".csv");
                    pathArray.Add(tempString);
                    i++;
                }
                }
                else
                { Console.WriteLine ("Start of path {0} not found.  Please define path in paths.csv\n", pathStart); }
                       
        }

---

Thank You for your time. Please let me know if you need further explaination of what I am trying to accomplish.

Before any thinking, if you're using .Net 2.0, don't use ArrayList. Use a generic List; it's type-safe and has better performance.

I won't lie; I have no idea what you're asking. Can you explain in a more general sense what you're trying to accomplish (without code)? Here's what I've understood so far: you've got a bunch of .csv files. (Do they all have the same structure, or are some "master" files and some "data" files?) You want to pass in a file that contains other filenames. (Does this first file also have the start timestamp? Where does this start timestamp come from?) You want to go hunting through the collection of files for some record that has something to do with the timestamp. (Then you want to write out a file that contains all the records that matched the timestamp?) As you can see, I'm really shooting at flies here.

One thing to note is that C# is a different language than C. You'd be better off looking for the C# way to do things rather than how to write C in C#.

Thanks for the reply. Let me try again:

1. I'm new to C# so what is a generic list? If it is another class, I should be able to search it on msdn right?

2. I have a bunch of .csv files all of the same structure. Lets say 6 .csv files. Each represents a device. The .csv file contains a list of timestamps (500), each representing a time (start,end) of when the device was accessed. I want to load them all into memory so that I can search through them for form a complete timeline.

A timeline will have this structure

device1(start,end) --> device2(start,end) -->device3(start,end)--> ... -->device6(start,end)

The final summary file will have 500 of these lines.

My method is to pick up the first entry in the device1.csv file and pickup corresponding times as I move sequentally through the devices (stoping at device6). I want to know how much time the devices are taking to operate.

So my question is, how can I utilize ArrayList/generic List to perform this operation. That is, create a dynamic list of raw data that can be searched through and outputted.

Thanks, and please let me know if I can further clarify or re-explain. Your help is greatly appreciated. :)

Also, I am using MSVS 2005...I think it is the .net 2.0 framework.

All right...take what I say with a grain of salt; this is how I'd do it.

Sounds like each .csv file has a collection of entries, so we'll have a class Entry that contains whatever data each entry has, including the start and end timestamps. The collection you use to store these entries depends on how you use them. Do you always look up an entry by start timestamp? If so, a keyed collection like a dictionary would be the best data structure to use. If you absolutely have to look through the collection sequentially, then you can use the generic list. We'll explore both of these options later; for now, suffice to say that we have a class CsvFile that contains the appropriate collection of Entry objects.

If I understand you right, you may have n files. So, we need a collection of CsvFiles. This might as well be a generic list. The syntax to use a generic list is List<T> where T is the name of the class. So, List<int> would be a list of ints. For your purpose, you'd want a List<CsvFile>. Consider the following almost-code:

Code:

---

public CsvFile[] ReadFiles(string[] filenames)
{
  List<CsvFile> retValue = new List<CsvFile>();
  foreach(string s in filenames)
  {
    using(FileStream file = new FileStream(s))
    {
      //CsvParse takes a file stream and
      //returns a CsvFile with all the data from the file
      retValue.Add(CsvParse(file));
    }
  }
  return retValue.ToArray();
}

---

You'll notice the return value is an array of CsvFiles, not the List<CsvFile>. The reason for this is that the array is readonly (in the sense that you can't add or delete elements; you can still change the elements).

The next step is to do your timeline construction from all the .csv data you have in your CsvFile array. This is where the structure of the collection of Entries makes a difference. Before I can go on, I need you to answer a question: what is the structure of the timeline? Does every entry in the timeline have the same start timestamp? Does every entry in the timeline have a start timestamp equal to the end timestamp of the previous entry (except, of course, for the first one)? How are the entries in the timeline related?

The structure of the timeline is basically this:

(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)

There is some dead-time between the devices so a timeline might look like this:

(89138.73934),(89138.73938),(89138.73940),(89138.7 3941), ... ,(89138.74084),( 89138.74277)

The .csv file for device1 contains n entries of (start,end) times, as does the other .csv files for each device. It is my programs job to:

-start at the first (start,end) entry in the device1 file
-move to device2.csv, use previous end time to search for the next corresponding time, save it
-move to device3.csv, use previous end time to search for the next corresponding time, save it
.
.
.
-repeat process for the second (start,end) entry in the device1 file
.
.
.
-repeat for third entry...

my final output is a .csv file with timelines:
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
(device1-start),(device1-end),(device2-start), (device2-end), ... , (device6-start), (device6-end)
...

in the end, I use the timeline in conjunction with the MS Excel object model and do some analysis.

So the entries are related in that for one run of the machine, the machine accessed a sequence of these devices (1->6) and the timestamps represent how long it took the devices to perform their task. Every entry in the timeline as a different start time (corresponds to some time after the end time of the previous device).

Hope this helps.

There's probably some fancy data structure trick that you could use, but it's simplest to just use the List and swap it out for another data structure if you need a faster one. So, the definition of your CsvFile class could be a simple wrapper over an array of entries. Some more almost-code:

Code:

---

using System.Collections;

class CsvFile : IEnumerable<Entry>
{
  private Entry[] m_entries = new Entry[0];

  public void SetEntries(List<Entry> entries)
  {
    m_entries = entries.ToArray();
  }

  public IEnumerator<Entry> GetEnumerator()
  {
    foreach (Entry entry in m_entries)
      yield return entry;
  }

  IEnumerator IEnumerable.GetEnumerator()
  {
    return GetEnumerator()
  }
}

---

Naturally, if you're going to need to have random access to the entries stored in a file, you'd want to add an indexing operator. However, it sounds like you have to search each file sequentially, so I didn't add one.

Now all you need is a definition for CsvParse and a method to put it all together. CsvParse should do something like this:

Code:

---

public void CsvParse(FileStream file)
{
  CsvFile retValue = new CsvFile();
  List<Entry> list = new List<Entry>();

  /* for everything in the file, add it to the list */

  retValue.SetEntries(list);
  return retValue;
}

---

The timeline method should be something like this:

Code:

---

public void MakeTimelineFile(CsvFile[] files)
{
  //make sure files has at least one item in it
  List<Entry> list = new List<Entry>();
  foreach(Entry entry in files[0])
  {
    list.Add(entry);
    //save end timestamp of entry to some local variable
    for(int i = 1; i < files.Length; i++)
    {
      foreach(Entry nextEntry in files[i])
      {
        //if begin timestamp of nextEntry is close enough to end timestamp local variable
          //save end timestamp of nextEntry to local variable
          //add nextEntry to list
          //break;
      }
    }

    //do something with all the entries you've found
    list.Clear();
  }
}

---

You can see that this is inefficient because you have to search through each file's data from the beginning on every pass. If the files have entries in a chronological order, you could save some time by implementing an indexing operator on CsvFile and saving the current index of the entry on each pass through the loop.

Thank You. I'll slowly work through your code and post any questions.

I do have one to start off: why is it better to use List rather than ArrayList. What do you mean by type-safe? When I use ArrayList in the compiler, it shows up as blue, List stays black. Am I forgetting a library?

I turned off the color highlighting for class names a long time ago, but I'm guessing you need a using statement:

Code:

---

using System.Collections.Generic

---

List is better than ArrayList because it avoids several problems. First, an ArrayList can hold any object, so you could have an ArrayList with both an int and a float in it. This causes obvious problems when trying to cast the items you get from an ArrayList. The casting is another problem all by itself; casting is expensive, and ArrayList forces you to do a lot of it. ArrayList also has problems when it stores a value type, like an int or a struct. The ArrayList has to 'box' the value type so that it can store it, causing more performance penalties.

Why am I getting errors for "Entry"? - "The type or namespace 'Entry' could not be found (are you missing a using directive or an assembly reference?)

What is missing from the class definition CsvFile? It looks complete to me.

For the class definition CsvFile, would this be my "struct" as in C?
Here is what I used before:

Code:

---

typedef struct{                                                /* This struct defines the sub-units of time */
        char sec[10];
        char nsec[15];
        char msec[15];
} timeType;

typedef struct{                                                /* This struct defines the fields in each line of raw data */
        char count[4];
        timeType time;                                        /* time for the event */
        char desc[21];                                        /* something descriptive about this event */
        char val[11];
} eventType;

---

Sorry for the very general questions. I have more experience in C++, I just learned C and I started learning C# last week.

Is this:

Code:

---

private Entry[] m_entries = new Entry[0];

---

supposed to be replaced with elements of my struct?

What is supposed to be in the < > of : IEnumerable< > ?

Aye, you'll remember that I never defined an Entry class in code. That part is more or less up to you. If you want to define Entry similar to your structs, you'll need something like this:

Code:

---

class Time
{
        public string sec;
        public string nsec;
        public string msec;
}

class Entry
{
        public string count;
        public Time time = new Time();
        public string desc;
        public string val;
}

---

To read in one of your structs, you should read in the number of bytes for each field and then convert it to a string using the following function:

Code:

---

static string StringFromByteArray(byte[] bytes)
{
        StringBuilder strBuild = new StringBuilder();

        foreach (byte b in bytes)
                strBuild.Append((char)b);

        return strBuild.ToString();
}

---

If you wanted to get fancier, you could convert to other data types. For instance, the fields of the Time class and the count field of the Entry class look like they'd be more natural as ints.

Sorry, I didn't realize what Entry refered to. Thanks, I'll post further questions.