Thread: Parsing a file of ints into an int array?

I am trying to parse a file filled with integers (one per line, presumably EOF at the end) into an array of ints. The file will contain several thousand ints but I want o produce well designed code that will work with any size file and will do some error checking.

What is the most efficient way of figuring out the number of ints before asking malloc to reserve the memory for the array of ints?

If I use atoi() or strtol() for parsing, I presume I will have an outer loop checking if the next char is not EOF ( while (c != EOF) ) then have an inner loop parses line by line (looking for \n ??).

Should I use strtol (better error checking ? ) instead of atoi. All the ints will fit easily within a 32 bit int.

How do I go about doing this efficiently and well?

Thanks.

What is the most efficient way of figuring out the number of ints before asking malloc to reserve the memory for the array of ints?

Depends on where you want the efficiency to be. If you're willing to use a platform-specific (or non-portable, at least) construct, you can get the size of the file and use that to find an upper bound on potential memory usage: assume each line will be a single digit (plus newline), so divide the file size by 2 and allocate space for that many ints. Then you only have to read the file once; however, you'll allocate more memory than you actually need. This might not be an issue, and you can always realloc() at the end.

If you don't want to over-allocate, you'll just have to read the file twice, assuming you need to allocate the proper number up front: one read to count newlines, the next to read them. The second loop, on most sensible systems, will be fast because the file will be cached by the OS.

There's also the possibility of guessing at a reasonable number, and calling realloc() as necessary while looping. This doesn't fit your requirement of allocating all beforehand, but it's worth a mention.

If I use atoi() or strtol() for parsing, I presume I will have an outer loop checking if the next char is not EOF ( while (c != EOF) ) then have an inner loop parses line by line (looking for \n ??).

Should I use strtol (better error checking ? ) instead of atoi. All the ints will fit easily within a 32 bit int.

Read lines with fgets(), parse with strtol(). atoi() returns int, which might be 16 bits, and if you overflow with atoi(), it's undefined behavior. strtol() returns a long, which cannot be smaller than 32 bits, plus it will tell you if something went wrong.

1. you could do two passes through the file. first pass counts the number of ints, second pass allocates the array and stores the ints. tradeoff is reading the file twice.
2. you could do a single pass where you start with a guess of how big the file will be (maybe get the file size and guess from that) and then if the count is higher than you expected, realloc a new size. tradeoff is possible copying when doing the realloc.

strtol's error checking is not really enough, as it will return 0 if the value is not parseable but you don't know if the value was actually 0 or garbage. sscanf returns a separate indication that the value was convertible.
the end of file indication is when your fread, fgetc etc returns an end of file indication. its different depending on how you read a line.