Thread: Reading files of unknown size

  1. #1
    Registered User
    Join Date
    Nov 2012
    Posts
    8

    Reading files of unknown size

    I want my program to be able to read text files in the format:

    number number
    number number
    number number...

    and so on, so there are two columns of values. The problem I'm having is, there are a lot of numbers in the file, and it can vary. The program needs to read the numbers, put one column into one array, and the other in another, perform some operations on it all, and eventually pop out two different arrays. In other words, after I've got these arrays, I don't need them for much longer. I was hoping I could dynamically allocated some arrays to store the numbers and just free them as soon as I'm done with them.

    If the file wasn't of such variable size or if it was guaranteed to be under a certain number of variables, I would have used:

    Code:
    while ( fscanf(input, "%lf %lf\n", &col_1[], &col_2[]) == 2 )
    {
          no_rows++;
    }
    Or something like this. The problem is I want to allocate "no_rows" as the array size, which I don't have until after I have read the file. So I'm stuck. How would I go about accomplishing this?

    Thanks for any help

  2. #2
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,613
    You can dynamically allocate them still. It will just take a fair amount of work in C. Basically the idea is you have to rely on realloc() to grow your "array" size.

    See here: realloc

    Now we apply that knowledge to a program. Something like

    Code:
    col_1 = realloc( NULL, sizeof(col_1[0]) * size );
    if ( col_1 == NULL ) {
        /* bad*/
    }
    and the same thing for col_2. Then at a certain time when you need more elements you do:

    Code:
    void *temp = NULL;
    size *= 2;
    temp = realloc( col_1, sizeof(col_1[0]) * size );
    if( temp == NULL )  {
        /* bad */
    } 
    col_1 = temp;
    And the same thing for col_2 again. You will probably want to roll your own function with this inside, or something else that makes sense. You will use it frequently enough.

  3. #3
    Registered User
    Join Date
    Nov 2012
    Posts
    8
    That is very helpful, thank you. I had considered coding an array which could grow, I just had no idea how to go about it. This has helped hugely, thank you.

  4. #4
    SAMARAS std10093's Avatar
    Join Date
    Jan 2011
    Location
    Nice, France
    Posts
    2,694
    Do what whiteflags correctly said, OR, just use an elegant and simple to use simple-linked-list

    If you want you can see the functions I use for a list, in the link I have in my signature
    Code - functions and small libraries I use


    It’s 2014 and I still use printf() for debugging.


    "Programs must be written for people to read, and only incidentally for machines to execute. " —Harold Abelson

  5. #5
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,613
    Here we go. Another guy who thinks linked lists are the only dynamic data structure.

  6. #6
    SAMARAS std10093's Avatar
    Join Date
    Jan 2011
    Location
    Nice, France
    Posts
    2,694
    Here we go. Another guy that doesn't read the whole posts on a thread. I said that he has an alternative, not that he must use lists. He or she can select between lists and reallocation of an array.
    Code - functions and small libraries I use


    It’s 2014 and I still use printf() for debugging.


    "Programs must be written for people to read, and only incidentally for machines to execute. " —Harold Abelson

  7. #7
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,613
    I think a list is an inappropriate suggestion given the circumstances. A linked list has easy to implement insertion and deletion anywhere in the list, but the resulting list would be generated in sequential order, and I like to know that deletion in the middle is needed before employing lists. Every data structure has drawbacks and benefits. One drawback includes the code that needs to be implemented around most C lists.

  8. #8
    SAMARAS std10093's Avatar
    Join Date
    Jan 2011
    Location
    Nice, France
    Posts
    2,694
    Of course, I agree with you that no DS is perfect. It's a trade off for sure.

    He can use the code for the list from here as a starting point and then modify it a bit, so that every node holds two arrays(one per column). Fix the size of every array an a reasonable number and you are done.

    But I think, the list one can find in that link will just make the work, so I would go with this.
    Code - functions and small libraries I use


    It’s 2014 and I still use printf() for debugging.


    "Programs must be written for people to read, and only incidentally for machines to execute. " —Harold Abelson

  9. #9
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,613
    I would not be so eager to pimp out your own code unless you are willing to offer support.

  10. #10
    SAMARAS std10093's Avatar
    Join Date
    Jan 2011
    Location
    Nice, France
    Posts
    2,694
    Well, there are comments in almost every single line. But if one wants further explanation or finds a bug, he can of course contact me ( I have provided the contact link at the bottom of the page ). However, the point is not someone to use my code, but someone to use some code that is offered for free and as a result, get helped.
    Code - functions and small libraries I use


    It’s 2014 and I still use printf() for debugging.


    "Programs must be written for people to read, and only incidentally for machines to execute. " —Harold Abelson

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Error-size of te type is unknown
    By as_rule in forum C Programming
    Replies: 5
    Last Post: 08-29-2010, 08:28 AM
  2. unknown size array
    By archriku in forum C Programming
    Replies: 14
    Last Post: 05-07-2009, 11:29 PM
  3. string array of unknown size
    By ringo_shells in forum C++ Programming
    Replies: 5
    Last Post: 05-02-2007, 01:21 PM
  4. array of unknown size
    By richdb in forum C Programming
    Replies: 7
    Last Post: 02-25-2006, 11:48 AM
  5. Reading files - Buffer size
    By (TNT) in forum C++ Programming
    Replies: 1
    Last Post: 03-20-2004, 08:44 AM