Reading Input from file (Integer array)

This is a discussion on Reading Input from file (Integer array) within the C Programming forums, part of the General Programming Boards category; How can i read a file, which consists on about 10M integers separated by \n. I don't know how to ...

  1. #1
    Registered User
    Join Date
    May 2007
    Posts
    58

    Reading Input from file (Integer array)

    How can i read a file, which consists on about 10M integers separated by \n.

    I don't know how to use fscanf and i tried reading char by char and converting to int, but it's too slow.

    Thanks.

  2. #2
    Fear the Reaper...
    Join Date
    Aug 2005
    Location
    Toronto, Ontario, Canada
    Posts
    625
    I don't particularly see fscanf being faster than char by char and converting to int. Any reason why this has to be insanely fast ?
    Teacher: "You connect with Internet Explorer, but what is your browser? You know, Yahoo, Webcrawler...?" It's great to see the educational system moving in the right direction

  3. #3
    Registered User
    Join Date
    May 2007
    Posts
    58
    But it's really slow, like 1000 int in 3 seconds.

    I'm doing the sort-on-disk problem of programming pearls by the way.

  4. #4
    Fear the Reaper...
    Join Date
    Aug 2005
    Location
    Toronto, Ontario, Canada
    Posts
    625
    I guess the next step would be to see some code. I sort of doubt it's your reading method that's causing slowdown.
    Teacher: "You connect with Internet Explorer, but what is your browser? You know, Yahoo, Webcrawler...?" It's great to see the educational system moving in the right direction

  5. #5
    Registered User
    Join Date
    May 2007
    Posts
    58
    Code:
            while((c=getc(i)) != EOF)
            {
                if(isdigit(c))
                {
                    t = (t*10) + c - '0';
                } else {
    
                    if(t<x*8000000)
                    {
                        set(T,t);
                    }
                    t=0;
                }
    
            }
    This is the loop that reads the integers and adds them into a bit array.

  6. #6
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,158
    The code looks fine -- could be made clearer, but it looks okay. I doubt the slowness is because of this part of the code.

  7. #7
    Registered User
    Join Date
    May 2007
    Posts
    58
    This is the only part executing at the time, besides i used some printf's to count the amount of numbers introduced.

    Even set() is nothing really.

    Code:
    void set(char T[],int n)
    {
        T[n/8] &= (128 >> (n&#37;8));
    }

  8. #8
    Registered User
    Join Date
    Oct 2001
    Posts
    2,934
    >How can i read a file, which consists on about 10M integers separated by \n.
    Use malloc() to create the array, then use:
    Code:
       long i = 0;
       while (fscanf(fp, "&#37;d", &array[i++]) == 1);
    And depending on whether you actually need an array, you could just read into an int.

  9. #9
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,333
    Time this
    while((c=getc(i)) != EOF);
    Which does nothing, except read the file.

    If that's like 99% of the time you're seeing at the moment, then nothing in the loop has any impact on the performance.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  10. #10
    Registered User
    Join Date
    May 2007
    Posts
    58
    15 secons to read thw whole file, which is 114MB

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Data Structure Eror
    By prominababy in forum C Programming
    Replies: 3
    Last Post: 01-06-2009, 08:35 AM
  2. Replies: 9
    Last Post: 12-08-2008, 09:27 AM
  3. Looking for constructive criticism
    By wd_kendrick in forum C Programming
    Replies: 16
    Last Post: 05-28-2008, 09:42 AM
  4. Post...
    By maxorator in forum C++ Programming
    Replies: 12
    Last Post: 10-11-2005, 08:39 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21