Thread: Reading Input from file (Integer array)

  1. #1
    Registered User
    Join Date
    May 2007
    Posts
    58

    Reading Input from file (Integer array)

    How can i read a file, which consists on about 10M integers separated by \n.

    I don't know how to use fscanf and i tried reading char by char and converting to int, but it's too slow.

    Thanks.

  2. #2
    Fear the Reaper...
    Join Date
    Aug 2005
    Location
    Toronto, Ontario, Canada
    Posts
    625
    I don't particularly see fscanf being faster than char by char and converting to int. Any reason why this has to be insanely fast ?
    Teacher: "You connect with Internet Explorer, but what is your browser? You know, Yahoo, Webcrawler...?" It's great to see the educational system moving in the right direction

  3. #3
    Registered User
    Join Date
    May 2007
    Posts
    58
    But it's really slow, like 1000 int in 3 seconds.

    I'm doing the sort-on-disk problem of programming pearls by the way.

  4. #4
    Fear the Reaper...
    Join Date
    Aug 2005
    Location
    Toronto, Ontario, Canada
    Posts
    625
    I guess the next step would be to see some code. I sort of doubt it's your reading method that's causing slowdown.
    Teacher: "You connect with Internet Explorer, but what is your browser? You know, Yahoo, Webcrawler...?" It's great to see the educational system moving in the right direction

  5. #5
    Registered User
    Join Date
    May 2007
    Posts
    58
    Code:
            while((c=getc(i)) != EOF)
            {
                if(isdigit(c))
                {
                    t = (t*10) + c - '0';
                } else {
    
                    if(t<x*8000000)
                    {
                        set(T,t);
                    }
                    t=0;
                }
    
            }
    This is the loop that reads the integers and adds them into a bit array.

  6. #6
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    The code looks fine -- could be made clearer, but it looks okay. I doubt the slowness is because of this part of the code.

  7. #7
    Registered User
    Join Date
    May 2007
    Posts
    58
    This is the only part executing at the time, besides i used some printf's to count the amount of numbers introduced.

    Even set() is nothing really.

    Code:
    void set(char T[],int n)
    {
        T[n/8] &= (128 >> (n&#37;8));
    }

  8. #8
    Registered User
    Join Date
    Oct 2001
    Posts
    2,934
    >How can i read a file, which consists on about 10M integers separated by \n.
    Use malloc() to create the array, then use:
    Code:
       long i = 0;
       while (fscanf(fp, "&#37;d", &array[i++]) == 1);
    And depending on whether you actually need an array, you could just read into an int.

  9. #9
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    Time this
    while((c=getc(i)) != EOF);
    Which does nothing, except read the file.

    If that's like 99% of the time you're seeing at the moment, then nothing in the loop has any impact on the performance.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  10. #10
    Registered User
    Join Date
    May 2007
    Posts
    58
    15 secons to read thw whole file, which is 114MB

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Data Structure Eror
    By prominababy in forum C Programming
    Replies: 3
    Last Post: 01-06-2009, 09:35 AM
  2. Replies: 9
    Last Post: 12-08-2008, 10:27 AM
  3. Looking for constructive criticism
    By wd_kendrick in forum C Programming
    Replies: 16
    Last Post: 05-28-2008, 09:42 AM
  4. Post...
    By maxorator in forum C++ Programming
    Replies: 12
    Last Post: 10-11-2005, 08:39 AM