Read 1000000 chars into array.

This is a discussion on Read 1000000 chars into array. within the C Programming forums, part of the General Programming Boards category; Hi, I want to read 1MB chars from stdio into array. But I will never know how big is the ...

  1. #1
    Registered User nenpa8lo's Avatar
    Join Date
    Jan 2008
    Posts
    42

    Question Read 1000000 chars into array.

    Hi,
    I want to read 1MB chars from stdio into array. But I will never know how big is the input, it can be 1B and in next run it can be 432kB and so on. Allowed chars are:
    - digits,
    - letters,
    - white cars.
    scanf() does not work as it stops at white char. I know that I can use 'l' prefix and specify size e.g. 1000000 - however I'm not sure will scanf() stop after NULL if 'l=1000000'?

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Do you know from what you've just read how large the input is? If so, you could use fread().

    If you don't know, then there are two further options:
    1. Read a smaller piece into a buffer (using fread), then using your own array scanner (e.g. looking fore "newline", "}" or whatever indicates the end of your data), to copy from this buffer to the actual data array.

    2. Just loop around using fgetc(). This is the "simple, but slightly slower" method.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    Registered User nenpa8lo's Avatar
    Join Date
    Jan 2008
    Posts
    42
    input is STDIO (keyboard)

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Ok, so this brings the question who would type in 1MB of data all at once?

    On the other hand, "stdio" is a file of the type FILE * just like any other file you would want to read from, so both of my solutions above would work, although fgets() is certainly the better choice here.. Knowing that it's stdin, and expected to be used by hand [assuming we're not redirecting from a file], the performance is not critical [it would be different if you were to read huge amounts of data across from a file - because a file from disk can supply several megabytes per second, whilst keyboard input is generally in the 10s of bytes per second, or less], so fgetc() is certainly looking like a good choice to me.

    I'm suspecting you are asking something else, perhaps you are asking "How can I detect if the user stops typing, without hitting enter?"

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  5. #5
    Registered User nenpa8lo's Avatar
    Join Date
    Jan 2008
    Posts
    42
    Simply in Linux you can use ReadDataFromKB.out < file.txt
    And I want to handle what ever is in the file.txt. Data must be process till NULL character.

  6. #6
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Do you KNOW for sure that there is a NUL ('\0') character in your file? Otherwise, it makes more sense to process data until end of file.

    However, either (or both) case(s) can be handled quite simply with a loop around fgetc(), so is there any particular reason you don't want to do that?

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  7. #7
    Woof, woof! zacs7's Avatar
    Join Date
    Mar 2007
    Location
    Australia
    Posts
    3,459
    > However, either (or both) case(s) can be handled quite simply with a loop around fgetc(), so is there any particular reason you don't want to do that?
    Can you not use fread() instead? ie read BUFSIZ chunks at a time. Or am I missing something?

    Quote Originally Posted by man libc
    The value of BUFSIZ is chosen on each system so as to make stream I/O efficient.
    No point driving a steam car if you can drive a Ferrari
    Last edited by zacs7; 05-06-2008 at 06:31 AM.

  8. #8
    Day Dreamer
    Join Date
    Apr 2007
    Posts
    45
    Quote Originally Posted by zacs7 View Post
    Can you not use fread() instead? ie read BUFSIZ chunks at a time.
    That's exactly the question I want to ask!
    I would love to change the world but they dont give me the source code!

  9. #9
    Registered User nenpa8lo's Avatar
    Join Date
    Jan 2008
    Posts
    42
    Quote Originally Posted by matsp View Post
    Do you KNOW for sure that there is a NUL ('\0') character in your file? Otherwise, it makes more sense to process data until end of file.
    Mats
    Correct me if I'm wrong. When I do myfile.out < file.txt, file.txt is read in myfile.out as an keyboard, therefore I can not look for EOF. However I think that while(getc()) will work.

  10. #10
    Woof, woof! zacs7's Avatar
    Join Date
    Mar 2007
    Location
    Australia
    Posts
    3,459
    > Correct me if I'm wrong. When I do myfile.out < file.txt, file.txt is read in myfile.out as an keyboard, therefore I can not look for EOF. However I think that while(getc()) will work.

    Wrong . EOF will be returned appon the end of the stream also. Matsp has already said, stdin is just a FILE *. Treat it the same. Don't think of 'STDIN' as the keyboard, think of it as the standard stream for input, file or keyboard it doesn't matter.

    As matsp has said, fgetc() will work in a loop (same as getc()).
    Last edited by zacs7; 05-06-2008 at 06:54 AM.

  11. #11
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by zacs7 View Post
    > Correct me if I'm wrong. When I do myfile.out < file.txt, file.txt is read in myfile.out as an keyboard, therefore I can not look for EOF. However I think that while(getc()) will work.

    Wrong . EOF will be returned appon the end of the stream also. Matsp has already said, stdin is just a FILE *. Treat it the same.

    As matsp has said, fgetc() will work in a loop (same as getc()).
    And you will not get zero back from getc() unless the user presses CTRL-@ - or something equivalent to that.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Read coefficients of linear equations from file into 2d array
    By omaralqady in forum C++ Programming
    Replies: 6
    Last Post: 06-20-2009, 08:39 AM
  2. Replies: 16
    Last Post: 05-29-2009, 08:25 PM
  3. Replies: 1
    Last Post: 03-19-2009, 11:56 AM
  4. Read Array pro!!Plz help!!
    By Supra in forum C Programming
    Replies: 2
    Last Post: 03-04-2002, 03:49 PM
  5. Hi, could someone help me with arrays?
    By goodn in forum C Programming
    Replies: 20
    Last Post: 10-18-2001, 10:48 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21