Thread: Questions about fread() & fwrite()

  1. #1
    Registered User
    Join Date
    Nov 2016
    Posts
    5

    Questions about fread() & fwrite()

    fread() and fwrite() are advised to me as a way to do fast IO. They are really useful. But I met two problems about them. My platform is Win64, Mingw32, gcc-3.4.5.





    Problem I
    -------------------------------------

    c reference of fread() says:
    If this number differs from the count parameter, either a reading error occurred or the end-of-file was reached while reading.
    Seems fread() will stop if reaches EOF. And I wrote:

    Code:
    #include <stdio.h>
    
    #define IOBUFSIZ 100000char inbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      fread(inbuf, ~0u, 1, stdin);
      // do something ....
      return 0;
    }
    But it didn't read anything. The buffer is empty. And I wrote:

    Code:
    fread(inbuf, IOBUFSIZ, sizeof(char), stdin);
    Success.

    Why
    Code:
    fread(inbuf, ~0u, 1, stdin);
    didn't work?




    Problem II
    ------------------------------
    I turned to study fwrite():
    Code:
    #include <stdio.h>
    
    #define IOBUFSIZ = 100000;
    char outbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      // do something ...
      fwrite(outbuf, sizeof(outbuf), 1, stdout);
      return 0;
    }
    outbuf[] is global so it should be filled with zero. But the code print not only what I want to print, but also mountains of 'a' character following by. Where did the 'a' s came from?


    Thanks you for spending your time reading my post.

    Regards

  2. #2
    Registered User
    Join Date
    Nov 2016
    Posts
    5

    More Information

    I solved Problem II.

    I tried some code below on a linux server, which uses gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1), whose compile option is:

    gcc Main.c -o Main -fno-asm -O2 -Wall -lm --static -std=c99

    CODE RESULT
    Code:
    #include <stdio.h>#include <string.h>
    
    
    #define IOBUFSIZ 100000
    char inbuf[IOBUFSIZ];
    char outbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      fread(inbuf, ~0u, 1, stdin);
      memcpy(outbuf, inbuf, sizeof(outbuf));
      fwrite(outbuf, sizeof(outbuf), 1, stdout);
      return 0;
    }
    A Not allowed system call,TO FIX THIS , ask admin to recompile the client
    Code:
    #include <stdio.h>
    #include <string.h>
    
    
    #define IOBUFSIZ 100000
    char inbuf[IOBUFSIZ];
    char outbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      fread(inbuf, sizeof(inbuf), 1, stdin);
      memcpy(outbuf, inbuf, sizeof(outbuf));
      fwrite(outbuf, sizeof(outbuf), 1, stdout);
      return 0;
    }

    Success. Outputed what it got, without anything else.


    Now I solved problem II.
    Seems the extern 'a' s is a bug of windows because on ubuntu it didn't happen.

    But for problem I, why? A not allowed system call?

  3. #3
    Nasal Demon Xupicor's Avatar
    Join Date
    Sep 2010
    Location
    Poland
    Posts
    179
    As for fread(inbuf, ~0u, 1, stdin); - explain what you meant by this, exactly?

    As for the second problem - I don't get your behavior on GCC 6.1.0 on Windows at all. Please verify that your array is full-o'-null. ; )
    Code:
      for(int i = 0; i < IOBUFSIZ; ++i) {
        //printf("%d", (int)outbuf[i]); // if you want to see with your own eyes
        assert(outbuf[i] == '\0');
      }

  4. #4
    Registered User
    Join Date
    Jun 2015
    Posts
    1,640
    Remember that the 2nd argument to fread is the size of each element to read and the 3rd argument is the number of such elements to read. So you are asking to read 1 element of over 4-billion bytes. Unless you actually enter that many bytes the fread will fail (because it couldn't read a whole element) and return 0. Try switching the order of those two elements.

    As for fwrite, it doesn't stop writing just because it encounters a 0 byte. So your output seems reasonable since you are sending all those 0's to your terminal and it can print whatever it wants in response. Windows and Ubuntu respond differently, but in either case you are still sending all those zero bytes to the terminal. Normally you would only write the part of the buffer that has good data in it. And you would normally set the maximum amount to read to be less than or equal to your buffer size since otherwise it could overflow.
    Code:
    char buf[10000];
    size_t nelems_read = fread(buf, 1, sizeof buf, stdin);
    if (nelems_read == 0) {
        // read error
    }
    size_t nelems_written = fwrite(buf, 1, nelems_read, stdout);
    if (nelems_written != nelems_read) {
        // write error
    }
    Last edited by algorism; 11-05-2016 at 10:44 AM.

  5. #5
    Registered User
    Join Date
    Nov 2016
    Posts
    5

    About fread()

    Quote Originally Posted by Xupicor View Post
    As for fread(inbuf, ~0u, 1, stdin); - explain what you meant by this, exactly?

    As for the second problem - I don't get your behavior on GCC 6.1.0 on Windows at all. Please verify that your array is full-o'-null. ; )
    Code:
      for(int i = 0; i < IOBUFSIZ; ++i) {
        //printf("%d", (int)outbuf[i]); // if you want to see with your own eyes
        assert(outbuf[i] == '\0');
      }
    For fread(inbuf, ~0u, 1, stdin), I want it to read the whole input ( since its size is smaller than ~0u and fread() will stop when it saw EOF).

    I tried your code and the program runs successfully! It's amazing! There are no 'a' s!

    Thanks you. :)

  6. #6
    Registered User
    Join Date
    Nov 2016
    Posts
    5
    Quote Originally Posted by algorism View Post
    Remember that the 2nd argument to fread is the size of each element to read and the 3rd argument is the number of such elements to read. So you are asking to read 1 element of over 4-billion bytes. Unless you actually enter that many bytes the fread will fail (because it couldn't read a whole element) and return 0. Try switching the order of those two elements.

    As for fwrite, it doesn't stop writing just because it encounters a 0 byte. So your output seems reasonable since you are sending all those 0's to your terminal and it can print whatever it wants in response. Windows and Ubuntu respond differently, but in either case you are still sending all those zero bytes to the terminal. Normally you would only write the part of the buffer that has good data in it. And you would normally set the maximum amount to read to be less than or equal to your buffer size since otherwise it could overflow.
    Code:
    char buf[10000];
    size_t nelems_read = fread(buf, 1, sizeof buf, stdin);
    if (nelems_read == 0) {
        // read error
    }
    size_t nelems_written = fwrite(buf, 1, nelems_read, stdout);
    if (nelems_written != nelems_read) {
        // write error
    }
    I switched the order but it happened again. :(

    I tested the following code:

    Code:
    #include <stdio.h>#include <string.h>
    
    
    #define IOBUFSIZ 100000
    char inbuf[IOBUFSIZ];
    char outbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      fread(inbuf, 1, sizeof(inbuf)*2, stdin);
    
    
      for (char *a = inbuf; *a; ++a) {
        putchar(*a);
      }
    
    
      puts("\n\n---------------------------------------\n");
    
    
      for (char *a = outbuf; *a; ++a) {
        putchar(*a);
      }
    
    
      return 0;
    }
    And it seems outbuf contained what I read. It is array overflow, fwrite() wrote bytes to outbuf.

    Then I tried:

    Code:
    #include <stdio.h>#include <string.h>
    
    
    #define IOBUFSIZ 100000
    char inbuf[IOBUFSIZ];
    char outbuf[IOBUFSIZ];
    
    
    int main(int argc, char const *argv[])
    {
      fread(inbuf, 1, sizeof(inbuf)*2.1, stdin);
    
    
      for (char *a = inbuf; *a; ++a) {
        putchar(*a);
      }
    
    
      puts("\n\n---------------------------------------\n");
    
    
      for (char *a = outbuf; *a; ++a) {
        putchar(*a);
      }
    
    
      return 0;
    }
    And it outputed nothing. I tried to add up the number 2.1, same thing. I thought it was array overflow again. fwrite() should have written something to somewhere I don't know, and destroyed the run-time environment.

    For fwrite, I think it is right that it sent zero to terminal and the terminal just output whatever it wants. :)

    Thanks you.

  7. #7
    Registered User
    Join Date
    Jun 2015
    Posts
    1,640
    You can't tell fread to read twice as much as your buffer can hold! Why would you do that? That leads to undefined behavior and then anything can happen. You can ask fread to read LESS than the buffer size, but not more. Usually the actual buffer's size is used. (I should have emphasized last post that asking for billions of bytes is a strange request in the first place.)

    You don't seem to understand fread at all. It reads up to the requested number of elements (of the given size) into the pre-allocated buffer that must be large enough to hold the data. Also, remember that fread has no knowledge of strings and will not null-terminate anything for you. In fact, it is able to read zero bytes just fine and they can be part of the data. That's kind of the point. It can read binary data, not just text. The upshot is that you can't loop through the bytes of the buffer and terminate the loop with a zero byte since there may not be one where you need it.

    And the code you show makes no sense anyway since you don't have any fwrites in it at all.

    Also, you don't necessarily need a separate "output" buffer. You can read into a buffer, process the data in the buffer, and then write out from that same buffer.
    Code:
        char buf[1000];
        size_t n;
        while ((n = fread(buf, 1, sizeof buf, fin)) > 0) {
    
            // process buf ...
    
            if (fwrite(buf, 1, n, fout) != n) {
                perror("fwrite");
                exit(1);
            }
        }

  8. #8
    Registered User
    Join Date
    Nov 2016
    Posts
    5
    Quote Originally Posted by algorism View Post
    You can't tell fread to read twice as much as your buffer can hold! Why would you do that? That leads to undefined behavior and then anything can happen. You can ask fread to read LESS than the buffer size, but not more. Usually the actual buffer's size is used. (I should have emphasized last post that asking for billions of bytes is a strange request in the first place.)

    You don't seem to understand fread at all. It reads up to the requested number of elements (of the given size) into the pre-allocated buffer that must be large enough to hold the data. Also, remember that fread has no knowledge of strings and will not null-terminate anything for you. In fact, it is able to read zero bytes just fine and they can be part of the data. That's kind of the point. It can read binary data, not just text. The upshot is that you can't loop through the bytes of the buffer and terminate the loop with a zero byte since there may not be one where you need it.

    And the code you show makes no sense anyway since you don't have any fwrites in it at all.

    Also, you don't necessarily need a separate "output" buffer. You can read into a buffer, process the data in the buffer, and then write out from that same buffer.
    Code:
        char buf[1000];
        size_t n;
        while ((n = fread(buf, 1, sizeof buf, fin)) > 0) {
    
            // process buf ...
    
            if (fwrite(buf, 1, n, fout) != n) {
                perror("fwrite");
                exit(1);
            }
        }
    Very appreciated :P

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. help with fwrite,fread
    By gpadiernos in forum C Programming
    Replies: 5
    Last Post: 05-16-2010, 10:51 PM
  2. fread, fwrite
    By thescratchy in forum C Programming
    Replies: 2
    Last Post: 03-17-2010, 09:01 AM
  3. PLZ help in using fread,fwrite ..
    By jack_carver in forum C Programming
    Replies: 2
    Last Post: 01-20-2009, 01:20 AM
  4. fwrite() and fread()
    By Lau in forum C Programming
    Replies: 6
    Last Post: 12-05-2002, 10:18 AM
  5. fwrite - fread
    By PutoAmo in forum C Programming
    Replies: 2
    Last Post: 03-10-2002, 01:48 PM

Tags for this Thread