Thread: How does buffered output work exactly?

  1. #1
    Registered User
    Join Date
    Mar 2023
    Posts
    33

    How does buffered output work exactly?

    You would think that programs execute gradually, line by line. However, with printf (and perhaps other functions available), it isn't so simple.

    For example, i have this simple program for testing this:

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    
    int main ()
    {
        printf("\n Hello");
        printf("\n This is a test.");
        sleep (1);
        return 0;
    }
    The solution to this issue is here:

    c - Why is the line following printf(), a call to sleep(), executed before anything is printed? - Stack Overflow

    If fflush isn't in there, then what happens is that "Hello" prints to the screen, sleep is called, then the program prints "This is a test" before control is returned to me through the terminal.

    What I'm still curious is how exactly the printf's buffered output works to make it execute like that. I completely understand how to use fflush to make everything execute properly (putting it directly after the printf lines flushes all the text to stdout), but I am very curious how the stack is ordered in this miniature program to make it work like this in sudo code:

    Code:
    print hello
    sleep one second
    print this is a test

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,669
    Keep reading the SO answers until you get to the bit about \n.

    Where you put them in the string (or not at all) affects the results you see.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Feb 2019
    Posts
    1,077
    FILE type is an opaque type which keeps a buffer inside, used to streaming data. Some streams are "line buffered", like stdout, which means they flush the contents of this buffer if it is full OR if a '\n' is found. When you do:
    Code:
    printf( "\nHello" );
    printf() will add every single char to the stdout stream, but when a '\n' is writen, the entire buffer is flushed.
    Code:
    .+---------- flushes the buffer and writes '\n'.
     |   +------ just add those to the buffer, not flushing.
     V VVVVV
    "\nHello"
    The "Hello" part is added to the buffer inside stdout, and is kept there.

    The next printf: printf( " world\n" );, adds " world" to the buffer, but when '\n' is found, the entire buffer is flushed and '\n' is writen.

    That's why is advisable to call fflush() when there's no '\n' at the end of the string if you want it immediatly printed (or writen to file).

    Notice that stdout is line buffered, but stderr isn't. This is controlled by setvbuf() function:
    Code:
    int setvbuf( FILE *stream, char *buf, int mode, size_t size );
    With this function you can setup your own buffer and tell if the stream is line buffered or not. Like that:
    Code:
    static char buffer[32768]; // 32 KiB
    
    setvbuf( stdout, buffer, _IOLBF, sizeof buffer );
    Now we have stdout with a big buffer and it is line buffered (_IOLBF) -- The default buffer size is BUFSIZ (in my system, 8 KiB). You can change only the buffering scheme with:
    Code:
    // After that, stdout isn't line buffered anymore and the original buffer is kept.
    setvbuf( stdout, NULL, _IONBF, 0 );
    There's a _IOFBF too, which means the buffer is flushed only when it is full.

    PS: fopen() always create an object to a line buffered stream if "t" is informed in the mode or a fully buffered if "b".

    PS2: This buffering scheme is useful to keep functions dealing with streams fast. write(), which deals with file descriptors, isn't buffered and can block the thread until all data is writen.

  4. #4
    Registered User
    Join Date
    Feb 2019
    Posts
    1,077
    FILE type is an opaque type which keeps a buffer inside, used to streaming data. Some streams are "line buffered", like stdout, which means they flush the contents of this buffer if it is full OR if a '\n' is found. When you do:
    Code:
    printf( "\nHello" );
    printf() will add every single char to the stdout stream, but when a '\n' is writen, the entire buffer is flushed.
    Code:
     +---------- flushes the buffer and writes '\n'.
     |   +------ just add those to the buffer, not flushing.
     V VVVVV
    "\nHello"
    The "Hello" part is added to the buffer inside stdout, and is kept there.

    The next printf: printf( " world\n" );, adds " world" to the buffer, but when '\n' is found, the entire buffer is flushed and '\n' is writen.

    That's why is advisable to call fflush() when there's no '\n' at the end of the string if you want it immediatly printed (or writen to file).

    Notice that stdout is line buffered, but stderr isn't. This is controlled by setvbuf() function:
    Code:
    int setvbuf( FILE *stream, char *buf, int mode, size_t size );
    With this function you can setup your own buffer and tell if the stream is line buffered or not. Like that:
    Code:
    static char buffer[32768]; // 32 KiB
    
    setvbuf( stdout, buffer, _IOLBF, sizeof buffer );
    Now we have stdout with a big buffer and it is line buffered (_IOLBF) -- The default buffer size is BUFSIZ (in my system, 8 KiB). You can change only the buffering scheme with:
    Code:
    // After that, stdout isn't line buffered, but fully buffered,
    // and the original buffer is kept.
    setvbuf( stdout, NULL, _IOFBF, 0 );
    "Fully buffered" means the buffer will be flushed when it is full. There's a _IONBF too, which means no buffering (all chars are flushed immediately).

    PS: fopen() always create an object to a line buffered stream if "t" is informed in the mode or a fully buffered if "b".

    PS2: This buffering scheme is useful to keep functions dealing with streams fast. write(), which deals with file descriptors, isn't buffered and can block the thread until all data is writen.

    PS3: If you use an "external" buffer, make sure it still exists after program termination... This is INVALID:
    Code:
    int main( void )
    {
      char buffer[BUFSIZ];
    
      setvbuf( stdout, buffer, _IOLBF, sizeof buffer );
      printf( "ok\n" );
    
      // ops... buffer is local to main(). If will be allocated in the
      // stack and don't exist anymore after main returns!
    }
    That's why I made the buffer static (in the example before this one).
    Last edited by flp1969; 05-13-2023 at 08:35 AM.

  5. #5
    Registered User
    Join Date
    Feb 2019
    Posts
    1,077
    Sorry for the double posting (made a mistake editing the original here). Read #4 (#3 is out-to-date).
    Last edited by flp1969; 05-13-2023 at 08:26 AM.

  6. #6
    Registered User
    Join Date
    May 2012
    Location
    Arizona, USA
    Posts
    955
    Quote Originally Posted by flp1969 View Post
    PS: fopen() always create an object to a line buffered stream if "t" is informed in the mode or a fully buffered if "b".
    Perhaps in some oddball systems. Standard C says any string besides "r"/"w"/"a"/"r+"/"w+"/"a+", optionally with a "b" flag (and in C11 or later, optionally with an "x" flag), gives undefined behavior. The default (per the C standards) is to open a file in "text" mode anyway, so the "t" flag (meaning "text") seems pretty useless, IMO.

  7. #7
    Registered User
    Join Date
    Feb 2019
    Posts
    1,077
    Yep... my mistake... using 't' is wrong!

    One thing, thou: C11+ also defines "wx", "wbx", "w+x" and "w+bx".

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Does not work input or output structure to or from a file
    By Rassomaxa in forum C Programming
    Replies: 2
    Last Post: 04-18-2016, 07:37 AM
  2. exec and buffered output
    By optimus in forum C Programming
    Replies: 5
    Last Post: 04-26-2004, 03:25 AM
  3. Buffered vs Unbuffered
    By PutoAmo in forum C Programming
    Replies: 4
    Last Post: 10-24-2002, 08:54 PM
  4. why buffered?
    By Unregistered in forum C Programming
    Replies: 3
    Last Post: 08-21-2002, 11:35 AM
  5. Buffered input
    By CeeCee in forum C Programming
    Replies: 12
    Last Post: 11-25-2001, 03:15 AM

Tags for this Thread