C - Stdin Buffer

I use a linux system with glibc.

In general terms this is how I believed buffering worked in regards to stdin.

Working in a terminal emulator my keyboard is stdin and the characters I type are placed directly into stdin's buffer and once I hit <enter> a newline was generated and that was the cue for whichever input function I called in my program to start reading from stdin's buffer until some condition specified in my program was met. So, essentially stdin's buffer was line buffered. Using functions like getchar or fgets seemed to back up my thinking.

However, today I used fread on stdin for the first time and found this was not the case, at least not in this situation.

The man page states that it reads nmemb items of data. So, excluding EOF or an error, fread won't begin reading from stdin's buffer until I have entered nmemb or more items and the newline character. This is exactly the behaviour I observed and it seems to contradict my previous understanding.

From the glibc manual:

Code:

---

12.20.1 Buffering Concepts

  There are three different kinds of buffering strategies:


 

• Characters written to or read from an unbuffered stream are transmitted individually to or from the file as soon as possible.
• Characters written to a line buffered stream are transmitted to the file in blocks when a newline character is encountered.
• Characters written to or read from a fully buffered stream are transmitted to or from the file in blocks of arbitrary size.

 
Newly opened streams are normally fully buffered, with one exception: a stream connected to an interactive device such as a terminal is initially line buffered....

---

Using fread, stdin's buffer isn't unbuffered, or entirely line buffered, so it must be fully buffered.

So, arbitrary size, in my example, would be nmemb and only once that amount has been reached (with the exception of EOF or an error) will a new line flush stdin's buffer. This implies that stdin's buffer is aware of nmemb.

Am I on the right track? And if so how and when does stdin's buffer become aware of fread's nmemb?

Also I'm a little confused by the three different kinds of buffering strategies. Bullet points 1 and 3 say "Characters written to or read from..." but the 2nd bullet point only mentions writing to a line buffered stream. Does this mean you can't read from a line buffered stream? Using say fgets for an example, isn't stdin line buffered in that example since fgets starts reading once a newline is generated?

I'm still a beginner in C and programming so my apologizes in advance for incorrect terminology or if the answer to my questions are obvious.

Thank you for your time.

Quote:

---

Originally Posted by skynet99

The man page states that it reads nmemb items of data. So, excluding EOF or an error, fread won't begin reading from stdin's buffer until I have entered nmemb or more items and the newline character. This is exactly the behaviour I observed and it seems to contradict my previous understanding.

---

fread has its own buffer, i.e., the memory of the objects that you are trying to read into. So, fread doesn't have to read everything at once, and so it doesn't have to know beforehand how many characters are waiting to be consumed from the input stream. It just has to read what it can until its buffer is full, or EOF or an error is reached. If the input stream that it is reading from is buffered (whether line buffered or fully buffered), then that means it may have to wait (possibly multiple times) until that input buffer is flushed.

As an analogy, try writing a program to read 20 chars from stdin into a string by calling getchar in a loop until either 20 chars have been read or EOF is reached, then null terminate the string and print it. Enter a line of 10 chars. You will find that the string is not yet printed. Does this mean that stdin knows that 20 chars should be entered, hence stdin is fully buffered? No, it is because of your loop condition.

Thanks for your reply laserlight!

Quote:

---

fread has its own buffer, i.e., the memory of the objects that you are trying to read into. So, fread doesn't have to read everything at once, and so it doesn't have to know beforehand how many characters are waiting to be consumed from the input stream. It just has to read what it can until its buffer is full, or EOF or an error is reached. If the input stream that it is reading from is buffered (whether line buffered or fully buffered), then that means it may have to wait (possibly multiple times) until that input buffer is flushed.

---

I'm still confused. Does this mean that characters will be passed from stdin's buffer to fread's buffer then fread will read from its own buffer?

Quote:

---

As an analogy, try writing a program to read 20 chars from stdin into a string by calling getchar in a loop until either 20 chars have been read or EOF is reached, then null terminate the string and print it. Enter a line of 10 chars. You will find that the string is not yet printed. Does this mean that stdin knows that 20 chars should be entered, hence stdin is fully buffered? No, it is because of your loop condition.

---

I completely understand. I'll post a small program to help show what's tripping me up.

Code:

---

#include <stdio.h>

#define SIZE 10

int
main(void)
{
char buff[SIZE];
size_t bytes;

bytes = fread(buff, 1, SIZE, stdin);
fwrite(buff, 1, bytes, stdout);

putchar('\n');

return 0;
}

---

Control doesn't seem to be passed to fwrite to print the contents of buff until SIZE has been reached including a newline. So, if I type in 3 characters plus <enter> nothing happens. The program is still waiting for more input. Only when I enter 9 or more characters plus <enter> does the program continue on to print the contents of buff. This is the part I don't understand. If the characters I'm typing are first stored into stdin's buffer, how is it it knows to stop input at or after SIZE?

Quote:

---

Originally Posted by skynet99

Does this mean that characters will be passed from stdin's buffer to fread's buffer then fread will read from its own buffer?

---

No. fread reads from stdin into its own buffer. It is likely implemented using operating system specific API calls (and is meant to handle binary data, not just text), but you can imagine it being implemented with getchar (or rather, fgetc). My analogy of reading 20 chars with getchar is pretty much the rough equivalent of calling fread(some_array, 1, 20, stdin).

Quote:

---

Originally Posted by skynet99

If the characters I'm typing are first stored into stdin's buffer, how is it it knows to stop input at or after SIZE?

---

Let's suppose that stdin is unbuffered. When you enter those 3 characters and newline, all 4 of them are immediately read by fread and stored in fread's buffer. Since fread expects 10 characters, fread waits to read more characters from stdin. Hence, the program waits for more input.

Let's suppose that stdin is line buffered. When you enter those 3 characters and newline, because there's a terminating newline, all 4 of them are immediately read by fread and stored in fread's buffer. Since fread expects 10 characters, fread waits to read more characters from stdin. Hence, the program waits for more input.

Let's suppose that stdin is fully buffered, with a buffer size of 256. When you enter those 3 characters and newline, none them are read by fread. This time, it is stdin that's waiting for its own buffer to be filled (with another 252 characters) or otherwise flushed. Hence, the program waits for more input.

Quote:

---

Originally Posted by laserlight

No. fread reads from stdin into its own buffer. It is likely implemented using operating system specific API calls (and is meant to handle binary data, not just text), but you can imagine it being implemented with getchar (or rather, fgetc). My analogy of reading 20 chars with getchar is pretty much the rough equivalent of calling fread(some_array, 1, 20, stdin).

---

So, fread doesn't read from stdin's buffer but rather its own. Does this mean that every character that is entered is passed directly to fread's buffer one at a time?

Quote:

---

Let's suppose that stdin is unbuffered. When you enter those 3 characters and newline, all 4 of them are immediately read by fread and stored in fread's buffer. Since fread expects 10 characters, fread waits to read more characters from stdin. Hence, the program waits for more input.

Let's suppose that stdin is line buffered. When you enter those 3 characters and newline, because there's a terminating newline, all 4 of them are immediately read by fread and stored in fread's buffer. Since fread expects 10 characters, fread waits to read more characters from stdin. Hence, the program waits for more input.

Let's suppose that stdin is fully buffered, with a buffer size of 256. When you enter those 3 characters and newline, none them are read by fread. This time, it is stdin that's waiting for its own buffer to be filled (with another 252 characters) or otherwise flushed. Hence, the program waits for more input.

---

I understand every scenario you've listed here but since fread reads directly from stdin into its own buffer, the type of buffering stdin uses will not affect my program, is that correct?

Quote:

---

Originally Posted by skynet99

So, fread doesn't read from stdin's buffer but rather its own.

---

No, as you correctly wrote later, it reads from stdin into its own buffer.

Quote:

---

Originally Posted by skynet99

Does this mean that every character that is entered is passed directly to fread's buffer one at a time?

---

What exactly fread does is implementation defined by definition, but for efficiency it's likely to read in chunks of bytes rather than byte by byte.

Quote:

---

Originally Posted by skynet99

the type of buffering stdin uses will not affect my program, is that correct?

---

It could make a difference in terms of efficiency, and possibly even output, e.g., if you're reading in chunks and printing each chunk with a delimiter, you could end up with somewhat different output if the input if the input is line buffered versus fully buffered.

Quote:

---

Originally Posted by laserlight

It could make a difference in terms of efficiency, and possibly even output, e.g., if you're reading in chunks and printing each chunk with a delimiter, you could end up with somewhat different output if the input if the input is line buffered versus fully buffered.

---

This confuses me. If fread reads from stdin and not stdin's buffer then how would stdin's buffer type affect anything?

Thanks for your patience.

Quote:

---

Originally Posted by skynet99

If fread reads from stdin and not stdin's buffer then how would stdin's buffer type affect anything?

---

I didn't say fread doesn't read from stdin's buffer; I said that fread doesn't read from its own buffer. It's true that I carefully avoided using the phrase "stdin's buffer", and that's because conceptually fread reads from an input file stream (FILE*), not directly from an input buffer (basically a "holding area", so it would typically be a chunk of memory, e.g., an array of bytes). The notion of an input file stream having an input buffer is therefore implementation defined, i.e., it is not a strict requirement that an input file stream from which fread might read must have an input buffer, e.g., perhaps if the file stream is unbuffered it could do without one.

The buffering state of a stream is irrelevant to how a particular function operates with that stream.

fgets will read characters until the buffer is full, a \n is seen or end of file.
fread will read characters until the buffer is full or end of file.

Thank you for the replies. Things are clearer now. It's much appreciated.