Manipulating and Accessing Buffers in C

Consider, for instance, a simple C program as follows:

Code:

---

#include <stdio.h>
#include <stdlib.h>


int main()
{
    char arr[20];
    gets(arr);
    exit(EXIT_SUCCESS);
}

---

Ignore the deployment of scanf() for a moment. Nevertheless, the user enters the string and presses enter, which corresponds to the ASCII newline character (\n). Now, pray explain where the user-input gets stored initially. Is the string initially stored in a buffer and then passed to the character array "arr"? If so, it is passes serially or passed at once? If the aforementioned isn't applicable, is the string directly copied into the character array serially?

Moreover, on a tangential note, what precisely is a buffer in C? Where can I find resources apropos the functionality, description, and management of this specific buffer? Countless instances of my programs that invoke the scanf() and fgets() functions malfunction owing to a lingering newline character in the buffer. What does this imply? I was given to understand, much to my chagrin, that the program directly stores the data in the character array. Is this not the case? How does one manipulate and access the buffer?

Quote:

---

Originally Posted by Vercingetorix

Consider, for instance, a simple C program as follows:

Code:

---

#include <stdio.h>
#include <stdlib.h>


int main()
{
    char arr[20];
    gets(arr);
    exit(EXIT_SUCCESS);
}

---

Ignore the deployment of scanf() for a moment. Nevertheless, the user enters the string and presses enter, which corresponds to the ASCII newline character (\n). Now, pray explain where the user-input gets stored initially. Is the string initially stored in a buffer and then passed to the character array "arr"? If so, it is passes serially or passed at once? If the aforementioned isn't applicable, is the string directly copied into the character array serially?


Moreover, on a tangential note, what precisely is a buffer in C? Where can I find resources apropos the functionality, description, and management of this specific buffer? Countless instances of my programs that invoke the scanf() and fgets() functions malfunction owing to a lingering newline character in the buffer. What does this imply? I was given to understand, much to my chagrin, that the program directly stores the data in the character array. Is this not the case? How does one manipulate and access the buffer?

---

First of all, you should NEVER use gets()! Use fgets() instead!

gets() was depreciated in the C99 Standard, and removed completely form the C11 Standard. If you are using an older compiler that still allows the use of gets(), please install and use a newer compiler!

fgets(), scanf(), etc... are "Buffered" I/O. These functions will allocate some memory from the heap to be used internally. These functions will read a large chunk of data from standard input or a specified file into that internal buffer, and then store it into the variable defined in your code. You don't need to worry about where that internal "buffer" exists, or how the data is initially read, as you don't have direct access to it. Just use the fgets() or other function. BTW, if fgets() inputs a full line, the newline will be placed in the array along with the line. The internal details may be different for different Operating Systems.

A user defined "buffer" is some area of memory, a global or local array defined by the user, or can be some memory that was allocated dynamically from the heap.

Text Input from the user or a file, is line buffered. The function used waits until the newline has been pressed, then the data is received by your code.

A good up to date book on the C Programming language would teach you about this and other topics. Bottom line, don't get caught up in the internal details of Standard Library functions, just use them.

Check out the fgets() manpage online, along with the other functions of the Standard Library.

Quote:

---

Originally Posted by rstanley

First of all, you should NEVER use gets()! Use fgets() instead!

gets() was depreciated in the C99 Standard, and removed completely form the C11 Standard. If you are using an older compiler that still allows the use of gets(), please install and use a newer compiler!

fgets(), scanf(), etc... are "Buffered" I/O. These functions will allocate some memory from the heap to be used internally. These functions will read a large chunk of data from standard input or a specified file into that internal buffer, and then store it into the variable defined in your code. You don't need to worry about where that internal "buffer" exists, or how the data is initially read, as you don't have direct access to it. Just use the fgets() or other function. BTW, if fgets() inputs a full line, the newline will be placed in the array along with the line. The internal details may be different for different Operating Systems.

A user defined "buffer" is some area of memory, a global or local array defined by the user, or can be some memory that was allocated dynamically from the heap.

Text Input from the user or a file, is line buffered. The function used waits until the newline has been pressed, then the data is received by your code.

A good up to date book on the C Programming language would teach you about this and other topics. Bottom line, don't get caught up in the internal details of Standard Library functions, just use them.

Check out the fgets() manpage online, along with the other functions of the Standard Library.

---

There's lot to unpack here.

In what respect does fgets() differ from gets()?

Secondly, apropos the allocation of memory from the heap for the implementation of the buffer, what is the size of the said buffer? I ask this since I intend to create functions which input acutely voluminous tracts of texts through stdin for the sake of string manipulation and analysis. Ergo, it is vital that I know what the size of the stdin buffer is, lest it leads to complications.

Moreover, is "buffered I/O" equivalent to "line buffered"? If not, what precisely is it akin to? If I recall correctly, there are three types of buffers, eh?

And the precise reason why I intend to dwell over the functionality of buffers is on the account you've mentioned. For instance, I intend to input a sentence, but I wish to delete the newline before it gets attached to a string, so to speak, which is why it is necessary that I grasp how buffers function and how the manipulation of the said works.

Moreover, I've heard the term "flush" being used quite often with respect to buffers? What does that entail?

> In what respect does fgets() differ from gets()?
fgets won't overrun the end of the buffer you supply, so long as you're honest about the size of the buffer.

> what is the size of the said buffer?
Whatever you want it to be.

Typically, you do this.

Code:

---

char buff[BUFSIZ];  // defined in stdio, with a min value of 256 (it's 4K on my machine)
while ( fgets(buff,BUFSIZ,stdin) ) {
  // do stuff
}

---

> I intend to create functions which input acutely voluminous tracts of texts through stdin
So your "do stuff" may include things like malloc and strcpy.
It depends whether you can analyse a line there and then, or whether you need to read some/all of the file before you can proceed.

If really long lines are possible, and that's important to you, your first step is to see if buff contains a \n.
If it doesn't, then you know you have a long line.

> For instance, I intend to input a sentence, but I wish to delete
> the newline before it gets attached to a string
Yes, fgets() will store the newline if there is room in the buffer to store it.

gets() reads in a line, replacing the newline with the terminating Nul byte, and places it in the char array passed, without ANY error checking if the array is large enough to hold it! gets() will possibly overwrite space beyond the end of the array, crashing the program if you are lucky! For that reason, it has been REMOVED from the Standard Library, and fgets() has been recommended as the replacement!

fgets() will also attempt to read in a string, and place it in the array provided, up to one char short of the length of the array, which should be passed as the second argument to fgets(), allowing room for the terminating Nul byte! If the string is longer that the size passed minus one, the partial string will be terminated, and multiple reads will be needed to input the entire line. When the full string, or remaining portion of a long string has been read in, the newline will be at the end of the string in the array. You will need to remove the newline yourself.

Please study the manpage for fgets()!!!

You DO NOT need to know the size of the "buffer" internal to fgets()!!! You only need to worry about the the size of the array you pass to fgets()!!! fgets() will allocate and handle the use of the "internal buffer"!!!!!!!! As I said above, if the array YOU create and pass to fgets(), is shorter that the length of the string input, you will need multiple calls to fgets() to read in the entire length of the string. The size of the "Internal buffer" is COMPLETELY IRRELEVANT to this process!!! Programmers since C was invented in 1978 have been using fgets() without needing to worry about the internals of fgets()!!!

fgets(), fread(), etc... are "Buffered I/O" as apposed to read, etc... whitch is not buffered.

Line buffered means that fgets(), getchar(), etc... wait until the newline has been pressed or input, before reading in the data.

fflush() is a function to force the output of printf(), fprinf(), and other OUTPUT functions, out to the screen or disk file. fflush() should NEVER be used with an INPUT buffer!!! There are methods to "flush" the input buffer if needed.

Vercingetorix:

Short of taking a course in C Programming from a qualified instructor, you need to study a good book on the C Programming Language, cover to cover, and do all the exercises at the end of each chapter! Choose one of the three listed below:

C Programming, A Modern Approach
Author: K. N. King

C Primer Plus, 6th Edition
Stephen Prata

C How to Program, 8/e
Deitel & Deitel

This will instruct you on the topic of file I/O and all other topics you need to program in C.

Quote:

---

Originally Posted by Salem

> In what respect does fgets() differ from gets()?
fgets won't overrun the end of the buffer you supply, so long as you're honest about the size of the buffer.

> what is the size of the said buffer?
Whatever you want it to be.

Typically, you do this.

Code:

---

char buff[BUFSIZ];  // defined in stdio, with a min value of 256 (it's 4K on my machine)
while ( fgets(buff,BUFSIZ,stdin) ) {
  // do stuff
}

---

> I intend to create functions which input acutely voluminous tracts of texts through stdin
So your "do stuff" may include things like malloc and strcpy.
It depends whether you can analyse a line there and then, or whether you need to read some/all of the file before you can proceed.

If really long lines are possible, and that's important to you, your first step is to see if buff contains a \n.
If it doesn't, then you know you have a long line.

> For instance, I intend to input a sentence, but I wish to delete
> the newline before it gets attached to a string
Yes, fgets() will store the newline if there is room in the buffer to store it.

---

>fgets won't overrun the end of the buffer you supply, so long as you're honest about the size of the buffer.

In this context, the "buffer" you're referring to is the array wherein the input shall be stored, isn't it?

>Whatever you want it to be.

How does one define the buffer then? Secondly, why is it that before the input is passed to the character array, it is initially stored in environment-defined stdin buffer?

Quote:

---

Originally Posted by rstanley

gets() reads in a line, replacing the newline with the terminating Nul byte, and places it in the char array passed, without ANY error checking if the array is large enough to hold it! gets() will possibly overwrite space beyond the end of the array, crashing the program if you are lucky! For that reason, it has been REMOVED from the Standard Library, and fgets() has been recommended as the replacement!

fgets() will also attempt to read in a string, and place it in the array provided, up to one char short of the length of the array, which should be passed as the second argument to fgets(), allowing room for the terminating Nul byte! If the string is longer that the size passed minus one, the partial string will be terminated, and multiple reads will be needed to input the entire line. When the full string, or remaining portion of a long string has been read in, the newline will be at the end of the string in the array. You will need to remove the newline yourself.

Please study the manpage for fgets()!!!

You DO NOT need to know the size of the "buffer" internal to fgets()!!! You only need to worry about the the size of the array you pass to fgets()!!! fgets() will allocate and handle the use of the "internal buffer"!!!!!!!! As I said above, if the array YOU create and pass to fgets(), is shorter that the length of the string input, you will need multiple calls to fgets() to read in the entire length of the string. The size of the "Internal buffer" is COMPLETELY IRRELEVANT to this process!!! Programmers since C was invented in 1978 have been using fgets() without needing to worry about the internals of fgets()!!!

fgets(), fread(), etc... are "Buffered I/O" as apposed to read, etc... whitch is not buffered.

Line buffered means that fgets(), getchar(), etc... wait until the newline has been pressed or input, before reading in the data.

fflush() is a function to force the output of printf(), fprinf(), and other OUTPUT functions, out to the screen or disk file. fflush() should NEVER be used with an INPUT buffer!!! There are methods to "flush" the input buffer if needed.

---

>When the full string, or remaining portion of a long string has been read in, the newline will be at the end of the string in the array. You will need to remove the newline yourself.

Why will the newline be appended at the end of the string? Isn't there any mechanism to ensure that the fgets() function doesn't do this, short of writing a method to remove it myself?

>You DO NOT need to know the size of the "buffer" internal to fgets()!!!

For the sake of clarity, please edify me: what precisely is the relevance of the buffer to stdin? Why is it that my inputs are first passed to the buffer before they are assigned to their respective variables/character arrays? Moreover, what is the size of the said default stdin buffer? While I do understand that the degree of abstraction afforded by fgets() somewhat renders moot a meticulous examination of the internal buffer, I still wish to examine the functionality myself for the sake of clarity.

Moreover, what is the relevance of the setvbf() function? I am given to understand that one can define the buffering mode and buffer using this function? If, however, one doesn't invoke this function and inputs data through stdin to a character array using, say, the fgets() function, how is the buffer allocated to stdin and what is its size?

Furthermore, is buffering applicable in the scenario wherein I am sending output through stdout as well? Is it true that the outputted data will first pass through the stdout buffer (provided one exists) before it makes its way to stdout?

Lastly, what is the relevance of buffers? To a layman like me, it seems like a patently obvious illustration of memory wastage.

Quote:

---

Originally Posted by Vercingetorix

>Why will the newline be appended at the end of the string? Isn't there any mechanism to ensure that the fgets() function doesn't do this, short of writing a method to remove it myself?

---

That is the way Dennis Ritchie created the function. It cleans out the input buffer, for the next line to be input. Yes, you will need to deal with removing the newline if needed. No, there is any way of changing this short of finding the source and rewriting it.

Why are you so wrapped up in the theoretical understanding of the internals of Buffered I/O? Just use it!

The red string is my input:

Code:

---

#include <stdio.h>

#define DIM 256

int main(void)
{
  char ary[DIM] = "";    // Initialize all local variables

  fgets(ary, DIM, stdin); // Input a string from the user's keyboard

  printf("%s", ary);      // Output to the screen

  return 0;
}

/*  Run of program, and output:
$ ./foo
This is a test.
This is a test.
*/

---

>Why will the newline be appended at the end of the string? Isn't there any mechanism to ensure that the fgets() function doesn't do this, short of writing a method to remove it myself?

Salem mentioned one reason: long lines.

If the buffer you give to fgets() is not big enough to hold the entire line, there will not be a newline at the end, and the application can call fgets() again to get the rest of the line (or do whatever the application wants to do in this case--abort, discard the line, etc.).

If fgets() automatically removed the newline, then the application wouldn't have the option to handle this situation. (Note: gets() automatically removes the newline character because it always reads an entire line up to the newline character, even if the text overflows the user-provided buffer and overwrites something else that it shouldn't, which is why it's been removed from the latest C standards.)