Scanning new line

I have this function:

Code:

---

void prompt(Line *head) {
    char command[CMDLENGTH] = {};
    int numScanned = 0;
    int lineNumber = 1;
   
    printf("? ");
    numScanned = scanf("%s", command);
   
    while (strcmp(command, "q") != 0
            && strcmp(command, "x") != 0
            && numScanned != 0) {
            printf("command = %s\t", command);
       
        if (strcmp(command, "h") == 0) {
            helpCommand();
        } else if (strcmp(command, "p") == EQUAL
                    || strcmp(command, "+") == EQUAL
                    || strcmp(command, "-") == EQUAL
                    || strcmp(command, "\n") == EQUAL
                    || (stringIsNum(command) == YES
                    && stringToNum(command) > 0) ) {
            lineNumber = printCommand(head, command, lineNumber);
        } else {
            printf("Unknown command: ignoring\n");
        }
       
        printf("? ");
        numScanned = scanf("%s", command);
    }
   
}

---

And the way input works is by prompting the user with a question mark, then the user enters some character or number and pressed enter. I was first using getchar and then flushing out the new line character, but then I realized that the user can enter a multiple digit number as well, so I scrapped getchar and replaced it with scanf.

Now I have a problem with scanf. While I want the new line characters after another character / number is entered to be flushed out, the user is supposed to be able to send enter as well and I'm meant to treat that as I would with the + character.
The problem is that it seems like scanf doesn't read new line characters. So what can I do to get around this problem?

You may need to parse the input, possibly character by character, until the terminating characters are detected. Instead of using scanf, you would then parse the input yourself for the integers and other symbols.

Quote:

---

Originally Posted by laserlight

You may need to parse the input, possibly character by character, until the terminating characters are detected. Instead of using scanf, you would then parse the input yourself for the integers and other symbols.

---

In other words I should use getchar?

Use fgets() to read a whole line (ENTER included), then parse it.
You can parse it with strtol(), strtod(), sscanf() or character-by-character or a mixture.

Quote:

---

Originally Posted by Mentallic

In other words I should use getchar?

---

If you want to. The way that I might approach this is to tokenise the input stream of characters, then work on the stream of tokens (not necessarily as two distinct steps, e.g., once you have a token, you could process it instead of storing it for later processing). Reading character by character using getchar is one way to implement such an approach.

However, this approach is more for the case where the input is expected to be a continuous stream of tokens, but now that I look at your initial post again, you seem to be prompting token by token here.

Quote:

---

Originally Posted by qny

Use fgets() to read a whole line (ENTER included), then parse it.

---

As such, I think that the approach using fgets would be simpler. I was going to suggest this earlier, but it is tricky to handle if you want to parse many tokens at one go, unless you end up treating the string read as a stream of characters that you then parse for tokens. That said, there's a caveat: the newline character is not guaranteed to be in the input stored by fgets.

Thanks for the input guys. I think I'm going to give fgets and go, although I am curious as to why newline characters aren't guaranteed to be stored?

I don't know what parsing is, but I guess I could make a small function such as

Code:

---

char *removeNewLine(char *command)

---

so that it could remove the new line character at the end of the string, and I can carry on as I have been so far with scanf.

Oh and just an added question: Would sscanf work in this case, or would that just work in the same manner as scanf?

Quote:

---

... although I am curious as to why newline characters aren't guaranteed to be stored?

---

Code:

---

// example 1 - User input less than array size - newline gets stored:

char userString[10];

fgets(userString,10,stdin);

// user enters "Hello\n"

userString[0] = 'H'
userString[1] = 'e'
userString[2] = 'l'
userString[3] = 'l'
userString[4] = 'o'
userString[5] = '\n'
userString[6] = '\0'

---

Code:

---

// example 2 - User input greater than array size
// Protected from overflow, but newline does not get stored:

char userString[5];

fgets(userString,5,stdin);

// user enters "Goodbye\n"

userString[0] = 'G'
userString[1] = 'o'
userString[2] = 'o'
userString[3] = 'd'
userString[4] = '\0'

// no newline in array

---

Quote:

---

Would sscanf work in this case, or would that just work in the same manner as scanf?

---

You can use a simple loop to look for the newline and remove it if present.

If you use fgets(), you can only have one position for the newline: strlen() - 1. It can't be located anywhere else in the string.

Code:

---


//if line is the char array name:
int len = strlen(line)-1;
if(line[len]=='\n'
  line[len]='\0';

---

To remove the newline (LF or CR LF) at the end of the line, I recommend

Code:

---

line[strcspn(line, "\r\n")] = '\0';

---

If line is a pointer to the buffer (not the buffer itself), you can use

Code:

---

line += strspn(line, "\t\n\v\f\r ");

---

to skip over leading whitespace, too.

When doing both, I recommend

Code:

---

size_t len;

line += strspn(line, "\t\n\v\f\r ");
len = strcspn(line, "\r\n");
line[len] = '\0';

---

which also gives you the length of the remaining part in len.

Quote:

---

Originally Posted by Adak

Code:

---

if(line[len]=='\n'
  line[len]='\0';

---

---

and here you will happily modify data outside line when the input contains an embedded NUL byte ('\0') at the beginning or just after a newline. Ouch. It really is possible for fgets() to return a string which starts with a \0. To test, just try reading the output of command-line command

Code:

---

printf 'First line\n\0Bad line\nThird line\n'

---

Quote:

---

Originally Posted by Nominal Animal

To remove the newline (LF or CR LF) at the end of the line, I recommend

Code:

---

line[strcspn(line, "\r\n")] = '\0';

---

---

I don't recommend that because, as I noted earlier, fgets might not store the newline. Furthermore, since this is not binary input, we know that the newline will be mapped to '\n', regardless of the actual implementation defined newline sequence. Therefore, I recommend:

Code:

---

char *p = strchr(line, '\n');
if (p)
{
    *p = '\0';
}

---

Unless you need to get the string's length anyway, in which case checking the last character of the string (other than the null character) makes sense.

EDIT:
Oh, I just remembered: since strcspn's returns the string length if none of the search characters are found, you'll just end up assigning a null character to the position containing the terminating null character. Thus, in terms of correctness, Nominal Animal's suggestion is actually correct even in the face of no newline. However, the difference is that it does more than required, since strcspn is inherently meant to do more than strchr.

Quote:

---

It really is possible for fgets() to return a string which starts with a \0.

---

O_o

Well, that is true, but if the input expected has embedded nulls `fgets' is the wrong tool for the job.

Soma

Quote:

---

Originally Posted by laserlight

I don't recommend that because, as I noted earlier, fgets might not store the newline.

---

If line does not contain a CR or an LF, then strcspn(line, "\r\n") == strlen(line).

So, what's the problem? Edit: oh, okay: no problem as such, just a different preference.

Quote:

---

Originally Posted by laserlight

Code:

---

char *p = strchr(line, '\n');

---

---

strchr() is faster, because strspn() and strcspn() are not that efficient. I cannot think of a case where it would be measurable, though.

I suspect, but have not checked, that

Code:

---

char *p;
if ((p = strchr(line, '\r'))) *p = '\0';
if ((p = strchr(line, '\n'))) *p = '\0';

---

is faster than the equivalent

Code:

---

line[strcspn(line, "\r\n")] = '\0';

---

In practice, I believe the fastest is

Code:

---

size_t  len;

len = strlen(line);
while (len > 0 && (line[len] == '\n' || line[len] == '\r'))
    len--;
line[len] = '\0';

---

which additionally provides the length of the string left.

(To be precise, it does not work exactly like the above snippets, though. The former code snippets terminate the string at the first linefeed or carriage return, whereas this one only removes them at end of string. Since there should ever be at most one newline per line, the difference should be irrelevant.)

All of the above snippets are safe as long as line is not a null pointer.

Quote:

---

Originally Posted by Nominal Animal

If line does not contain a CR or an LF, then strcspn(line, "\r\n") == strlen(line).

So, what's the problem?

---

Refer to my edit.

Quote:

---

Originally Posted by phantomotap

Well, that is true, but if the input expected has embedded nulls `fgets' is the wrong tool for the job.

---

Embedded nulls, in my experience, are almost always user errors: reading the wrong file, or such. It does not matter that fgets() is the wrong tool to read such input.

I'm actually quite upset about people recommending a solution which may cause a buffer underrun bug. That's just not cool.

Quote:

---

It does not matter that fgets() is the wrong tool to read such input.

---

O_o

Of course it does.

If you are going to try to recover gracefully in the face of "user error" nothing shown so far is sufficient.

Quote:

---

I'm actually quite upset about people recommending a solution which may cause a buffer underrun.

---

O_o

[Edit]
Yes. This was just a joke so don't get in a twist.

I'm just pointing out the problem with your last bit of code.
[/Edit]

The `line[strlen(line)]' character will never be a newline character unless something goes horribly wrong.

[Edit]
*sigh*

Removed the wrong portion of the post.

I have no idea what was written here. Sorry.
[/Edit]

All joking aside, the code forwarded by you will not get rid of the newline in the "\0bad input\n" form of string so it is just as likely to cause a serious problem at a later point.

Quote:

---

That's just not cool.

---

I agree.

That's the only reason I posted.

I just wanted to make the original poster aware that trying to recover from an unexpected situation (like nulls in the input) requires an alternative strategy.

[Edit]
The reason being, you kind of imply that your code would sufficient to deal with embedded nulls in string input.
[/Edit]

Soma

Quote:

---

Originally Posted by phantomotap

If you are going to try to recover gracefully in the face of "user error" nothing shown so far is sufficient.

---

The suggestion by Adak causes a buffer underrun bug. It reads a byte just before the buffer, and if that byte equals 10, it replaces it with a zero.

My suggestion does not have that bug. I would also be happy with

Code:

---

int len = strlen(line);
if (len > 0 && line[len - 1] == '\n')
    line[--len] = '\0';

---

which, due to the short circuit evaluation rules in C, does not try to access the byte just before the buffer, either. And the len ends up with the correct value in all cases (even when the line was longer than the available buffer and was only partially read).

Quote:

---

Originally Posted by phantomotap

The `line[strlen(line)]' character will never be a newline character unless something goes horribly wrong.

---

Right. It will always be a NUL character. It is safe to replace it with another NUL character.

(.. assuming line != NULL. Which should be checked right after the fgets() call anyway.)

Quote:

---

Originally Posted by phantomotap

All joking aside, the code forwarded by you will not get rid of the newline in the "\0bad input\n" form of string so it is just as likely to cause a serious problem at a later point.

---

My code does not try to access the char just before the buffer, though.

Given that input, my code yields *line == '\0' (and strlen(line) == 0 ), i.e. and empty string, but nothing bad happens.

To be honest, I did first write the proper solution, using POSIX.1-2008 getline(), but I thought it would either scare the OP, or I'd just get some negative feedback about it, so I rewrote my reply before posting it.

Code:

---

char    *data = NULL;
size_t  size = 0;

char    *line;
size_t  len;
ssize_t full;

/* Some form of input loop .. */
while (1) {

    full = getline(&data, &size, file handle);
    if (full < (ssize_t)1)
        end of file or error, abort/break

    /* Input sanitization */
    if (!clean(data, (size_t)full, &line, &len))
        input is suspect, ignore/warn/abort

    /* Ignore empty lines (and comments, if cleaned) */
    if (len < 1)
        continue;

    /* Now have len chars at line. */

}

free(data);
data = NULL;
size = 0;

---

The input sanitization function,

Code:

---

int clean(char *const input, const size_t inputlen, char **const result, size_t *const resultlen);

---

takes the size bytes of input in input, removes unwanted characters or at least checks the string, saves the start of the contents and length of the contents in the last two pointers supplied by the user unless NULL, and returns 0 if the input was safe, and nonzero otherwise.

The implementation details are very domain-specific. You might want to simply remove all embedded NUL bytes and control characters, and replace consecutive whitespace with a single space; this works well with semi-interactive human input, say a game. In other cases you might wish to replace NULs and (some) other ASCII control characters with escape codes (for example, ASCII NULs with "\\0" or ("&" "#0;"), the numeric HTML entity reference for ASCII NUL character).

If there is interest, I'd be happy to show a couple of practical examples.

Quote:

---

Right. It will always be a NUL character. It is safe to replace it with another NUL character.

---

You misunderstand.

The line of code I was referring to was the bug in the `while' condition.

Code:

---

while (len > 0 && (line[len] == '\n' || line[len] == '\r'))

---

Quote:

---

My code does not try to access the char just before the buffer, though.

---

I didn't say it did.

I just said that what you were implying about your code being sufficient in the face of embedded nulls was wrong.

Soma

Quote:

---

Originally Posted by phantomotap

You misunderstand.

---

I did! :headpalm:

I apologize. It is a typo; the code should of course be

Code:

---

size_t  len;
 
len = strlen(line);
while (len > 0 && (line[len-1] == '\n' || line[len-1] == '\r'))
    len--;
line[len] = '\0';

---

Big thanks for noticing it.

Hmm.. I don't see how to edit the post. It might be good to fix the post, so nobody gets tripped by it.

Quote:

---

I don't see how to edit the post. It might be good to fix the post, so nobody gets tripped by it.

---

O_o

I know that the forum software prevents edits after a certain time.

I don't know what that time is, but it has probably passed.

*shrug*

I should have been more specific in the first pass.

Soma

Actually, I much prefer the simpler methods rather than using functions that haven't been mentioned in class.

Quote:

---

Originally Posted by Adak

If you use fgets(), you can only have one position for the newline: strlen() - 1. It can't be located anywhere else in the string.

Code:

---


//if line is the char array name:
int len = strlen(line)-1;
if(line[len]=='\n'
  line[len]='\0';

---

---

I don't know why I didn't think of this. I instead had

Code:

---

for (i=0; i<CMDLENGTH && command[i+1] != '\0'; i++);
        if (command[i] == '\n' && i != 0) {
            command[i] = '\0';
        }

---

But yours is much simpler. Thanks!

Oh and I've already set my string length limits to be much larger than user input should be, plus our professor specifically told us he doesn't want us spending unnecesssary amounts of time concocting crazy scenarios that could break our program and trying to defend against that.

Err... Adak's solution is wrong. If you want to stick to strlen, then it should be:

Code:

---

size_t len = strlen(line);
if (len > 0 && line[len - 1] == '\n')
{
    --len;
    line[len] = '\0';
}

---

I realized that, but I only needed the concept :)

Now that I've tried adding it to a function, I'm getting an error that has to do with arrays vs pointers:

Code:

---

char command[CMDLENGTH] = {};
        command = removeNewLine(command);

char *removeNewLine(char *command) {
    int length = strlen(command)-1;
   
    if(command[length] == '\n' && length != 0) {
        command[length]='\0';
    }
   
    return command;
}

---

Is giving me the error

Quote:

---

error: incompatible types when assigning to type ‘char[102]’ from type ‘char *’

---

It's been a while since I've used arrays, so I've clearly forgotten how to pass and return arrays as opposed to strings.

EDIT:

Quote:

---

Originally Posted by laserlight

Err... Adak's solution is wrong. If you want to stick to strlen, then it should be:

Code:

---

size_t len = strlen(line);
if (len > 0 && line[len - 1] == '\n')
{
    --len;
    line[len] = '\0';
}

---

---

The if statement should test len > 1 should'nt it? If len = 1 then the user input must've only been a new line, which I want to keep.

You cannot assign to an array. In this case, it will suffice to call:

Code:

---

removeNewLine(command);

---

By the way, I did not subtract the 1 immediately from the result of strlen in case strlen is 0. Then, I did the check for > 0 before I accessed line[len - 1]. Granted, if the line comes from fgets within the constraints of your classroom environment, you will not need to be this paranoid, but you might as well be.

Quote:

---

Originally Posted by laserlight

You cannot assign to an array. In this case, it will suffice to call:

Code:

---

removeNewLine(command);

---

---

Ahh thanks :)

Quote:

---

Originally Posted by laserlight

By the way, I did not subtract the 1 immediately from the result of strlen in case strlen is 0. Then, I did the check for > 0 before I accessed line[len - 1]. Granted, if the line comes from fgets within the constraints of your classroom environment, you will not need to be this paranoid, but you might as well be.

---

Before calling the function removeNewLine, I already check that the return value of fgets is not NULL, so then strlen can't be 0 from what I can see.

Quote:

---

Originally Posted by Mentallic

Before calling the function removeNewLine, I already check that the return value of fgets is not NULL, so then strlen can't be 0 from what I can see.

---

You could read Nominal Animal's post #9 for a weird situation where strlen can still return 0 after reading with fgets. However, that aside, consider: separate from the issue of fgets, someone reading your code would ask if the removeNewLine function works correctly on an empty string. With your implementation, it does not. So you should either document that fact (e.g., with a comment, or maybe even an assertion), or just handle empty strings gracefully.

int length = strlen(command)-1;
if(command[length] == '\n' && length != 0)

Forgive me for not posting this in the original context but this is still exhibiting out of bounds access. The undefined behavior can only be mitigated if the length != -1 check is performed first.

Order of execution always matters.

Quote:

---

Originally Posted by whiteflags

The undefined behavior can only be mitigated if the length != -1 check is performed first.

---

It is a little more complicated than that, methinks: strlen returns a size_t, which is an unsigned integer type, so if the return value is 0, then the subtraction would result in a large positive number. This is then converted to int, the conversion being implementation defined. Assuming that we do arrive at -1 in the end, the check that Mentallic wrote compares with 0, not -1, so it is wrong anyway.

Quote:

---

Originally Posted by laserlight

You could read Nominal Animal's post #9 for a weird situation where strlen can still return 0 after reading with fgets. However, that aside, consider: separate from the issue of fgets, someone reading your code would ask if the removeNewLine function works correctly on an empty string. With your implementation, it does not. So you should either document that fact (e.g., with a comment, or maybe even an assertion), or just handle empty strings gracefully.

---

You make a good point.

Quote:

---

Originally Posted by whiteflags

int length = strlen(command)-1;
if(command[length] == '\n' && length != 0)

Forgive me for not posting this in the original context but this is still exhibiting out of bounds access. The undefined behavior can only be mitigated if the length != -1 check is performed first.

Order of execution always matters.

---

I'll change my code to

Code:

---

int length = strlen(command)-1;

if (length > 0 && command[length] == '\n')

---

which looks like it'll handle NULL strings.

Quote:

---

Originally Posted by laserlight

It is a little more complicated than that, methinks: strlen returns a size_t, which is an unsigned integer type, so if the return value is 0, then the subtraction would result in a large positive number. This is then converted to int, the conversion being implementation defined. Assuming that we do arrive at -1 in the end, the check that Mentallic wrote compares with 0, not -1, so it is wrong anyway.

---

Unsigned ints do that? Oh boy...

Ok, I'm going to leave strlen as it is then.

Code:

---

    int length = strlen(command);
   
    if (length > 1 && command[length-1] == '\n') {
        command[length-1]='\0';
    }

---

Yep. Why not just make length a size_t? Also, note that if you are replacing the last character of a string with a null character, then you have effectively reduced the length by 1. Thus, if you want to use the length variable later on, then for correctness you should subtract 1 from it too, hence the decrement in my example in post #21. In your case, you are writing it in a function that will not use length afterwards, so it is arguably okay.

Quote:

---

Originally Posted by laserlight

Yep. Why not just make length a size_t?

---

I didn't know what size_t represented until now, and we haven't used it in class yet.

Quote:

---

Originally Posted by laserlight

Also, note that if you are replacing the last character of a string with a null character, then you have effectively reduced the length by 1. Thus, if you want to use the length variable later on, then for correctness you should subtract 1 from it too, hence the decrement in my example in post #21. In your case, you are writing it in a function that will not use length afterwards, so it is arguably okay.

---

Yes I understand that. This code is in a void function right now so I could have added the int return, but I haven't found a reason to need the string length of the user input.