Understanding strtok from string.h library

From my book I did the following example

Code:

---

#include <stdio.h>
#include <string.h>

int main()
{
  char string[] = "This is a sentence with 7 tokens";
  char *tokenPtr;

  printf( "%s\n%s\n\n%s\n",
          "The string to be tokenized is:", string,
          "The tokens are:" );

  tokenPtr = strtok( string, " " );

  while ( tokenPtr != NULL ) {
    printf( "%s\n", tokenPtr );
    tokenPtr = strtok( NULL, " " );
  }

  return 0;
}

---

What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?

Quote:

---

Originally Posted by yougene

From my book I did the following example
What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?

---

strtok() overwrites the delimiters it finds with null characters, effectively turning your original string into a number of smaller strings. That means your original string is modified; if you don't like that, you can make a copy of the string and work with it.

Quote:

---

Originally Posted by yougene

From my book I did the following example

Code:

---

#include <stdio.h>
#include <string.h>

int main()
{
  char string[] = "This is a sentence with 7 tokens";
  char *tokenPtr;

  printf( "%s\n%s\n\n%s\n",
          "The string to be tokenized is:", string,
          "The tokens are:" );

  tokenPtr = strtok( string, " " );

  while ( tokenPtr != NULL ) {
    printf( "%s\n", tokenPtr );
    tokenPtr = strtok( NULL, " " );
  }
  printf("%s\n", string);
  return 0;
}

---

What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?

---

Only if you forget how %s works: %s prints a C-style string, meaning it stops if and only if it encounters a \0 character. Since strtok replaces the end-of-token character with a \0, %s stops reading.

Note the line I added in red above -- add that to your file and see what happens.

Ahh ok. So every instance of " " is replaced with "\0"?

Yes.

Thanks guys

Note that this:

printf( "%s\n%s\n\n%s\n",
"The string to be tokenized is:", string,
"The tokens are:" );

Can be written as

printf( "The string to be tokenized is:\n%s\n\nThe tokens are:\n" string );

Which is much less confusing than what you have.