Thread: Understanding strtok from string.h library

  1. #1
    Registered User
    Join Date
    May 2006
    Posts
    182

    Understanding strtok from string.h library

    From my book I did the following example

    Code:
    #include <stdio.h>
    #include <string.h>
    
    int main()
    {
      char string[] = "This is a sentence with 7 tokens";
      char *tokenPtr;
    
      printf( "%s\n%s\n\n%s\n",
              "The string to be tokenized is:", string,
              "The tokens are:" );
    
      tokenPtr = strtok( string, " " );
    
      while ( tokenPtr != NULL ) {
        printf( "%s\n", tokenPtr );
        tokenPtr = strtok( NULL, " " );
      }
    
      return 0;
    }
    What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?

  2. #2
    Registered User
    Join Date
    Sep 2007
    Posts
    1,012
    Quote Originally Posted by yougene View Post
    From my book I did the following example
    What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?
    strtok() overwrites the delimiters it finds with null characters, effectively turning your original string into a number of smaller strings. That means your original string is modified; if you don't like that, you can make a copy of the string and work with it.

  3. #3
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    Quote Originally Posted by yougene View Post
    From my book I did the following example

    Code:
    #include <stdio.h>
    #include <string.h>
    
    int main()
    {
      char string[] = "This is a sentence with 7 tokens";
      char *tokenPtr;
    
      printf( "%s\n%s\n\n%s\n",
              "The string to be tokenized is:", string,
              "The tokens are:" );
    
      tokenPtr = strtok( string, " " );
    
      while ( tokenPtr != NULL ) {
        printf( "%s\n", tokenPtr );
        tokenPtr = strtok( NULL, " " );
      }
      printf("%s\n", string);
      return 0;
    }
    What I'm noticing is that the token isn't being saved to a new array. A pointer is just pointed at the existing array. If that is the case how come printf doesn't print everything after tokenPtr? You would expect %s to print the whole array no?
    Only if you forget how %s works: %s prints a C-style string, meaning it stops if and only if it encounters a \0 character. Since strtok replaces the end-of-token character with a \0, %s stops reading.

    Note the line I added in red above -- add that to your file and see what happens.

  4. #4
    Registered User
    Join Date
    May 2006
    Posts
    182
    Ahh ok. So every instance of " " is replaced with "\0"?

  5. #5
    Registered User hk_mp5kpdw's Avatar
    Join Date
    Jan 2002
    Location
    Northern Virginia/Washington DC Metropolitan Area
    Posts
    3,817
    Yes.
    "Owners of dogs will have noticed that, if you provide them with food and water and shelter and affection, they will think you are god. Whereas owners of cats are compelled to realize that, if you provide them with food and water and shelter and affection, they draw the conclusion that they are gods."
    -Christopher Hitchens

  6. #6
    Registered User
    Join Date
    May 2006
    Posts
    182
    Thanks guys

  7. #7
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Note that this:

    printf( "&#37;s\n%s\n\n%s\n",
    "The string to be tokenized is:", string,
    "The tokens are:" );

    Can be written as

    printf( "The string to be tokenized is:\n%s\n\nThe tokens are:\n" string );

    Which is much less confusing than what you have.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. 20q game problems
    By Nexus-ZERO in forum C Programming
    Replies: 24
    Last Post: 12-17-2008, 05:48 PM
  2. Library Wrapper
    By cusavior in forum C Programming
    Replies: 3
    Last Post: 03-25-2008, 10:27 AM
  3. Difficulty choosing graphics library
    By jdiperla in forum Game Programming
    Replies: 11
    Last Post: 02-27-2008, 06:35 PM
  4. link with C runtime library
    By George2 in forum C++ Programming
    Replies: 26
    Last Post: 02-05-2008, 01:56 AM
  5. Global variable in shared library
    By krock923 in forum C Programming
    Replies: 5
    Last Post: 01-11-2008, 04:56 PM