Thread: reset strtok()

  1. #1
    Registered User
    Join Date
    Aug 2012
    Posts
    9

    reset strtok()

    Hallo!
    How can I reset strtoc()?
    After I have caled it twice "strtok(string, " ") ",
    it works not fine "strtok(NULL, " ")".
    Is there a solution?
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    
    int main(void)
    {
        char string[] = "aa bbb cc aa dd gg";
        char *ptr = NULL;
        
        ptr = strtok(string, " ");
        /* do something */
        ptr = strtok(string, " ");
        while (ptr) {
            puts(ptr);
            ptr = strtok(NULL, " ");
        }
        return 0;
    }

  2. #2
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    strtok is destructive, so the solution is to copy the string, then call strtok with the copy. Thus, if you need to call strtok to process the string again, you still have the original to work with.

    Actually, why do you need to do this?
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  3. #3
    Registered User
    Join Date
    Aug 2012
    Posts
    9
    Quote Originally Posted by laserlight View Post
    Actually, why do you need to do this?
    I have to find out, whether some word is contained more than one time in the string.

  4. #4
    Registered User
    Join Date
    Sep 2012
    Posts
    357
    Hint: there's a function in the C Standard library named strstr() (prototype in <string.h>) which may be of help.

  5. #5
    Registered User
    Join Date
    Aug 2012
    Posts
    9
    Quote Originally Posted by laserlight View Post
    the solution is to copy the string, then call strtok with the copy.
    What is wrong in my code?
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    
    int main(void)
    {
        char *pos1, *pos2;
        char string[256];
        
        printf("Enter a string: ");
        fgets(string, 255, stdin);
        
        pos1 = string;
        char copy[strlen(string)+1];
        strcpy(copy, string);
        pos1 = strtok( string, " ,");    
        while (pos1) {
            pos2 = strtok( copy, " ,");
            while (pos2) {
    printf("pos2 %s\n", pos2); //for debug
                if(!strcmp(pos1, pos2) && (string-pos1)<(copy-pos2))
                    puts(pos2);
                pos2 = strtok( NULL, " ,");
            }
    printf("pos1 %s\n", pos1); //for debug
        pos1 = strtok( NULL, " ,");
        }
    
        return 0;
    }

  6. #6
    Registered User
    Join Date
    Sep 2012
    Posts
    357
    Write your own version of strtok().
    Then you'll understand what you are doing wrong.

    after the initial strtok() with a non-NULL 1st parameter you have to call strtok() with a NULL for the 1st parameter until all the string is processed.
    You cannot "initialize" strtok() in the middle of another strtok() run.
    Last edited by qny; 10-02-2012 at 09:10 AM. Reason: some explanation

  7. #7
    Registered User
    Join Date
    May 2012
    Posts
    1,066
    Quote Originally Posted by SashaN View Post
    What is wrong in my code?
    Code:
    pos1 = strtok( string, " ,");    
    while (pos1) {
        pos2 = strtok( copy, " ,");
    You can't use strtok one two different strings simultaneously. The moment you use strtok() on "copy", tokenizing "string" doesn't work any more. The reason is that strtok() saves internally the state of the current string processing, thus if you provide a new string, this information is overwritten.
    If you really want to tokenize two strings you could use the non-standard strtok_r (e.g. supported by the GNU C Library) or strtok_s (new in C11).

    Bye, Andreas

  8. #8
    Registered User
    Join Date
    Aug 2012
    Posts
    9
    Thank you for your help.
    strtok() works now as I will, although the program is not ready yet.
    Here the code.
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    
    void get_string(char *string, int number);
    
    int main(void)
    {
      char *pos1, *pos2;
      char *str1, *str2;
      char string[256];
      int counter = 0;
      int i;
    
      printf("Enter a string: ");
      get_string(string, 255);
      pos1 = string;
    
      char copy[strlen(string)+1];
    
      while (pos1) {
        str1 = malloc(strlen(string)+1);
        strcpy(str1, string);
        pos1 = strtok( str1, " ,");
        i = 0;
        while (i < counter) {
          pos1 = strtok( NULL, " ,");
          i++;
        }
        counter++;
        if(pos1) {
          str2 = malloc(strlen(string)+1);
          strcpy(str2, string);
          pos2 = strtok( str2, " ,");
          while (pos2) {
            if (!strcmp(pos1, pos2)) {
              if ((pos1-str1)>(pos2-str2))
                puts(pos1);
            }
            pos2 = strtok( NULL, " ,");
          }
          free(str2);
       }
       free(str1);
      }
      return 0;
    }
    
    void get_string(char *string, int number)
    {
      char ch;
      char *ptr;
      ptr = string;
      ch = getchar();
      while (ch!='\n' && (ptr-string)<(number-1)) {
        *ptr = ch;
        ++ptr;
        ch = getchar();
      }
      *ptr = '\0';
    }

  9. #9
    Registered User
    Join Date
    May 2012
    Posts
    1,066
    Code:
    int main(void)
    {
      ...
      char copy[strlen(string)+1];
       
      while (pos1) {
        str1 = malloc(strlen(string)+1);
        ...
        if(pos1) {
          str2 = malloc(strlen(string)+1);
        ...  
        }
        ...
      }
      return 0;
    }
    You're calculating the length of "string" several times. Since "string" is constant I recommend to calculate it once and save its value in a variable.

    You're also tokenizing "string" several times. I would store every token in an array after checking if it isn't already there. Then you would only tokenize the string once.

    Bye, Andreas

  10. #10
    Registered User
    Join Date
    Aug 2012
    Posts
    9
    Quote Originally Posted by qny View Post
    Write your own version of strtok().
    Do you have same example? For I don't understand how it works.
    That the point I do not understand. It receives a source string and returns tokens terminated by '\0'.
    But it doesn't alter the source string (suppose I), how does it work?

  11. #11
    Registered User
    Join Date
    Sep 2012
    Posts
    357
    Quote Originally Posted by SashaN View Post
    Do you have same example? For I don't understand how it works.
    That the point I do not understand. It receives a source string and returns tokens terminated by '\0'.
    But it doesn't alter the source string (suppose I), how does it work?
    Indeed, it does alter the source string.
    And it keeps information in static (local) variables.

  12. #12
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    That is what I mean by "strtok is destructive".
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  13. #13
    Registered User
    Join Date
    May 2012
    Posts
    1,066
    Quote Originally Posted by SashaN View Post
    Do you have same example? For I don't understand how it works.
    Here's the version from GNU libc.
    Here's an old version from NetBSD libc.
    Here's a version from Microsoft.

    Bye, Andreas

  14. #14
    Registered User
    Join Date
    Nov 2011
    Location
    Buea, Cameroon
    Posts
    197
    i had a recent problem with strtok using it on an array of strings for adding the null at the the present of any space but after the while loop the characters had changed drastically can some one give me a clue on this??

  15. #15
    - - - - - - - - oogabooga's Avatar
    Join Date
    Jan 2008
    Posts
    2,808
    Quote Originally Posted by Nyah Check View Post
    i had a recent problem with strtok using it on an array of strings for adding the null at the the present of any space but after the while loop the characters had changed drastically can some one give me a clue on this??
    You'd have to post code that demonstrates the problem.
    The cost of software maintenance increases with the square of the programmer's creativity. - Robert D. Bliss

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. strtok reset
    By lavinpj1 in forum C Programming
    Replies: 3
    Last Post: 04-25-2006, 06:00 PM
  2. Direct3D won't reset
    By Magos in forum Game Programming
    Replies: 1
    Last Post: 05-10-2004, 01:23 PM
  3. how to reset getline?
    By MKashlev in forum C++ Programming
    Replies: 6
    Last Post: 08-11-2002, 08:51 AM
  4. Is the origin reset...
    By fletch in forum Windows Programming
    Replies: 4
    Last Post: 07-29-2002, 05:59 PM
  5. help why does cnt not reset to 1??
    By datainjector in forum C Programming
    Replies: 4
    Last Post: 07-17-2002, 12:56 AM