Thread: String tokenization problem

  1. #1
    Registered User
    Join Date
    Aug 2016
    Posts
    1

    String tokenization problem

    I am trying to tokenize an array of strings. I first tried it without the for-loop but it printed a bunch of strange characters and I believe that is because I was passing an entire array to strtok instead of a single string. Now, with the for-loop, passing just 1 string at a time, it just crashes. What am I doing wrong? Please help!

    Here is my code

    Code:
    #define _CRT_SECURE_NO_WARNINGS
    #include<stdio.h>
    #include<string.h>
    #include<stdlib.h>
    
    void main()
    {
        char* s[] = { "12, 34, 56, 78", "82.16, 41.296",
                      "2, -3, 5, -7, 11, -13, 17, -19",
                      "9.00009, 90.0009, 900.009, 9000.09, 90000.9" };
        
        for (int i = 0; i <= sizeof(s) / sizeof(s[0]); i++) {
    
            char *token = strtok(s[i], ", ");
    
            while (token != NULL) {
                printf("%s\n", token);
                token = strtok(NULL, ", ");
            }
        }
        return 0;
    }
    Thanks in advance!

  2. #2
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,612
    > char* s[]
    Pretty sure the problem is the type of variable s is. strtok works, in part, by searching for and replacing characters in the delimiter string with '\0' and then returning a pointer to the start of the string. Since this requires that you change the string variable, it needs to be stored in writeable memory. So s would become something like:
    Code:
    #define S_MAX 100
    char s[S_MAX][4] = { ... };
    The reason that an array of char pointers won't work is because the memory that they point to is actually stored in the read-only section of the executeable. To get writeable memory, you have to tell the compiler to use an array of arrays.

    Well, you could also use an alternative method that doesn't try to change s.
    Code:
    #define _CRT_SECURE_NO_WARNINGS
    #include<stdio.h>
    #include<string.h>
    #include<stdlib.h>
     
    int main()
    {
        char* s[] = { "12, 34, 56, 78", "82.16, 41.296",
                      "2, -3, 5, -7, 11, -13, 17, -19",
                      "9.00009, 90.0009, 900.009, 9000.09, 90000.9" };
        int i; 
        for (i = 0; i < sizeof(s) / sizeof(s[0]); i++) {
            char *token = s[i];
            int pos = strcspn(token, ", ");
     
            while (1) {
                printf("%.*s\n", pos, token);
                token += pos;
                pos = strspn(token, ", "); 
                /* move past delimiters */
                token += pos;
                /* check for end of string */
                if (*token == '\0') {
                    break;
                }
                pos = strcspn(token, ", ");
            }
        }
        return 0;
    }
    Last edited by whiteflags; 08-14-2016 at 02:46 AM.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 4
    Last Post: 05-01-2013, 10:11 PM
  2. Replies: 0
    Last Post: 04-27-2013, 06:36 AM
  3. Replies: 1
    Last Post: 04-27-2013, 04:36 AM
  4. Math equation tokenization
    By Epy in forum Tech Board
    Replies: 1
    Last Post: 12-15-2011, 10:05 AM
  5. Replies: 22
    Last Post: 07-28-2011, 01:26 PM

Tags for this Thread