Using strtok() in conjunction with fgets() to parse file data.

This is a discussion on Using strtok() in conjunction with fgets() to parse file data. within the C Programming forums, part of the General Programming Boards category; Full Disclosure: This is part of an assignment I have received from school. In the assignment we are forbidden to ...

  1. #1
    Registered User
    Join Date
    Feb 2013
    Posts
    9

    Using strtok() in conjunction with fgets() to parse file data.

    Full Disclosure: This is part of an assignment I have received from school. In the assignment we are forbidden to use fscanf().

    I have been trying to get this to work, but I've started to realize that I do not have a complete understanding of what strtok() actually does. I'm getting this warning when debugging: "assignment makes integer from pointer without cast."

    This warning happens when assigning str to goal and assist, and I think it is because they are, when dereferenced, integers. The code below correctly assigns the name into the correct spot, but leaves nonsense data in the goal and assist arrays. ex:-7880, -7888

    file example:
    NAME GOALS ASSISTS
    JOHN 1 2

    Code:
    void readLinesFromFile( FILE* Ptr, int* goal, int* assist, char** name, int lines ){/*
     * Reads lines from files and populates the arrays with the corresponding info.
     */
            int index;
            char hold[ MAX_LINE ] = { 0 };
            char* str = NULL;
    
    
            for( index = 0; index < lines; index++ ){
                    fgets( hold, MAX_LINE, Ptr );
    
    
                    str = strtok( hold, " " );
                    *( name + index ) = str;
    
    
                    str = strtok( NULL, " " );
                    *( goal + index ) = str;
    
    
                    str = strtok( NULL, " " );
                    *( assist + index ) = strtok( hold, " " );
            }//endfr
    }//endfn
    From what I understand about strtok(), it returns a string, and takes in a character array and a key value that tells it when to stop. In the online examples I've seen, they use NULL in the first field. I'm not sure why. I've read the definition on this site and I'm just missing some understanding about the function. I'd appreciate any help that could be provided.

  2. #2
    Registered User
    Join Date
    Nov 2012
    Posts
    1,220
    Quote Originally Posted by ofCircuits View Post
    I have been trying to get this to work, but I've started to realize that I do not have a complete understanding of what strtok() actually does.
    I'm getting this warning when debugging: "assignment makes integer from pointer without cast."


    It also helps if you show us the line which produced the error message. Because the line numbers might be different on the message board, you should indicate with the full text of the line exactly which line produced the message.

    As far as strtok goes, it tokenizes a string. In my opinion it is meant for exactly one use case in mind, basically something like this
    Code:
    const char delim[] = " ,";
    char someString[] ="- This is, a sample , string.";
    char *tok = strtok(someString, delim);
    while (tok != NULL)
    {
        printf ("Found token: %s\n", tok);
        tok = strtok(NULL, delim);
    }
    If you're using it for anything other than that, it probably won't work. For example, I would personally start with that boilerplate code and then modify it to include a counter, etc. if you really want to. HOWEVER if you know how many tokens are in your line, you may as well use sscanf. The return value indicates how many tokens were successfully matched, so it is easy to detect errors in your data.

  3. #3
    Registered User
    Join Date
    Feb 2013
    Posts
    9

    Thank you for your help.

    Quote Originally Posted by c99tutorial View Post
    It also helps if you show us the line which produced the error message. Because the line numbers might be different on the message board, you should indicate with the full text of the line exactly which line produced the message.

    Quote Originally Posted by ofCircuits
    This warning happens when assigning str to goal and assist, and I think it is because they are, when dereferenced, integers.
    This corresponds with lines 18 and 22.

    Quote Originally Posted by c99tutorial View Post
    As far as strtok goes, it tokenizes a string. In my opinion it is meant for exactly one use case in mind, basically something like this
    Quote Originally Posted by c99tutorial View Post
    Code:
    const char delim[] = " ,";
    char someString[] ="- This is, a sample , string.";
    char *tok = strtok(someString, delim);
    while (tok != NULL)
    {
        printf ("Found token: %s\n", tok);
        tok = strtok(NULL, delim);
    }
    This seems to be the standard example I've been able to find online, but like I said:

    Quote Originally Posted by ofCircuits
    From what I understand about strtok(), it returns a string, and takes in a character array and a key value that tells it when to stop. In the online examples I've seen, they use NULL in the first field. I'm not sure why.
    I guess I'm just looking for some help to achieve a deeper understanding of what is going on so that I may fix my problem.

    Quote Originally Posted by c99tutorial View Post
    If you're using it for anything other than that, it probably won't work. For example, I would personally start with that boilerplate code and then modify it to include a counter, etc. if you really want to. HOWEVER if you know how many tokens are in your line, you may as well use sscanf. The return value indicates how many tokens were successfully matched, so it is easy to detect errors in your data.
    I must use strtok(). Thank you for your help.


    EDIT: I seem to be making some headway with your template. One thing I've noticed is that the newline is being tokenized with my last value, but I think I can handle that.
    Last edited by ofCircuits; 02-17-2013 at 05:10 PM.

  4. #4
    Registered User
    Join Date
    Nov 2012
    Posts
    1,220
    One more thing: it's not valid to really assign `tok' anywhere after it's been returned by strtok. For example, don't do this:

    Code:
    myarray[i] = tok;
    Instead, you should make a copy like this

    Code:
    strcpy(myarray[i], tok);
    myarray must be an array of character pointers, each of which has enough room to store tok.

  5. #5
    Registered User
    Join Date
    Feb 2013
    Posts
    9
    I see, but in cases when I need to convert tok to an integer to store an integer?

    Code:
    int i = atoi( tok )
    Is that not valid.

    Also, why do we run a while loop until tok is NULL? Is this just to insure every piece of the string is covered? And if we know the number of tokens would that mean we could go without it?

  6. #6
    Registered User
    Join Date
    Nov 2012
    Posts
    1,220
    Quote Originally Posted by ofCircuits View Post
    I see, but in cases when I need to convert tok to an integer to store an integer?
    Yes, then it's fine. It probably helps to see your string like this

    char someString[] = "One Two Three Four";

    when you run strtok with a delim of " " it will scan through to find the first space and then stick in a \0, so now your string is "One\0Two Three Four" and the return value points to the beginning of "One". Each time you call it, it just moves this internal pointer on to the next token. When it gets to the end it returns NULL.

    If you know how many tokens there are, use sscanf. If you are required to use strtok for some obscure reason, then use strtok, but I would always check for NULL because that's what the spec says it returns:

    Man Page for strtok (freebsd Section 3) - The UNIX and Linux Forums

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Replies: 13
    Last Post: 07-06-2011, 08:48 PM
  2. fgets() and strtok()
    By surlyTomato in forum C Programming
    Replies: 6
    Last Post: 08-16-2009, 04:19 PM
  3. trying to use strtok() function to parse CL
    By ohaqqi in forum C Programming
    Replies: 15
    Last Post: 07-01-2007, 10:38 PM
  4. Can C++ be used in conjunction with XML Documents?
    By Junior89 in forum C++ Programming
    Replies: 3
    Last Post: 11-25-2005, 04:10 PM
  5. How can I parse data from a file
    By figo2476 in forum C Programming
    Replies: 5
    Last Post: 08-19-2005, 09:07 AM

Tags for this Thread


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21