Splitting A String Into Tokens!

This is a discussion on Splitting A String Into Tokens! within the C Programming forums, part of the General Programming Boards category; Right, I am having a little trouble with making a string tokeniser to split up a string! NOTE: This is ...

  1. #1
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121

    Exclamation Splitting A String Into Tokens!

    Right, I am having a little trouble with making a string tokeniser to split up a string!

    NOTE: This is for university coursework. So please just give examples; not a complete solution!

    and a typical request looks something like:

    BREW POT-1 HTCPCP/1.0\r\nAccept-Additions:#milk-type;skimmed#milk-amount;lots\r\n

    obviously there is a little bit missing in the above request (syrup, sweetener etc.)

    All i am trying to do is to split up the string into tokens so i can process the request! at the server end!!!!!

    i want to be able to have the tokens;


    BREW
    POT-1
    HTCPCP/1.0
    ....

    using delimeters " ;#\r\n"

    How can i do this??? ...All i want to do at the moment is print the tokens to the screen as the request arrives!!!


    Here is the code that I have done so far, but its not working!!!!!!!!
    Code:
      
    char Array[MAX_ARRAY] = "BREW POT-1 HTCPCP/1.0\r\nAccept-Additions:#milk-type;skimmed#milk-amount;lots\r\n";
      char *TmpArray;
      char *Tokens[MAX_TOKEN];
      int TokenCount;
    
      char *ptr;
        
      TokenCount = 0;
      TmpArray = strdup(Array);
      for (ptr = TmpArray; *ptr;) {
        while (*ptr == ' ')               /* ignore leading space AND treat consecutive spaces as a single separator */
          *(ptr++) = 0;                   /* replace the space with end-of-string */
        if (++TokenCount >= MAX_TOKEN)    /* check for too many tokens.  Ignore the extras. */
          Tokens[TokenCount] = ptr;       /* Save the address of the current token */
        while (*ptr && *ptr != ' ')       /* Skip the contents of the token */
          ptr++;
      }
      
      for (i = 0; i < TokenCount; i++){
        printf("tokens: %s\n", Tokens[i]);
      }
    Thanks for your help!!!! Matt.

  2. #2
    Just Lurking Dave_Sinkula's Avatar
    Join Date
    Oct 2002
    Posts
    5,006
    I'd wager the assignment is geared toward a strtok type solution akin to this.
    Code:
    void parse ( char *text, const char *delimiters )
    {
        char *token = strtok ( text, delimiters ); /* find first token */
        while ( token )
        {
            printf ( "token = \"%s\"\n", token ); /* do stuff */
            token = strtok ( NULL, delimiters ); /* find remaining tokens */
        }
    }
    If you do something like this, do not pass a string literal as the parameter text -- you need to pass a modifiable string such as this.
    Code:
        char msg[] = "BREW POT-1 HTCPCP/1.0\r\nAccept-Additions:"
                     "#milk-type;skimmed#milk-amount;lots\r\n";
        parse ( msg, " ;#\r\n" );
    [edit](Which you are already doing, I'm just highlighting this.)
    Last edited by Dave_Sinkula; 02-14-2006 at 09:04 AM.
    7. It is easier to write an incorrect program than understand a correct one.
    40. There are two ways to write error-free programs; only the third one works.*

  3. #3
    Registered User
    Join Date
    Mar 2004
    Posts
    66
    Quote Originally Posted by Dave_Sinkula
    I'd wager the assignment is geared toward a strtok type solution akin to this.
    You would think. However out lecturer hates strtok with a passion.

  4. #4
    Just Lurking Dave_Sinkula's Avatar
    Join Date
    Oct 2002
    Posts
    5,006
    Quote Originally Posted by jamie85
    You would think. However out lecturer hates strtok with a passion.
    Wonderful!

    Here are some other ideas.
    Parsing a String into Tokens Using sscanf
    Parsing a String into Tokens Using strcspn 1
    7. It is easier to write an incorrect program than understand a correct one.
    40. There are two ways to write error-free programs; only the third one works.*

  5. #5
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121
    Thanks for the help Dave!!! ...I have managed to write a string tokeniser without using strtok and its pretty simple!!!!!! Well, heres the code for it.....

    Code:
    char ** stringTokenizer(char * theData, char * theDelims){
      
      static char *Token[5];   /* ARRAY TO STORE POINTERS TO TOKENS */
      int cnt = 0;              /* TOKEN COUNTER */
      
      /* LOOP THROUGH INPUT STRING UNTIL END OF STRING CHAR IS FOUND */
      while (*theData!='\0') {
        /* IS THE CURRENT CHAR A DELIMETER CHAR? */
        while (*theData!='\0' && strchr(theDelims,*theData)) {
          *theData='\0';   /* REPLACE DELIMETER CHAR WITH STRING TERMINATION CHAR */
          theData++;       /* POINT THE THE FIRST CHAR OF THE NEXT TOKEN */
        } 
        Token[cnt++]=theData;  /* RECORD THE POSITION OF THE NEXT TOKEN */
        /* EAT UP ALL NON-DELIM CHARS IN CURRENT TOKEN */
        while (*theData!='\0' && !strchr(theDelims,*theData)) { 
          theData++; /* MOVE TO NEXT CHAR IN STRING */
        }
      }
      Token[cnt] = 0;
      return Token;   /* RETURN THE ARRAY OF TOKENS */
      
    }

  6. #6
    Registered User
    Join Date
    Mar 2006
    Location
    bristol
    Posts
    5

    rubbish

    thats rubbish itll never work!

  7. #7
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121

    U Idiot

    Quote Originally Posted by bll2109
    thats rubbish itll never work!
    You idiot ben! ...write your own tokeniser! stop stealing my code!!!

  8. #8
    Registered User
    Join Date
    Mar 2006
    Location
    bristol
    Posts
    5
    Quote Originally Posted by bobthebullet990
    You idiot ben! ...write your own tokeniser! stop stealing my code!!!
    what you on about i wrote it in the first place and you stole it from me! idiot matt

  9. #9
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121
    Quote Originally Posted by bobthebullet990
    You idiot ben! ...write your own tokeniser! stop stealing my code!!!
    Get a life u looser!!! Goto cern and never return! later aligator! over and out!

  10. #10
    Registered User
    Join Date
    Mar 2006
    Location
    bristol
    Posts
    5
    Quote Originally Posted by bobthebullet990
    Get a life u looser!!! Goto cern and never return! later aligator! over and out!
    your only jealous

  11. #11
    Devil's Advocate SlyMaelstrom's Avatar
    Join Date
    May 2004
    Location
    Out of scope
    Posts
    4,062
    *clicks the lock button he wishes he has*
    Sent from my iPadŽ

  12. #12
    Registered User
    Join Date
    Mar 2006
    Location
    bristol
    Posts
    5
    Quote Originally Posted by SlyMaelstrom
    *clicks the lock button he wishes he has*
    lock down mate!! lock down!!

  13. #13
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121

    Lockdown

    Watch out guys! the time is goin all hyroglifical and the shutters are goin down! QUICK Hit the execute button! ...Its not supposed to be like this! ...how is it sposed to be Lock????? (This is the question!)

  14. #14
    Matt Conway bobthebullet990's Avatar
    Join Date
    Nov 2005
    Location
    Cambridge
    Posts
    121
    HUH! ...So u wanna convert an int to a char array and then put that char array into the send buffer! u idiot! U just print the int straight into the send buffer!!!

    point proved mate!!! L.O.L...........................

  15. #15
    Registered User
    Join Date
    Mar 2006
    Location
    bristol
    Posts
    5
    how about we get on with some work instead of sending lame ass messages from 2 machines away of each other!

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. String splitting algorithm help
    By (TNT) in forum C++ Programming
    Replies: 7
    Last Post: 05-12-2007, 11:28 AM
  2. Program using classes - keeps crashing
    By webren in forum C++ Programming
    Replies: 4
    Last Post: 09-16-2005, 03:58 PM
  3. Linked List Help
    By CJ7Mudrover in forum C Programming
    Replies: 9
    Last Post: 03-10-2004, 09:33 PM
  4. creating class, and linking files
    By JCK in forum C++ Programming
    Replies: 12
    Last Post: 12-08-2002, 01:45 PM
  5. Again Character Count, Word Count and String Search
    By client in forum C Programming
    Replies: 2
    Last Post: 05-09-2002, 11:40 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21