Thread: how to parse a string

  1. #16
    Registered User
    Join Date
    Jan 2008
    Posts
    569
    so it will look for either ":" or " " , whichever occurs first? correct??

    and how do I know if the deliminiter uses the ":" or " "

  2. #17
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    (1) Right.

    (2) You don't. (Unless, of course, you make a copy of the original string, subtract the pointer to the original word from the pointer returned from strtok, and use that as an index into your copy to see what the original character was.)

  3. #18
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Strtok is a devil's function. Surely there is some better alternative?
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  4. #19
    Registered User
    Join Date
    Jan 2008
    Posts
    569
    what other alternative is there to parse a string?

  5. #20
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    I don't know of any other C function, but it's not difficult to write your own.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  6. #21
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Elysia View Post
    Strtok is a devil's function. Surely there is some better alternative?
    Such as? What's so bad with it? Yes, it modifies the input string, and it's not re-entrant (but both of these things are clearly stated in the docs), but otherwise, it does what it says on the label - splits strings into tokens based on the separator string. There is a strtok_r() function that allows re-entrant use.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  7. #22
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Quote Originally Posted by matsp View Post
    Such as?
    I don't know of any alternative.

    What's so bad with it?
    The worse possible thing is that it damages the input string. If you want to keep it intact, you can't use strtok or you have to make a copy. Expensive and unnecessary.
    The second is that it keeps track of the position using a static variable to my knowledge, which is still worse. It loses the meaning of a good function. A good function allows you to specify where to begin the search. Either as an index or as a (const) char* variable.

    Yes, it modifies the input string, and it's not re-entrant (but both of these things are clearly stated in the docs), but otherwise, it does what it says on the label - splits strings into tokens based on the separator string.
    The documentation is not at fault - the function itself is.

    There is a strtok_r() function that allows re-entrant use.
    I can't find that one in Visual Studio's documentation... Non-standard or is Microsoft simply lacking support?
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  8. #23
    Registered User
    Join Date
    Jan 2008
    Posts
    569
    I guess I am too lazy to write a new function to do this, strtok is fine for me

  9. #24
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Yes, it would of course be more flexible to copy out the new string, but then you only need THAT if you need to keep the original string intact, so someone else would complain about that - no such thing as pleasing everyone at once.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  10. #25
    Registered User
    Join Date
    Jan 2008
    Posts
    569
    and by the way , I don't need to keep the original string as it is for my purpose right now. so strtok is fine

  11. #26
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Non-destructive version:
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <stdint.h>
    #include <assert.h>
    
    const char* strtokv2(const char* strToSearch, const char* strDelimiter);
    const char* strtokv3(const char* strToSearch, const char* strDelimiter, char* pBuffer, size_t nBufferSize);
    
    const char* strtokv2(const char* strToSearch, const char* strDelimiter)
    {
    	size_t nStrLength = strlen(strToSearch);
    	size_t nDelLength = strlen(strDelimiter);
    	const char* strEnd = strToSearch + nStrLength;
    	const char* p = strToSearch;
    	while (p < strEnd && *p == ' ') p++;
    	while (p++ < strEnd)
    	{
    		if (*p == *strDelimiter)
    		{
    			bool bFound = true;
    			if ((size_t)(strEnd - p) < nDelLength) // Length of remaining string - length of delimiter
    			for (size_t i = 1; i < nDelLength; i++)
    			{
    				if (p[i] != strDelimiter[i])
    				{
    					bFound = false;
    					break;
    				}
    			}
    			if (bFound)
    				return p;
    		}
    	}
    	return NULL;
    }
    
    const char* strtokv3(const char* strToSearch, const char* strDelimiter, char* pBuffer, size_t nBufferSize)
    {
    	uint32_t nLength;
    	char* pFound;
    	if (!strToSearch) return NULL;
    	pFound = (char*)strtokv2(strToSearch, strDelimiter);
    	if (!pFound)
    		nLength = strlen(strToSearch) + 1; // +1 for NULL
    	else
    		nLength = (pFound - strToSearch) + 1; // +1 for NULL
    	assert(nBufferSize >= nLength);
    	memcpy(pBuffer, strToSearch, nLength - 1);
    	pBuffer[nLength - 1] = 0;
    	if (pFound)
    		return pFound + 1;
    	else
    		return pFound;
    }
    
    int main (void)
    {
    	char str[] = "This         is            a           test            string";
    	char mytokenizedstr[50][20] = {0};
    	int index = 0;
    	const char* pSearchPos = str;
    
    	while ( (pSearchPos = strtokv3( pSearchPos, " ", mytokenizedstr[index], sizeof(mytokenizedstr[index]) )) != NULL ) index++;
    	for (int i = 0; i < 5; i++)
    		puts(mytokenizedstr[i]);
    	puts(str);
    	return 0;
    }
    Added strtokv3. Should do all strtok does and better.
    Actually, I think I'll save this and recommend it over strtok.
    This is the proper way of doing it.
    Last edited by Elysia; 04-20-2008 at 04:17 PM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  12. #27
    Registered User
    Join Date
    Jan 2008
    Posts
    569
    great job Elysia.. you're a master.. thanks though for the code.. I might use it

  13. #28
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Yes, it's not a very difficult function to replace. And you have to bear in mind that a lot of standard functions are pretty simple, and may not suit all purposes.

    [I'm not sure your code actually works, the
    Code:
    if(*p == *strDelimiter)
    looks wrong to me].

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  14. #29
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    I tested it and it works.
    It compares the current char pointed to by p to the first char in the string strDelimiter.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  15. #30
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    But your test-case uses a single-char string. Try something where the first match is the second (or any other nth, where n >= 2 [delimiter matches for index > 0]) char in the delimiter string - I dont _THINK_ that will work - I haven't tried it myself, but it doesn't look like the you're doing the right thing in that case.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Interpreter.c
    By moussa in forum C Programming
    Replies: 4
    Last Post: 05-28-2008, 05:59 PM
  2. Calculator + LinkedList
    By maro009 in forum C++ Programming
    Replies: 20
    Last Post: 05-17-2005, 12:56 PM
  3. Classes inheretance problem...
    By NANO in forum C++ Programming
    Replies: 12
    Last Post: 12-09-2002, 03:23 PM
  4. creating class, and linking files
    By JCK in forum C++ Programming
    Replies: 12
    Last Post: 12-08-2002, 02:45 PM
  5. Warnings, warnings, warnings?
    By spentdome in forum C Programming
    Replies: 25
    Last Post: 05-27-2002, 06:49 PM