strtok_r()

[paperbox005]

anyone have the implementation of the strtok_r() function?
i've been searching on google, but it only returns how to use the function, but not how it actually is implemented..

[jverkoey]

here's the strtok function, couldn't find strtok_r though

 /***
 *strtok.c - tokenize a string with given delimiters
 *
 *	   Copyright (c) Microsoft Corporation. All rights reserved.
 *
 *Purpose:
 *	   defines strtok() - breaks string into series of token
 *	   via repeated calls.
 *
 *******************************************************************************/
 
 #include <cruntime.h>
 #include <string.h>
 #ifdef _MT
 #include <mtdll.h>
 #endif  /* _MT */
 
 /***
 *char *strtok(string, control) - tokenize string with delimiter in control
 *
 *Purpose:
 *	   strtok considers the string to consist of a sequence of zero or more
 *	   text tokens separated by spans of one or more control chars. the first
 *	   call, with string specified, returns a pointer to the first char of the
 *	   first token, and will write a null char into string immediately
 *	   following the returned token. subsequent calls with zero for the first
 *	   argument (string) will work thru the string until no tokens remain. the
 *	   control string may be different from call to call. when no tokens remain
 *	   in string a NULL pointer is returned. remember the control chars with a
 *	   bit map, one bit per ascii char. the null char is always a control char.
 *
 *Entry:
 *	   char *string - string to tokenize, or NULL to get next token
 *	   char *control - string of characters to use as delimiters
 *
 *Exit:
 *	   returns pointer to first token in string, or if string
 *	   was NULL, to next token
 *	   returns NULL when no more tokens remain.
 *
 *Uses:
 *
 *Exceptions:
 *
 *******************************************************************************/
 
 char * __cdecl strtok (
 		char * string,
 		const char * control
 		)
 {
 		unsigned char *str;
 		const unsigned char *ctrl = control;
 
 		unsigned char map[32];
 		int count;
 
 #ifdef _MT
 		_ptiddata ptd = _getptd();
 #else  /* _MT */
 		static char *nextoken;
 #endif  /* _MT */
 
 		/* Clear control map */
 		for (count = 0; count < 32; count++)
 				map[count] = 0;
 
 		/* Set bits in delimiter table */
 		do {
 			    map[*ctrl >> 3] |= (1 << (*ctrl & 7));
 		} while (*ctrl++);
 
 		/* Initialize str. If string is NULL, set str to the saved
 		 * pointer (i.e., continue breaking tokens out of the string
 		 * from the last strtok call) */
 		if (string)
 				str = string;
 		else
 #ifdef _MT
 				str = ptd->_token;
 #else  /* _MT */
 				str = nextoken;
 #endif  /* _MT */
 
 		/* Find beginning of token (skip over leading delimiters). Note that
 		 * there is no token iff this loop sets str to point to the terminal
 		 * null (*str == '\0') */
 		while ( (map[*str >> 3] & (1 << (*str & 7))) && *str )
 				str++;
 
 		string = str;
 
 		/* Find the end of the token. If it is not the end of the string,
 		 * put a null there. */
 		for ( ; *str ; str++ )
 			    if ( map[*str >> 3] & (1 << (*str & 7)) ) {
 					    *str++ = '\0';
 					    break;
 				}
 
 		/* Update nextoken (or the corresponding field in the per-thread data
 		 * structure */
 #ifdef _MT
 		ptd->_token = str;
 #else  /* _MT */
 		nextoken = str;
 #endif  /* _MT */
 
 		/* Determine if a token has been found. */
 		if ( string == str )
 				return NULL;
 		else
 				return string;
 }

[Princeton]

The only difference between strtok and strtok_r is that strtok_r is reentrant. That means that the function does not maintain an internal state between calls. Every implementation of strtok that I know of uses a static variable to save the state so that the next call will begin at the proper location. strtok_r is not a standard function so it can be implemented in different ways as the interface is not set in stone if I recall correctly.

An example of the interface that strtok_r might use is as follows.

char *strtok_r(char *str, const char *sep, char **last);

Then instead of saving the position in a static variable, it is saved in last and last is then used as the first argument for subsequent calls.

char *last = string;

while ((t = strtok_r(last, ":", &last)) != NULL) {
  /* Do something with t */
}

[Hammer]

http://www.freebsd.org/cgi/cvsweb.cg...in&hideattic=0