Thread: String Tokenizing

Troi oi...
Here, I just wrote this real quick. Maybe it'll work...

Code:

const char* pipeTokenize(const char *str, char *tokenized, int bufSize) {
  *tokenized = NULL;
  int size = 0;

  while (*str != NULL && size < bufSize) {

    if (*str == '|') {
      ++str;
      break;
    }

    ++size;
    *tokenized = *str;
     *++tokenized = NULL;

    ++str;
  }

  return *str == NULL ? NULL : str;
}

int main() {
  string delimited = "|field2||field4";
  char buffer[256];

  const  char *p  = delimited.c_str();
  while (p != NULL) {
    p = pipeTokenize(p, buffer, 256);
    
    cout << "Hey dummy, here's a field: " << buffer << endl;
  }

  return 0;
}

What's wrong with my function splitting to a vector of strings? After all we're using C++ here, right?

This was on the AP Computer Science exam I took today, sorry if it's already been said.

Originally posted by CheesyMoo
This was on the AP Computer Science exam I took today, sorry if it's already been said.

Yeah, several people have come on the board telling us about your AP computer science exam... It's funny you didn't notice.

Can I have a hug?

I had the same problem because I was reading in from a tab delimited file.... Here is what I did... I used the CStrings replace function to replace "\t\t" with "\t \t" so that there would be at least one space in between tabs and therefore the toke would be picked up as a blank space. Later if I found a blank space I would just copy an empty sting into that token. Of course this means that you have to move the buffer into a string and then back into the buffer. I am not quite sure what the equivalent functions of CString are in string to accomplish this but I am sure that they exist.

Code:

while( cFile.ReadString( strInline ) == TRUE)
{
      //Trime all spaces to the right in case of empty string
      strInline.TrimRight();
			
     if( strInline <= " ")
	continue;

    //Remove all back to back tabs with space in between them
    while( strInline.Find("\t\t") > -1)
            strInline.Replace("\t\t", "\t \t");

    char *buffer;
    char *pToken;

    //Create a new buffer
    buffer = new char[ strInline.GetLength()+1];

    strcpy(buffer, strInline.GetBuffer( strInline.GetLength()+1 ));

    pToken = strtok(buffer, "\t");
    int i = 0;
    CString strTokens[31];

    strTokens[i++] = pToken;
    if(strTokens[i-1] <= " ")
    strTokens[i-1] = _T("");

   while( pToken != NULL && i < 30)
   {
          /* Get next token: */
          pToken = strtok( NULL, "\t");
          strTokens[i++] = pToken;

          if( strTokens[i-1] <= " ")
	strTokens[i-1] = _T("");
   }