You know, drawing on Elkvis' example in post #7, you could do something like this: Look ma, no need for manual memory management!
O_o
That is fine if you only need a single delimiter; otherwise, a `std::getline' loop is not a replacement for `strtok'.
You can loop over `find_first_not_of' and `find_first_of' to get something more of a replacement for `strtok'.
a few years ago, I did some profiling of the various methods to tokenize a string, and found that strtok was generally about 5-10% faster than the method that laserlight and I are suggesting, but at the expense of not being thread safe, and just generally being less safe overall.
I'm curious if your `strtok' code was actually equivalent. I can easily see a simple `strtok' loop being faster--even more than 10%, but I have a difficult time believing a reliably measurable difference is found in copying the provided string, creating an array of pointers, allocating the individual strings, and copying the tokenized segments into the allocated strings.
Soma
[Edit]
Code:
#include <iterator>
#include <string>
struct token_iterator:
public std::iterator<std::forward_iterator_tag, std::string>
{
token_iterator():
mString()
, mDelimiters()
, mOrigin(std::string::npos)
, mTerminus(std::string::npos)
{
}
token_iterator
(
const std::string & fString
, const std::string & fDelimiters
):
mString(fString)
, mDelimiters(fDelimiters)
, mOrigin(fString.find_first_not_of(fDelimiters))
, mTerminus(fString.find_first_of(fDelimiters, mOrigin))
{
}
std::string operator * ()
{
return(mString.substr(mOrigin, mTerminus - mOrigin));
}
token_iterator & operator ++ ()
{
mOrigin = mString.find_first_not_of(mDelimiters, mTerminus);
mTerminus = mString.find_first_of(mDelimiters, mOrigin);
return(*this);
}
std::string mString;
std::string mDelimiters;
std::string::size_type mOrigin;
std::string::size_type mTerminus;
};
bool operator !=
(
const token_iterator & fLHS
, const token_iterator & fRHS
)
{
return(fLHS.mOrigin != fRHS.mOrigin);
}
#include <iostream>
#include <vector>
int main()
{
using namespace std;
vector<string> s(token_iterator("This is$only!a@test.", " $!@."), token_iterator());
copy(s.begin(), s.end(), ostream_iterator<string>(cout, "\n"));
}
[/Edit]