multiple string searching woes

**lawina** · 06-07-2006

I am trying to do a multiple pattern search (50 of them) in a large log file (around 300MB size) using the strstr function:

Code:

 	while( getline( ifstreamobj , line_in_logfile ) )
	{
		if (strstr(line_in_logfile.c_str(), pattern1) != 0 ) ++Addcounter1 ;
		if (strstr(line_in_logfile.c_str(), pattern2) != 0 ) ++Addcounter2 ;
		if (strstr(line_in_logfile.c_str(), pattern3) != 0 ) ++Addcounter3 ;
	            ...........
		so on, upto 50 patterns

	}

I also created a script having multiple greps, like "grep -c pattern1 logfile" which does a similar function.

I timed both the C++ program above and the script. The c++ program above took "twice the time" the script took to execute.
I tried using fopen and fgets instead of fstreams, but only ended up getting the same results.

Its really frustating to be humbled by a shell script

Can someone suggest me an alternated logic in my c++ program.
Thanks

**Prelude** · 06-07-2006

>Its really frustating to be humbled by a shell script
Why? grep can school strstr up and down, left and right, any day of the week.

>Can someone suggest me an alternated logic in my c++ program.
Pipe the script through it?

**anon** · 06-07-2006

Firstly you could use the string class all the way: find instead of strstr.

Secondly you could use an array of patterns and an array of counters to save typing. (pattern[n] and counter[n] instead of patternn and countern)

**jafet** · 06-07-2006

You're using c_str() to make huge copies of your file, and passing them around! Remember that the string class needs to copy the whole string into an array, tack on a NULL onto the end of the string THEN return it, which is a nice waste of CPU. Just use find(), like anon said.

EDIT
I just found out that string just returns it's own string as a const char* in my implementation, so there wasn't as much overhead as I thought. But it's worth checking out.

Thread: multiple string searching woes

Thread Tools

Search Thread

Display

multiple string searching woes

Similar Threads

C++ std routines

lvp string...

Again Character Count, Word Count and String Search

string handling