Thread: How Difficult is This?

Salem:
I'm studying your code.
Why do you call bsearch the second time against 10% of the master file?
Also, with rand() you could end up searching the same rec twice yes?

Why do sometimes you dim function args as const?

SSD drives are apparently, our future:

Intel’s SSD Plans : Intel's X25-M Solid State Drive Reviewed

VERY fast, and no moving parts. Pricey, but it will come down more, no doubt.

What's the purpose of all these url's you're gathering?

@Salem: I know absolutely nothing about C#, except that it needs dot net to run. Never looked into it. Yes, brute force and the most basic but efficient algorithm, was my goal. Data structures might depend on what else he's doing.

Youza - 250Mb/s is fast. My drives read at 120/s sustained.
My main drive is 650 Gig though, they'll have to make 'em bigger.

I'm doing Internet data polling for clients, mostly for SEO work.

> Why do you call bsearch the second time against 10% of the master file?
> Also, with rand() you could end up searching the same rec twice yes?
The whole last section is just a bit of bench-marking to see how long searching takes.
0.01 seconds for the first result is noise, so I wanted a bigger sample.

Reading the whole of the 2nd file is sub-optimal as well, since there is no need to store anything. But it was quick and easy to re-use the same code. It will save some short-term RAM use, but it won't do much to change the slowness of the file reading.

> Why do sometimes you dim function args as const?
- To stop me changing things by accident
- To give the compiler a little extra something to play with at optimisation time. Certain things which are known to be const can be optimised better.

> fprintf(fp1, strcat(db[i].base_url,"\n"));
Use
fprintf(fp1, "%s\n", db[i].base_url);

1. strcat is attempting to append a character to a string where the exact length has been allocated. This is a buffer overflow.
2. Any % characters that appear in this string will cause fprintf to look for additional (and non-existent) additional parameters.


> result[numRecords].full_url = r.full_url;
> result[numRecords].base_url = r.base_url;
> numRecords++;
At this point, you need to set your isUnique member to false as well.
realloc won't zero this memory for you.

Zero (0) will do it.

The person who wrote the article is an idiot.
Specialised algorithms like Boyer Moore a specifically designed for searching under different conditions than naieve algorithms are designed for. E.g. Longer strings to search for, more limited alphabet, high number of prefix but not whole word matches, searching through very long datasets etc. It's apples and oranges.
Using an algorithm like that to search for "the" would be like using quicksort on 5 items. Bubble sort is much faster in that case.

Surely it would have been fairer to put as much effort into optimising the C++ code as it was to whoever optimised the assembly version.

The benchmark is flawed in other ways too. Things like the switch statement having the strstr variant as the first case. Hmm, assuming the compiler doesn't optimise the switch to a jump-table, which one will that one make fastest?! Caching effects are not taken into consideration. I could go on...

Originally Posted by MAtkins

So, tell me about those SSD drives . . .
I'm running Win 7 64 bit on an Alienware Aurora - overclocked at I think 3.67 Ghz Quad core - 6 Gig RAM.
It's got a radiator in it.

Ooh nice! Just got a new machine at work yesterday. Win 7 64 bit, 2.9GHz 8 core, 12 Gig RAM, and a couple of 10000RPM raptor 512 Gig hard drives.

One guy at work had an SSD in his old machine, and oh yeah is that fast!