Printable View
That is indeed a major problem with the test. How about all test functions actually contain the contents of the file in a string in the end?Quote:
Also note that the third method fills an entire string with the data, while the other two keep overwriting the previous line/word.
I only wanted to test the read speed without any other calls getting in the way. I can try a few more things when I get home.Quote:
Originally Posted by anon
But the real use case would be completely different than in the test?Quote:
Originally Posted by cpjust
I guess it actually makes more sense to time programs that actually do something. If it turns out that file input is really slowing down the program, you should always be able to rewrite that part and compare the performance in real use cases.
But then to make it a fair test, wouldn't I also need to change Test3 to split up the one large string into individual lines?Quote:
Originally Posted by anon
You could. Or you could rewrite the first two to concatenate the strings as they are read in. It still won't be a completely fair test, but it will be closer.
I'm not sure what you mean about rdbuf(), I've never used it, but I added Test4() using istream.read(), and I also store all lines in 1 string and then print the final string size:Quote:
Originally Posted by Daved
Here's the new output:Code:#include <iostream>
#include <iterator>
#include <fstream>
#include <string>
#include <cstdio>
#include <ctime>
using namespace std;
void Test1( ifstream& file )
{
string line;
string fileData;
while ( !file.eof() )
{
getline( file, line );
fileData += line;
}
cout << "Test1 Bytes Read: " << fileData.size() << endl;
}
void Test2( ifstream& file )
{
string line;
string fileData;
while ( file >> line )
{
fileData += line;
}
cout << "Test2 Bytes Read: " << fileData.size() << endl;
}
void Test3( ifstream& file )
{
string fileData( (istreambuf_iterator<char>( file )),
istreambuf_iterator<char>() );
cout << "Test3 Bytes Read: " << fileData.size() << endl;
}
void Test4( ifstream& file )
{
char block[BUFSIZ + 1];
string fileData;
while ( file.read( block, BUFSIZ ) )
{
fileData += block;
}
cout << "Test4 Bytes Read: " << fileData.size() << endl;
}
typedef void (*TestFunc)( ifstream& );
clock_t TimeFunc( TestFunc func, const char* filename )
{
ifstream file( filename );
clock_t start = clock();
func( file );
clock_t end = clock();
return (end - start);
}
int main()
{
const char* filename = "E:/Test_10MB.txt"; // 10,240,000 bytes
for ( int i = 0; i < 1; ++i )
{
clock_t time1 = TimeFunc( &Test1, filename );
clock_t time2 = TimeFunc( &Test2, filename );
clock_t time3 = TimeFunc( &Test3, filename );
clock_t time4 = TimeFunc( &Test4, filename );
cout << endl << "Func1() time is: " << time1
<< endl << "Func2() time is: " << time2
<< endl << "Func3() time is: " << time3
<< endl << "Func4() time is: " << time4 << endl;
}
return 0;
}
Surprisingly, none of the 4 functions reports the right number of bytes (10,240,000). Does anyone know why that is? I know the >> operator doesn't extract whitespace and getline() doesn't extract the newline, but I would have expected the last two functions to store the right number of bytes.Code:Test1 Bytes Read: 10040000
Test2 Bytes Read: 9040000
Test3 Bytes Read: 10140000
Test4 Bytes Read: 10139648
Func1() time is: 1140
Func2() time is: 3485
Func3() time is: 1296
Func4() time is: 375
Of course, there is no local string, I don't know if that would add overhead or not.Code:#include <sstream>
...
void Test5( ifstream& file )
{
ostringstream ostr;
ostr << file.rdbuf();
cout << "Test5 Bytes Read: " << ostr.str().size() << endl;
}
You'd also want to add a null terminator to the block variable in test4.
Man, look how fast Test4 is. WOO-HOO!!
OK, now with Test5, here's the results:
I still don't know why none of them report the correct size?Code:Test1 Bytes Read: 10040000
Test2 Bytes Read: 9040000
Test3 Bytes Read: 10140000
Test4 Bytes Read: 10139648
Test5 Bytes Read: 10140000
Func1() time is: 1250
Func2() time is: 3171
Func3() time is: 1157
Func4() time is: 343
Func5() time is: 704
How many lines does the file have?
If it's a Windows file, all "newline" would be CR+LF in the file, and LF only in the data inside the program [unless you open the file in binary mode].
--
Mats
I don't feel like counting them, but 100,000 sounds about right (10240000 - 10140000).Quote:
Originally Posted by CornedBee
You mean you haven't got "wc" or an editor that is capable of showing you line numbers [for large files]?Quote:
Originally Posted by cpjust
--
Mats
I don't know what "wc" is, but I just opened it in VC++ (I'm surprised it didn't barf opening a 10MB file), and when I went to the bottom it showed line number 100001Quote:
Originally Posted by matsp
So Test4 still isn't quite right because of the null terminator issue, and Test3 and Test5 are correct (assuming you want newline conversion).
I'm still interested to see if you can get Test4 to give the same size output as Test3 and Test5. How do you tell how many bytes were read by read? You need to set a null terminator at that value every time (or at least the last time, the other times you can set block[BUFSIZ] to null).