Reading lines from a file

[Epy]

I know I'm going to end up feeling really stupid, but I give on this one. I have the following code:

#include <fstream>
#include <iostream>

int main(void)
{
	std::ifstream file;
	char temp[1024];

	file.open("20110803.txt");
	while (1)
	{
		file.getline(temp, 1024);
		std::cout << temp << std::endl;
	}
	file.close();

	return 0;
}

First off, I know about the endless loop, I'm trying to get one thing working at a time.

The text file is ~1.5 GB and when I run the program, only about the first 4.8 MB is shown before it shows nothing but newlines, that is, it does not get all the way through the file.

What the hell is going on here?

[Epy]

If it has anything to do with it, I'm on Windows and using mingw as the compiler.

[jimblumberg]

Why are you using the C-string instead of a std::string? You really should be using your getline() in your while statement, or else check the stream status after every use of getline().

Jim

[Epy]

Because that is the method provided by the fstream class...I did see the getline provided by string, but figured that the previous would be much faster. As I said, this is a very large file and performance is very important. And as stated above, I know it runs in an endless loop. The original code used !file.eof() in the while loop.

[MK27]

If you want to read a file that big, you don't want to load it all into memory at once, so you are right to use a max chunk size via fstream's getline. I don't think the global getline() will allow you to do that. I dunno if one would be faster than the other tho, unless you used a delimiter that got continuously further and further apart in the file, meaning the string would keep needing to be reallocated.

Have you put in some error checks?

using namespace std;
[...]
    file.open("20110803.txt");
    if (file.fail()) {
    	cerr << "Open failed.\n";
    	return -1;
    }
    while (!file.eof()) {
    	if (!file.good()) {
    		cerr << "IO error.\n";
    		break;
    	}

Hmmm....

from http://www.cplusplus.com/reference/i...tream/getline/

If the function stops reading because this size is reached, the failbit internal flag is set.

"This size" would be the second parameter. Ie, if you have a stretch of 1024 characters without a '\n' and you didn't check file.good(), you will end up in an infinite loop.

[jimblumberg]

You need to somehow check the status of the input stream after your read. If your read fails, the stream will be put into an error state, and will not retrieve any more information. If the stream is in an error state then the cout will just keep printing what ever input it retrieved before the failure.

The original code used !file.eof() in the while loop.

This is also usually not a good idea, read the FAQ about using eof to control an input loop.


Also I don't think you will notice much difference in speed using the std::string.

Jim

[Epy]

Well isn't std::string going to dynamically allocate a string according to the size of the line read in? Whereas using a char array is fixed and should be fine as long as it's big enough, right?

It seems to complete just fine with the following:

#include <fstream>
#include <string>
#include <iostream>

int main(void)
{
	std::ifstream file;
	std::string line;

	file.open("20110803.txt");
	while (std::getline(file, line))
		std::cout << line << std::endl;
	file.close();

	return 0;
}

Still, why the heck doesn't the first code work?

[Epy]

I know from C that you are not supposed to use feof(), but could not find a better thing to use with fstream's facilities.

[MK27]

Sorry, I tardily edited this into #5 above, it would seem to be the most likely cause:

Hmmm....

from http://www.cplusplus.com/reference/i...tream/getline/

If the function stops reading because this size is reached, the failbit internal flag is set.

"This size" would be the second parameter. Ie, if you have a stretch of 1024 characters without a '\n' and you didn't check file.good(), you will end up in an infinite loop.

[jimblumberg]

The way you are using the getline in post #7 is one of the better ways of controlling the input loop. That will test the stream status after the read and if the read fails it will end the loop. This will test not only for eof() but for any error reading the stream.

Jim

[Epy]

Well, I don't know what the heck the deal was with the C facilities...ticks me off. I've got my program written now that I'm just using strings everywhere. Thanks guys.

[Epy]

Sorry, I tardily edited this into #5 above, it would seem to be the most likely cause:

Oh, so what you're saying is that I actually had a line longer than 1024 and it screwed up the works? Damn. It occurred to me, but none of the lines in this file should even approach a 1/4th of that. That's what I get for not error checking. Thanks again.