Like Tree1Likes
  • 1 Post By MK27

Reading lines from a file

This is a discussion on Reading lines from a file within the C++ Programming forums, part of the General Programming Boards category; I know I'm going to end up feeling really stupid, but I give on this one. I have the following ...

  1. #1
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949

    Reading lines from a file

    I know I'm going to end up feeling really stupid, but I give on this one. I have the following code:

    Code:
    #include <fstream>
    #include <iostream>
    
    int main(void)
    {
    	std::ifstream file;
    	char temp[1024];
    
    	file.open("20110803.txt");
    	while (1)
    	{
    		file.getline(temp, 1024);
    		std::cout << temp << std::endl;
    	}
    	file.close();
    
    	return 0;
    }
    First off, I know about the endless loop, I'm trying to get one thing working at a time.

    The text file is ~1.5 GB and when I run the program, only about the first 4.8 MB is shown before it shows nothing but newlines, that is, it does not get all the way through the file.

    What the hell is going on here?

  2. #2
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    If it has anything to do with it, I'm on Windows and using mingw as the compiler.

  3. #3
    Registered User
    Join Date
    May 2010
    Posts
    2,690
    Why are you using the C-string instead of a std::string? You really should be using your getline() in your while statement, or else check the stream status after every use of getline().

    Jim

  4. #4
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    Because that is the method provided by the fstream class...I did see the getline provided by string, but figured that the previous would be much faster. As I said, this is a very large file and performance is very important. And as stated above, I know it runs in an endless loop. The original code used !file.eof() in the while loop.

  5. #5
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    If you want to read a file that big, you don't want to load it all into memory at once, so you are right to use a max chunk size via fstream's getline. I don't think the global getline() will allow you to do that. I dunno if one would be faster than the other tho, unless you used a delimiter that got continuously further and further apart in the file, meaning the string would keep needing to be reallocated.

    Have you put in some error checks?

    Code:
    using namespace std;
    [...]
        file.open("20110803.txt");
        if (file.fail()) {
        	cerr << "Open failed.\n";
        	return -1;
        }
        while (!file.eof()) {
        	if (!file.good()) {
        		cerr << "IO error.\n";
        		break;
        	}
    Hmmm....

    from http://www.cplusplus.com/reference/i...tream/getline/
    If the function stops reading because this size is reached, the failbit internal flag is set.
    "This size" would be the second parameter. Ie, if you have a stretch of 1024 characters without a '\n' and you didn't check file.good(), you will end up in an infinite loop.
    Last edited by MK27; 01-09-2012 at 02:15 PM.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  6. #6
    Registered User
    Join Date
    May 2010
    Posts
    2,690
    You need to somehow check the status of the input stream after your read. If your read fails, the stream will be put into an error state, and will not retrieve any more information. If the stream is in an error state then the cout will just keep printing what ever input it retrieved before the failure.
    The original code used !file.eof() in the while loop.
    This is also usually not a good idea, read the FAQ about using eof to control an input loop.


    Also I don't think you will notice much difference in speed using the std::string.

    Jim

  7. #7
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    Well isn't std::string going to dynamically allocate a string according to the size of the line read in? Whereas using a char array is fixed and should be fine as long as it's big enough, right?

    It seems to complete just fine with the following:

    Code:
    #include <fstream>
    #include <string>
    #include <iostream>
    
    int main(void)
    {
    	std::ifstream file;
    	std::string line;
    
    	file.open("20110803.txt");
    	while (std::getline(file, line))
    		std::cout << line << std::endl;
    	file.close();
    
    	return 0;
    }
    Still, why the heck doesn't the first code work?

  8. #8
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    I know from C that you are not supposed to use feof(), but could not find a better thing to use with fstream's facilities.

  9. #9
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Sorry, I tardily edited this into #5 above, it would seem to be the most likely cause:

    Hmmm....

    from http://www.cplusplus.com/reference/i...tream/getline/
    If the function stops reading because this size is reached, the failbit internal flag is set.
    "This size" would be the second parameter. Ie, if you have a stretch of 1024 characters without a '\n' and you didn't check file.good(), you will end up in an infinite loop.
    jimblumberg likes this.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  10. #10
    Registered User
    Join Date
    May 2010
    Posts
    2,690
    The way you are using the getline in post #7 is one of the better ways of controlling the input loop. That will test the stream status after the read and if the read fails it will end the loop. This will test not only for eof() but for any error reading the stream.

    Jim

  11. #11
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    Well, I don't know what the heck the deal was with the C facilities...ticks me off. I've got my program written now that I'm just using strings everywhere. Thanks guys.

  12. #12
    Epy
    Epy is offline
    Fortran lover Epy's Avatar
    Join Date
    Sep 2009
    Location
    California, USA
    Posts
    949
    Quote Originally Posted by MK27 View Post
    Sorry, I tardily edited this into #5 above, it would seem to be the most likely cause:
    Oh, so what you're saying is that I actually had a line longer than 1024 and it screwed up the works? Damn. It occurred to me, but none of the lines in this file should even approach a 1/4th of that. That's what I get for not error checking. Thanks again.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Reading lines from a File
    By DivineSlayer936 in forum C++ Programming
    Replies: 12
    Last Post: 04-02-2007, 12:12 PM
  2. Reading Lines from a file
    By phoenix-47 in forum C++ Programming
    Replies: 3
    Last Post: 12-13-2005, 02:32 PM
  3. Reading Lines From a File
    By Dangerous Dave in forum C++ Programming
    Replies: 6
    Last Post: 02-22-2005, 12:17 PM
  4. Reading new lines in a file
    By abrege in forum C++ Programming
    Replies: 11
    Last Post: 12-16-2002, 08:57 PM
  5. Reading Certain Lines In File?
    By SyntaxBubble in forum Windows Programming
    Replies: 8
    Last Post: 02-06-2002, 10:02 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21