Initialize vectors from streams via itr adapters

**sean_cantab** · 11-26-2016

Trying to learn above topic and some simple code for reading stringstream and fstream directly into vector<string> with or without std::copy is working fine. However when I try to read from std::cin it seems I can't quit the stream, every time I enter some input and press Enter I get another cursor prompt. How do I get out of this loop and print the vector, assuming of course that it's been populated from cin in the meantime?

Code:

#include <iostream>
#include <string>
#include <fstream>
#include <sstream>
#include <iterator>
#include <vector>
using namespace std;
int main()
{
    //stringstream ss("The quick brown fox jumped over the lazy dog.");
    //fstream file("F:\\test1.txt");
	
    vector<string> col;
    // copy(istream_iterator<string>(ss), istream_iterator<string>(), back_inserter(col));
	// copy(istream_iterator<string>(file), istream_iterator<string>(), back_inserter(col));
    copy(istream_iterator<string>(cin), istream_iterator<string>(), back_inserter(col));


	// vector<string> col((istream_iterator<string>(ss)), istream_iterator<string>());
	// vector<string> col((istream_iterator<string>(file)), istream_iterator<string>());
	// vector<string> col((istream_iterator<string>(cin)), istream_iterator<string>());


    for (auto& elem : col)
    {
        cout<<elem <<"*";
    }
}

ps: how does std::copy working w/o #include<algorithm>?
pps: started from this link: c++ - How can I use std::copy to read directly from a file stream to a container? - Stack Overflow

**whiteflags** · 11-26-2016

Normally you have to signal the end of input. The nicest signal to send - what the program would expect to be sent - is EOF. Type Ctrl-Z or Ctrl-D depending on the platform.

ps: how does std::copy working w/o #include<algorithm>?

It doesn't? Something else must have included it as an implementation detail. Include it explicitly anyway.

**sean_cantab** · 11-27-2016

Thanks, it was Ctrl-Z on mine (C::B, gcc 4.9.2, Windows 10). Added in <algorithm>

**sean_cantab** · 11-30-2016

Coming back to this thread ... I wanted to see if I could read into a struct object that has a vector<int> data member using iterator adapters. I set out the struct, overloaded operators and main() as below but the problem is that only the first line of data is read from the file into an object and then failbit is set. And if I try to clear() ifstream (either as part of the overload definition or in main()) the program enters a never ending loop

Code:

#include <iostream>
#include <vector>
#include <string>
#include <fstream>
#include <iterator>
#include <algorithm>

using namespace std;
struct Person
{
     string name = " A N Other";
     vector<int> vec{};
};
istream& operator >> (istream& inFile, Person& p)
{
    static const auto delimiter = ';';
    getline(inFile, p.name, delimiter);
    copy(istream_iterator<int>(inFile), istream_iterator<int>(), back_inserter(p.vec));
    //inFile.clear();


    return inFile;
}
ostream& operator << (ostream& os, const Person& p)
{
    os<<"Name: "<<p.name<<"; Numbers: ";
    for(auto& elem : p.vec)
    {
        os<<elem<<"\t";
    }
    os<<"\n";
    return os;
}
int main()
{
    fstream inFile("F:\\test.txt");
    vector<Person> p_vec;
    while(inFile)
    {
        Person p{};
        inFile>>p;
        p_vec.emplace_back(p);
      //  inFile.clear();


        if(inFile.fail())
        {
            cout<<"Failure!\n";
          // inFile.clear();
        }
    }
    for(auto& elem : p_vec)
    {
        cout<<elem;
    }
}

My questions are:
a. I now notice that even in my first post under this thread the ifstream state was set to fail() after reading the line of text. Since that file had one line only I hadn't picked this up earlier. Why does this happen with iterator adapters
b. what is the right way to use clear() in this case, if at all?
Sample text file:
John Smith; 45 98 12
Jane Doe; 14 76 98
A N Other; 78 97 34
Many thanks

edit: if after inFile>>p I add the lines recommended here: Issues with reading series of 2D matrices from file

Code:

if(inFile) 
       {
            p_vec.emplace_back(p);
        }

then, of course, even the first line of data will not be read

**whiteflags** · 11-30-2016

The problem is that istream iterator on some level uses operator>>. When you read integers with copy the only thing that will stop it is EOF or a reading error.
So what happens is you'll read all of this and error out here because istream_iterator is still expecting integers.
John Smith; 45 98 12
Jane Doe; 14 76 98

So what do you do instead? Well if you wanted to keep this approach you could do:

Code:

Person p;
getline(inFile, line);
istringstream ss(line);
ss >> p;

Or you can give up an using copy so much. It's not appropriate for everything.

Code:

istream& operator >> (istream& is, Person& p)
{
   getline(is, p.name, ';');
   p.vec.resize(3);
   for (auto& i : p.vec) {
      is >> i;
   }
   return is;
}

Code is untested, by the way, but it should be close if not perfect.

**jimblumberg** · 11-30-2016

You're going to have problems because of the way your overloaded operator>> is defined.

Code:

istream& operator >> (istream& inFile, Person& p)
{
    static const auto delimiter = ';';
    getline(inFile, p.name, delimiter);
    copy(istream_iterator<int>(inFile), istream_iterator<int>(), back_inserter(p.vec));
    //inFile.clear();
 
 
    return inFile;
}

The getline() is fine, although you don't need to define a variable for the delimiter, just use a character constant instead ';'.

But your copy will not stop until it encounters a stream error, which in this case will happen when it tries to read the first character of the next line. The operator>> knows it can't insert a character in a numeric variable.

b. what is the right way to use clear() in this case, if at all?

I would recommend using a stringstream to process the lines. Something like:

Code:

istream& operator >> (istream& inFile, Person& p)
{
    string temp;
    getline(inFile, temp);
    stringstream sin(temp);
    getline(sin, p.name, ';');
    copy(istream_iterator<int>(inFile), istream_iterator<int>(), back_inserter(p.vec)); 
 
    return inFile;
}

Then the copy() operation will fail because it reaches the "eof()" of the stringstream but this failure will not effect the underling stream.

But normally when you encounter a stream error you need to clear() the stream error and then remove the offending data from the stream.

Jim

**sean_cantab** · 11-30-2016

Many thanks to you both

Or you can give up an using copy so much. It's not appropriate for everything.

yeah, I suppose you're right. There are alternative ways I can do this but just wanted to see how far I could push this line of approach
Jim: afraid still having the same issues with the stringstream approach. In fact before fail() sstringstream is not able to read even the int's of the first object that ifstream could

when you encounter a stream error you need to clear() the stream error and then remove the offending data from the stream.

if possible could you or anyone else advice if this can be done at all on my previous post or is it a situation beyond redemption?
Thanks again
PS: btw, when I quote someone in my reply how can I put the name of the person before what s/he said? thanks

**algorism** · 11-30-2016

Your main problem is that you're not testing for end-of-file properly. You need to test for a bad inFile right after the first getline call in operator>>. Doing that pretty much fixes it:

Code:

istream& operator>>(istream& inFile, Person& p) {
    getline(inFile, p.name, ';');
    if (inFile) {
        copy(istream_iterator<int>(inFile), istream_iterator<int>(),
             back_inserter(p.vec));
        inFile.clear();
    }
    return inFile;
}

BTW, I don't think your use of emplace_back is gaining you anything. Maybe try a push_back(move(p)).

And you should check that the file was actually opened. The way your code is it will actually handle this situation, but it's better to explicitly test and report the problem.

**jimblumberg** · 11-30-2016

afraid still having the same issues with the stringstream approach.

That's probably because you're not using the correct "begin", "end" iterators in your copy() call. Try something like:

Code:

    std::istream_iterator<int> eos;  // end-of-stream iterator
    std::istream_iterator<int> iit (sin);
    copy(iit, eos, back_inserter(p.vec));

But I also question using the copy() for this operation. IMO, all you're doing is obfuscating your code. IMO using the iterative approach is much clearer.

Jim

**sean_cantab** · 11-30-2016

algorism, Jim: both your approaches work! Many thanks

algorism: I could (sort of) understand on the other thread (Issues with reading series of 2D matrices from file) why and where I had to check eof() but here I am still trying to figure out why I need to check eof() 'so early' right after the first getline(). I'd checked if(inFile) within main() that prevents the last line of output being printed twice but had not considered the eof() check within the operator overload itself. Yes, I should try and remember to practice the push_back + move idiom & file opening checks from now on
program: #10973439 - Pastie

Jim: indeed you're right, I'd not updated the iterator adapter stream to stringstream from ifstream once you'd suggested the stream change, my bad. I see both you and whiteflags have reservations about the copy() approach in this case and I agree with your observations. I was just trying to see what could be done and try and learn something in the process. Thanks again
program: #10973445 - Pastie

**whiteflags** · 11-30-2016

The only reason I can give why algorism's code works uses the following line of thinking. At the end of the file where there is no more content, we definitely want to fail. On the other hand, failing because copy experienced a reading error is something that we expect and will handle.

That leads to a series of steps like this:
1. try to get a name
2. if step 1 was successful:
2a. copy integers to the person's vector
2b. clear the read error from copy.
3. return the istream

Now you might be wondering why you can't just write something like:

Code:

#include <iostream>
#include <vector>
#include <string>
#include <iterator>
#include <algorithm>
using namespace std;

struct Person
{
    string name;
    vector<int> vec;
};

istream& operator >> (istream& is, Person& p)
{
   getline(is, p.name, ';');
   istream_iterator<int> is_get(is), eos;
   copy(is_get, eos, back_inserter(p.vec));
   if (!is.eof()) // deliberately wrong place to check eof
      is.clear();
   return is;
}

int main()
{
    stringstream fakefile;
    fakefile << "John Smith; 45 98 12\n";
    fakefile << "Jane Doe; 14 76 98\n";
    fakefile << "A N Other; 78 97 34\n";

    Person p;
    while (fakefile >> p) {
        cout << p.name << " ";
        cout << p.vec[0] << ", " << p.vec[1] << ", " << p.vec[2] << endl;
    }
}

result:
John Smith 45, 98, 12
Jane Doe 45, 98, 12

It's just not equivalent. I'd go as far as to say that testing after getline is where you must test if you were married to this approach. The reason is because istreams will read files top-down, and it makes sense that copy would encounter eof before anything else, because of the last complete record on the file. You are planning on ignoring the signal the first time, and finally failing when getline() tries a subsequent read.

I hope that was clear.

**algorism** · 11-30-2016

Originally Posted by whiteflags

The only reason I can give why algorism's code works uses the following line of thinking.

That was my reasoning exactly.

However, I should have said that this approach seems a little wacky to me. I would definitely read the entire line first and then pick it apart without the use of copy.

And in general you should always check for eof/errors directly after every i/o operation.

**sean_cantab** · 12-01-2016

Where exactly does istream stumble after reading the first object? - whiteflags says (@ #11) "istreams will read files top-down and it makes sense that copy would encounter eof before anything else" and jim says (@ #6) "copy will not stop until it encounters a stream error, which in this case will happen when it tries to read the first character of the next line" . The vocal failure report I'd added @ #4 seem to be more in line with Jim's suggestion. That's why I'm struggling to see (a) how istream reaches eof() after the first read and, if so, (b) how just checking for eof() is sufficient to make the program work as we'd done istream.clear() in any case even w/o the eof() check
I'm sorry, my questions might sound very silly to you experienced programmers but I could be missing something fundamental and would like to try and clear it up if possible. Many thanks

**whiteflags** · 12-01-2016

Originally Posted by sean_cantab

Where exactly does istream stumble after reading the first object? - whiteflags says (@ #11) "istreams will read files top-down and it makes sense that copy would encounter eof before anything else" and jim says (@ #6) "copy will not stop until it encounters a stream error, which in this case will happen when it tries to read the first character of the next line" . The vocal failure report I'd added @ #4 seem to be more in line with Jim's suggestion. That's why I'm struggling to see (a) how istream reaches eof() after the first read and, if so, (b) how just checking for eof() is sufficient to make the program work as we'd done istream.clear() in any case even w/o the eof() check
I'm sorry, my questions might sound very silly to you experienced programmers but I could be missing something fundamental and would like to try and clear it up if possible. Many thanks

I wasn't trying to argue with what Jim said when I posted. If you remember the post I made a little earlier I attempted to show why copy would eventually experience an error.

Originally Posted by whiteflags

The problem is that istream iterator on some level uses operator>>. When you read integers with copy the only thing that will stop it is EOF or a reading error.
So what happens is you'll read all of this and error out here because istream_iterator is still expecting integers.
John Smith; 45 98 12
Jane Doe; 14 76 98

So, imagine this is being read by algorithm's code. The "John Smith" part is read by the getline() statement. Since that succeeded, we do the copy line. The copy will read all of the numbers (on some level with operator >>) and fail when it reaches the J in "Jane". This is the error that Jim and I were talking about. In algorism's code, the error is recovered by calling clear().

All I wanted to add by explaining algorism's method was that there was an error that you cannot expect to recover from - the end of the file. For a tangible example, imagine that the Jane line is the last line in the file. Well, the getline() statement reads "Jane Doe". Since that succeeded, the copy will read the remaining numbers. Copy will keep going until it encounters EOF. Using the same logic as before, clear() will erase the initial signal.

This let's you continue the loop and process Jane Doe as a person. The next iteration though, getline() will also encounter EOF. Algorism's check will fail here, and break the loop. This is the expected behavior.

**jimblumberg** · 12-01-2016

That's why I'm struggling to see (a) how istream reaches eof() after the first read

With the original code the stream never reaches eof(), it has a read failure (failbit is set) after the first read. These are two distinct failure modes, checking for eof() will not detect the failbit, and checking fail() will not detect eof().

Part of your problem is that you're not properly determining the cause of the read failure, even the latest code that uses the string stream is really not determining the cause of the read fail. You really should be insuring that the failure was that you encountered eof() when using the string stream and didn't have one of the other stream failures. If, when using the stringstream, you encounter a read error other than eof() it means that you have a probable corrupt data file.

Jim

Thread: Initialize vectors from streams via itr adapters

Thread Tools

Search Thread

Display

Initialize vectors from streams via itr adapters

Similar Threads

Can't get Blutooth adapters working

Vectors in vectors - pointers and iterators

initialize vectors

Getting Information about Network Adapters

help with streams

Tags for this Thread