Storing columns of numerical data from csv files to a vector

Hello, I have a csv file with two columns, one contains the date and time and the other contains numerical data. How would I get column two only (numerical data) stored in a vector?

I currently have:

Code:

.
.
.
int main(int argc, char *argv[])
{
	string filename, line;

	cout<<"Enter the filename of the wind speed data "<<endl;
	getline (cin,filename);

	ifstream inFile;
	inFile.open(filename.c_str());

	if (!inFile){
		cout<<"Error opening - " <<filename<<endl;
		return -1;
	}

	while(!inFile.eof())
	{
		getline(inFile,line);
		stringstream lineStream(line);
		string bit;
		vector<string>DataStore;
		
while(getline(lineStream, bit, ',')){
			DataStore.push_back(bit);
		}

		inFile.close();
.
.
.

As you might expect, this stores the entire row (including unwanted data from column one) as opposed to just the data in column two.

Any help or ideas greatly appreciated!

#1

Code:

while(!inFile.eof())
{
    getline(inFile,line);
    ...

Don't use an eof test to control your loop. Test the result of the getline call itself directly instead.

Code:

while(getline(inFile,line))
{
    ...

#2

Code:

while(!inFile.eof())
{
    getline(inFile,line);
    ...
    infile.close();

This outer loop would only execute once because it closes the file. As such it looks like it would only read the first line of the file.


#3

Code:

while(!inFile.eof())
{
    getline(inFile,line);
    stringstream lineStream(line);
    string bit;
    vector<string>DataStore;
		
    while(getline(lineStream, bit, ',')){
        DataStore.push_back(bit);
    }

I find it curious that your vector is local to the outer loop. Is this intentional? It does not feel right to me.


#4 As for your question, if you truly only wish to store the second column, one option would be to just call getline twice and only store the second string:

Code:

string bit;
vector<string>DataStore;
		
getline(lineStream, bit, ',');  // Get/throw away first column
getline(lineStream, bit, ',');  // Get/keep second column
DataStore.push_back(bit);  // Store second column in vector

Thanks a lot, fantastic reply. You are quite right on all points, I've adjusted my program accordingly.

I tried your method for storing the column. It gets rid of the first column which is great, but only stores every second number in the second column.

Could that be down to the way I'm displaying the vector contents?

Code:

void print (string WindData)
{
	cout << WindData << ' ';
}

.
.
.

for_each (DataStore.begin(), DataStore.end(), print);                     
		cout << endl;

Thanks again.

O_o

No. The way you are displaying the contents is fine. It is a little verbose, but that's not a big deal.

I didn't read the original source, but hk_mp5kpdw is probably assuming that you have more than two columns or just got the terminating condition wrong. Try changing the terminating character (third parameter) of the second `std::getline' call to newline.

Soma

Changed it to newline, same result I'm afraid.

Repost your current code with the changes you've made thus far. Perhaps a small sample of what the data - a few lines from the CSV file - looks like.

Code:

.
.
.
int main(int argc, char *argv[])
{
	string filename, line;
	vector<string>DataStore;

	cout<<"Enter the filename of the wind speed data "<<endl;
	getline (cin,filename);

	ifstream inFile;
	inFile.open(filename.c_str());

	if (!inFile){
		cout<<"Error opening - " <<filename<<endl;
		return -1;
	}

	while(getline(inFile,line))
	{
		getline(inFile,line);
		stringstream lineStream(line);
		string bit;

		getline(lineStream, bit, ',');
		getline(lineStream, bit, '\n');
		DataStore.push_back(bit);  

	}
.
.
.

Here is a sample of the data. In column one there is something like this:

Sat Aug 11 20:00:00 BST 2001
-99999
Sat Aug 11 22:00:00 BST 2001
Sat Aug 11 23:00:00 BST 2001
Sun Aug 12 00:00:00 BST 2001

And in column two:

9.774444436
-99999
8.231111104
7.71666666
8.231111104


Like I said, the first column is removed which is good, but only every second number from the second column is currently being stored in the vector.

Thanks!

In the following snippet:

Code:

	while(getline(inFile,line))
	{
		getline(inFile,line);
		stringstream lineStream(line)

you do not need the second getline().

Jim

Ahh, quite right jimblumberg, thanks :-)

Thanks for dragging me through this everybody, it's finally working.

Cheers!