Thread: Storing columns of numerical data from csv files to a vector

  1. #1
    Registered User
    Join Date
    Mar 2011
    Posts
    12

    Storing columns of numerical data from csv files to a vector

    Hello, I have a csv file with two columns, one contains the date and time and the other contains numerical data. How would I get column two only (numerical data) stored in a vector?

    I currently have:
    Code:
    .
    .
    .
    int main(int argc, char *argv[])
    {
    	string filename, line;
    
    	cout<<"Enter the filename of the wind speed data "<<endl;
    	getline (cin,filename);
    
    	ifstream inFile;
    	inFile.open(filename.c_str());
    
    	if (!inFile){
    		cout<<"Error opening - " <<filename<<endl;
    		return -1;
    	}
    
    	while(!inFile.eof())
    	{
    		getline(inFile,line);
    		stringstream lineStream(line);
    		string bit;
    		vector<string>DataStore;
    		
    while(getline(lineStream, bit, ',')){
    			DataStore.push_back(bit);
    		}
    
    		inFile.close();
    .
    .
    .
    As you might expect, this stores the entire row (including unwanted data from column one) as opposed to just the data in column two.

    Any help or ideas greatly appreciated!

  2. #2
    Registered User hk_mp5kpdw's Avatar
    Join Date
    Jan 2002
    Location
    Northern Virginia/Washington DC Metropolitan Area
    Posts
    3,817
    #1
    Code:
    while(!inFile.eof())
    {
        getline(inFile,line);
        ...
    Don't use an eof test to control your loop. Test the result of the getline call itself directly instead.
    Code:
    while(getline(inFile,line))
    {
        ...

    #2
    Code:
    while(!inFile.eof())
    {
        getline(inFile,line);
        ...
        infile.close();
    This outer loop would only execute once because it closes the file. As such it looks like it would only read the first line of the file.


    #3
    Code:
    while(!inFile.eof())
    {
        getline(inFile,line);
        stringstream lineStream(line);
        string bit;
        vector<string>DataStore;
    		
        while(getline(lineStream, bit, ',')){
            DataStore.push_back(bit);
        }
    I find it curious that your vector is local to the outer loop. Is this intentional? It does not feel right to me.


    #4 As for your question, if you truly only wish to store the second column, one option would be to just call getline twice and only store the second string:
    Code:
    string bit;
    vector<string>DataStore;
    		
    getline(lineStream, bit, ',');  // Get/throw away first column
    getline(lineStream, bit, ',');  // Get/keep second column
    DataStore.push_back(bit);  // Store second column in vector
    "Owners of dogs will have noticed that, if you provide them with food and water and shelter and affection, they will think you are god. Whereas owners of cats are compelled to realize that, if you provide them with food and water and shelter and affection, they draw the conclusion that they are gods."
    -Christopher Hitchens

  3. #3
    Registered User
    Join Date
    Mar 2011
    Posts
    12
    Thanks a lot, fantastic reply. You are quite right on all points, I've adjusted my program accordingly.

    I tried your method for storing the column. It gets rid of the first column which is great, but only stores every second number in the second column.

    Could that be down to the way I'm displaying the vector contents?

    Code:
    void print (string WindData)
    {
    	cout << WindData << ' ';
    }
    
    .
    .
    .
    
    for_each (DataStore.begin(), DataStore.end(), print);                     
    		cout << endl;
    Thanks again.

  4. #4
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    O_o

    No. The way you are displaying the contents is fine. It is a little verbose, but that's not a big deal.

    I didn't read the original source, but hk_mp5kpdw is probably assuming that you have more than two columns or just got the terminating condition wrong. Try changing the terminating character (third parameter) of the second `std::getline' call to newline.

    Soma

  5. #5
    Registered User
    Join Date
    Mar 2011
    Posts
    12
    Changed it to newline, same result I'm afraid.

  6. #6
    Registered User hk_mp5kpdw's Avatar
    Join Date
    Jan 2002
    Location
    Northern Virginia/Washington DC Metropolitan Area
    Posts
    3,817
    Repost your current code with the changes you've made thus far. Perhaps a small sample of what the data - a few lines from the CSV file - looks like.
    "Owners of dogs will have noticed that, if you provide them with food and water and shelter and affection, they will think you are god. Whereas owners of cats are compelled to realize that, if you provide them with food and water and shelter and affection, they draw the conclusion that they are gods."
    -Christopher Hitchens

  7. #7
    Registered User
    Join Date
    Mar 2011
    Posts
    12
    Code:
    .
    .
    .
    int main(int argc, char *argv[])
    {
    	string filename, line;
    	vector<string>DataStore;
    
    	cout<<"Enter the filename of the wind speed data "<<endl;
    	getline (cin,filename);
    
    	ifstream inFile;
    	inFile.open(filename.c_str());
    
    	if (!inFile){
    		cout<<"Error opening - " <<filename<<endl;
    		return -1;
    	}
    
    	while(getline(inFile,line))
    	{
    		getline(inFile,line);
    		stringstream lineStream(line);
    		string bit;
    
    		getline(lineStream, bit, ',');
    		getline(lineStream, bit, '\n');
    		DataStore.push_back(bit);  
    
    	}
    .
    .
    .

    Here is a sample of the data. In column one there is something like this:

    Sat Aug 11 20:00:00 BST 2001
    -99999
    Sat Aug 11 22:00:00 BST 2001
    Sat Aug 11 23:00:00 BST 2001
    Sun Aug 12 00:00:00 BST 2001

    And in column two:

    9.774444436
    -99999
    8.231111104
    7.71666666
    8.231111104


    Like I said, the first column is removed which is good, but only every second number from the second column is currently being stored in the vector.

    Thanks!
    Last edited by stobbz; 03-17-2011 at 08:19 AM.

  8. #8
    Registered User
    Join Date
    May 2010
    Posts
    4,632
    In the following snippet:
    Code:
    	while(getline(inFile,line))
    	{
    		getline(inFile,line);
    		stringstream lineStream(line)
    you do not need the second getline().

    Jim

  9. #9
    Registered User
    Join Date
    Mar 2011
    Posts
    12
    Ahh, quite right jimblumberg, thanks :-)

    Thanks for dragging me through this everybody, it's finally working.

    Cheers!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. comparing data in two files
    By nynicue in forum C Programming
    Replies: 25
    Last Post: 06-18-2009, 07:35 PM
  2. Storing and accessing data from and to .txt files.
    By Glauber in forum C++ Programming
    Replies: 9
    Last Post: 05-27-2008, 02:59 PM
  3. Bitmasking Problem
    By mike_g in forum C++ Programming
    Replies: 13
    Last Post: 11-08-2007, 12:24 AM
  4. Adding files of numerical data
    By Boucho in forum C Programming
    Replies: 4
    Last Post: 02-06-2006, 05:27 PM
  5. Operators for 3D Vector Mathematics
    By Anarchist in forum C++ Programming
    Replies: 10
    Last Post: 01-31-2003, 07:33 PM