Serializing a Vector

This is a discussion on Serializing a Vector within the C++ Programming forums, part of the General Programming Boards category; Before someone hits me with a "Use Boost" - know that I want to know how to do this myself. ...

  1. #1
    C++ Enthusiast M.Richard Tober's Avatar
    Join Date
    May 2011
    Location
    Georgia
    Posts
    56

    Serializing a Vector

    Before someone hits me with a "Use Boost" - know that I want to know how to do this myself. And I've had a few beers. Here it is:

    Code:
    // Serializing a vector -
    // Why I shouldn't read Reddit while watching youtube CompSci videos
    
    #include <iostream>
    #include <fstream>
    #include <vector>
    
    using namespace std;
    
    struct vec3d
    {
        vec3d() {};
        vec3d(double x, double y, double z) {this->x = x; this->y = y; this->z = z;}
        virtual ~vec3d() {}
        double x;
        double y;
        double z;
    };
    
    typedef vector<vec3d> PointList;
    
    int main()
    {
        cout << "Structure" << endl;
        cout << "---------" << endl;
    
        PointList Points;
        vec3d P1(1.99, 5.0,  9.0);      Points.push_back(P1);
        vec3d P2(2.0,  6.0,  10.555);   Points.push_back(P2);
        vec3d P3(3.0,  7.55, 11.0);     Points.push_back(P3);
        vec3d P4(4.5,  8.0,  12.0);     Points.push_back(P4);
    
        // Searializing struct to point.dat
        ofstream os("points.dat", ios::binary);
        if(!os.is_open()) {
            cout << "Error opening points.dat for write" << endl;
            return 1;
        }
    
        //int magicnumber = sizeof(Points);
    
        os.write((char*)&Points, sizeof(Points));
        os.close();
    
    
        PointList NewPoints;
        int file_sz;
    
        // Reading from it
        ifstream is("points.dat", ios::binary);
        if(!is.is_open()) {
            cout << "Error opening points.dat for read" << endl;
            return 1;
        }
        is.seekg(0, ios::end);
        file_sz = is.tellg();
        is.seekg(0, ios::beg);
    
        //is.read((char*)&NewPoints, magicnumber);
        NewPoints.resize(file_sz);
        is.read((char*)&NewPoints, file_sz);
    
        // Display
        cout << "Output:" << endl;
        PointList::const_iterator it;
    
        for(it = NewPoints.begin(); it != NewPoints.end(); ++it)
        {
            cout << it->x << " " << it->y << " " << it->z << endl;
        }
        return 0;
    }
    So, I seem to get the data I want, but then there's what looks like a seg fault. Big old Glibc error.
    Code:
    *** glibc detected *** /storage/Projects/Recent/File Input and Output/bin/Debug/File Input and Output: corrupted double-linked list: 0x0000000000866110 ***
    Sounds like a cool error.

    Now, I would think the size of the file would suffice as the size of the NewPoints vector. But there's padding and other stuff going on, I'm sure. If someone has time, show mercy - waxing educational is welcome.

    Writing and reading a vector to a binary file should not be that hard, and it probably isn't - but c++ is an expert's language... and I'm not there yet.

    Side note: I picked up Scott Meyers Effective C++, I love it. Very approachable style, not nearly as dry as Koenig & Moo.

    Edit: I know that's a c-style ungreppable cast, and I hate it too. But the C++ static_cast wouldn't even compile. I'm so ashamed, so very
    ashamed.

    Edit 2: Reading Salem's recommended link
    [36] Serialization and Unserialization ..Updated!.., C++ FAQnow... =)
    Last edited by M.Richard Tober; 11-22-2011 at 08:40 AM. Reason: Salem Saves The World
    Eventually, I decided that thinking was not getting me very far and it was time to try building.
    Rob Pike, "The Text Editor sam"

  2. #2
    C++ Enthusiast M.Richard Tober's Avatar
    Join Date
    May 2011
    Location
    Georgia
    Posts
    56

    Someday I suppose...

    Until my brain gets bigger, this text mode solution will have to do:
    Code:
    // Serializing a vector -
    // Why I shouldn't read Reddit while watching youtube CompSci videos
    
    #include <iostream>
    #include <fstream>
    #include <vector>
    #include <string>
    
    using namespace std;
    
    struct vec3d
    {
        double x;
        double y;
        double z;
    };
    
    typedef vector<vec3d> PointList;
    PointList::iterator it;
    
    int main()
    {
        /// Blab
        cout << "Structure" << endl;
        cout << "---------" << endl;
    
        /// Data
        PointList Points;
        vec3d P1 = {1.99, 5.0,  9.0   };    Points.push_back(P1);
        vec3d P2 = {2.0,  6.0,  10.555};    Points.push_back(P2);
        vec3d P3 = {3.0,  7.55, 11.0  };    Points.push_back(P3);
        vec3d P4 = {4.5,  8.0,  12.0  };    Points.push_back(P4);
    
        /// Open File
        ofstream os("points.dat");
        if(!os.is_open()) {
            cout << "Error opening points.dat for write" << endl;
            return 1;
        }
        else {
            cout << "Opened points.dat for write" << endl;
            /// Write Data
            for(it = Points.begin(); it != Points.end(); ++it) {
                os << " " << it->x << " " << it->y << " " << it->z;
            }
            os.close();
        }
    
        /// Open File
        PointList NewPoints;
        ifstream is("points.dat");
        if(!is.is_open()) {
            cout << "Error opening points.dat for read" << endl;
            return 1;
        }
        else {
            cout << "Opened points.dat for read" << endl;
            vec3d *vec_ptr = new vec3d;
            while(!is.eof()) {
                is >> vec_ptr->x; is >> vec_ptr->y; is >> vec_ptr->z;
                NewPoints.push_back(*vec_ptr);
            }
            is.close();
        }
    
        /// Display
        cout << "Output:" << endl;
        for(it = NewPoints.begin(); it != NewPoints.end(); ++it) {
            cout << it->x << " " << it->y << " " << it->z << endl;
        }
        return 0;
    }
    Eventually, I decided that thinking was not getting me very far and it was time to try building.
    Rob Pike, "The Text Editor sam"

  3. #3
    Registered User
    Join Date
    May 2010
    Posts
    2,680
    One of the things I noticed in your first program was:
    Code:
    os.write((char*)&Points, sizeof(Points));
    This is incorrect, on my machine sizeof(Points) evaluates to 12 which is not actually the correct size for your class. You should probably be using sizeof(vec3d) for one element, this evaluates to 28 on my machine. So to write the whole vector you would need to write (sizeof(vec3d) * Points.size()) bytes.

    However when I tried to write out the entire vector and then read in the entire vector I got the same error you received. When I wrote each element of the vector separately and the read each element separately it worked.

    Code:
    // Serializing a vector -
    // Why I shouldn't read Reddit while watching youtube CompSci videos
    
    #include <iostream>
    #include <fstream>
    #include <vector>
    
    using namespace std;
    
    struct vec3d
    {
        vec3d() : x(0.0), y(0.0), z(0.0) {};
        vec3d(double xa, double ya, double za) : x(xa), y(ya), z(za) {}
        virtual ~vec3d() {}
        double x;
        double y;
        double z;
    };
    
    typedef vector<vec3d> PointList;
    
    int main()
    {
        cout << "Structure" << endl;
        cout << "---------" << endl;
    
        PointList Points;
    
        vec3d P1(1.99, 5.0,  9.0);      Points.push_back(P1);
        vec3d P2(2.0,  6.0,  10.555);   Points.push_back(P2);
        vec3d P3(3.0,  7.55, 11.0);     Points.push_back(P3);
        vec3d P4(4.5,  8.0,  12.0);     Points.push_back(P4);
    
        // Searializing struct to point.dat
    
        ofstream os("points.dat", ios::binary);
        if(!os.is_open()) {
            cout << "Error opening points.dat for write" << endl;
            return 1;
        }
       for(size_t i = 0; i < Points.size(); ++i)
       {
          os.write((char*)&Points[i], sizeof(vec3d));
       }
    
        os.close();
    
    
        PointList NewPoints;
    
        int file_sz;
    
        // Reading from it
        ifstream is("points.dat", ios::binary);
        if(!is.is_open()) {
            cout << "Error opening points.dat for read" << endl;
            return 1;
        }
    
    
        is.seekg(0, ios::end);
        file_sz = is.tellg();
        is.seekg(0, ios::beg);
        NewPoints.resize(file_sz/sizeof(vec3d));
    
        for(size_t i = 0; i < NewPoints.size();++i)
          is.read((char*)&NewPoints[i], sizeof(vec3d));
    
    
    
        // Display
        cout << "Output:" << endl;
        PointList::const_iterator it;
    
        for(it = NewPoints.begin(); it != NewPoints.end(); ++it)
        {
            cout << it->x << " " << it->y << " " << it->z <<  endl;
        }
    
        return 0;
    }
    Jim

  4. #4
    Registered User hk_mp5kpdw's Avatar
    Join Date
    Jan 2002
    Location
    Northern Virginia/Washington DC Metropolitan Area
    Posts
    3,793
    One option is that you can overload the stream insertion/extraction operators (>>/<<) for both the vec3d struct and the PointList container:
    Code:
    #include <iostream>
    #include <fstream>
    #include <vector>
    #include <algorithm>
    #include <iterator>
    
    using namespace std;
    
    struct vec3d
    {
        vec3d() : x(0.0), y(0.0), z(0.0) {};
        vec3d(double xa, double ya, double za) : x(xa), y(ya), z(za) {}
        virtual ~vec3d() {}
        double x;
        double y;
        double z;
    };
    
    // Overload vec3d stream insertion operator
    ostream& operator<<(ostream& lhs, const vec3d& rhs)
    {
        return lhs << rhs.x << ' ' << rhs.y << ' ' << rhs.z;
    }
    
    // Overload vec3d stream extraction operator
    istream& operator>>(istream& lhs, vec3d& rhs)
    {
        return lhs >> rhs.x >> rhs.y >> rhs.z;
    }
    
    typedef vector<vec3d> PointList;
    
    // Overload PointList stream insertion operator
    ostream& operator<<(ostream& lhs, const PointList& rhs)
    {
        copy(rhs.begin(),rhs.end(),ostream_iterator<vec3d>(lhs,"\n"));
        return lhs;
    }
    
    // Overload PointList stream extraction operator
    istream& operator>>(istream& lhs, PointList& rhs)
    {
        copy(istream_iterator<vec3d>(lhs),istream_iterator<vec3d>(),
             back_inserter(rhs));
        return lhs;
    }
    
    int main()
    {
        /////////////////////////////////////
        // Test output of PointList container
        PointList points_out;
        ofstream outfile("test.txt");
    
        // Fill points_out container
        points_out.push_back( vec3d(1.99, 5.0,  9.0)    );
        points_out.push_back( vec3d(2.0,  6.0,  10.555) );
        points_out.push_back( vec3d(3.0,  7.55, 11.0)   );
        points_out.push_back( vec3d(4.5,  8.0,  12.0)   );
    
        // Write all the vec3d objects in the points_out container to outfile
        outfile << points_out;
        outfile.close();
    
        ////////////////////////////////////
        // Test input of PointList container
        PointList points_in;
        ifstream infile("test.txt");
    
        // Read from infile into our points_in container
        infile >> points_in;
    
        // Now spit what we just read into points_in back to console
        cout << points_in;
    
        return 0;
    }
    "Owners of dogs will have noticed that, if you provide them with food and water and shelter and affection, they will think you are god. Whereas owners of cats are compelled to realize that, if you provide them with food and water and shelter and affection, they draw the conclusion that they are gods."
    -Christopher Hitchens

  5. #5
    Registered User gardhr's Avatar
    Join Date
    Apr 2011
    Posts
    151
    A wholesale binary write would only work IFF the object and all of it's sub-objects contained no pointers. An std::vector, for example, is basically composed of a pointer to a chunk of bytes and a length parameter. Writing it out in binary is useless because the actual data itself is untouched in the serialization process. Iterating through the vector to binary-write each element indeed works in this case because a vec3d contains no pointers, but otherwise you'd run into the same issue. The easiest and most portable way to serialize data is to use text IO. I'd stick with that, if I were you.

  6. #6
    C++ Enthusiast M.Richard Tober's Avatar
    Join Date
    May 2011
    Location
    Georgia
    Posts
    56
    Thank you jim, hk, and gardhr - excellent solutions and advice. Functional, and educational!
    Eventually, I decided that thinking was not getting me very far and it was time to try building.
    Rob Pike, "The Text Editor sam"

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Serializing classes
    By Whyrusleeping in forum C++ Programming
    Replies: 48
    Last Post: 06-17-2011, 05:08 AM
  2. Serializing/deserializing problem
    By vsla in forum C Programming
    Replies: 3
    Last Post: 04-21-2008, 03:55 PM
  3. Simple Question memset(vector<int>, 0, sizeof(vector<int>))
    By HyperShadow in forum C++ Programming
    Replies: 6
    Last Post: 12-10-2007, 03:56 PM
  4. Serializing problem.. (can't use >> operator)
    By RancidWannaRiot in forum Windows Programming
    Replies: 2
    Last Post: 10-29-2005, 11:10 AM
  5. Serializing a class
    By Prog.Patterson in forum C++ Programming
    Replies: 4
    Last Post: 10-27-2005, 10:21 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21