Serializing a Vector

[M.Richard Tober]

Before someone hits me with a "Use Boost" - know that I want to know how to do this myself. And I've had a few beers. Here it is:

// Serializing a vector -
// Why I shouldn't read Reddit while watching youtube CompSci videos

#include <iostream>
#include <fstream>
#include <vector>

using namespace std;

struct vec3d
{
    vec3d() {};
    vec3d(double x, double y, double z) {this->x = x; this->y = y; this->z = z;}
    virtual ~vec3d() {}
    double x;
    double y;
    double z;
};

typedef vector<vec3d> PointList;

int main()
{
    cout << "Structure" << endl;
    cout << "---------" << endl;

    PointList Points;
    vec3d P1(1.99, 5.0,  9.0);      Points.push_back(P1);
    vec3d P2(2.0,  6.0,  10.555);   Points.push_back(P2);
    vec3d P3(3.0,  7.55, 11.0);     Points.push_back(P3);
    vec3d P4(4.5,  8.0,  12.0);     Points.push_back(P4);

    // Searializing struct to point.dat
    ofstream os("points.dat", ios::binary);
    if(!os.is_open()) {
        cout << "Error opening points.dat for write" << endl;
        return 1;
    }

    //int magicnumber = sizeof(Points);

    os.write((char*)&Points, sizeof(Points));
    os.close();


    PointList NewPoints;
    int file_sz;

    // Reading from it
    ifstream is("points.dat", ios::binary);
    if(!is.is_open()) {
        cout << "Error opening points.dat for read" << endl;
        return 1;
    }
    is.seekg(0, ios::end);
    file_sz = is.tellg();
    is.seekg(0, ios::beg);

    //is.read((char*)&NewPoints, magicnumber);
    NewPoints.resize(file_sz);
    is.read((char*)&NewPoints, file_sz);

    // Display
    cout << "Output:" << endl;
    PointList::const_iterator it;

    for(it = NewPoints.begin(); it != NewPoints.end(); ++it)
    {
        cout << it->x << " " << it->y << " " << it->z << endl;
    }
    return 0;
}

So, I seem to get the data I want, but then there's what looks like a seg fault. Big old Glibc error.

*** glibc detected *** /storage/Projects/Recent/File Input and Output/bin/Debug/File Input and Output: corrupted double-linked list: 0x0000000000866110 ***

Sounds like a cool error.

Now, I would think the size of the file would suffice as the size of the NewPoints vector. But there's padding and other stuff going on, I'm sure. If someone has time, show mercy - waxing educational is welcome.

Writing and reading a vector to a binary file should not be that hard, and it probably isn't - but c++ is an expert's language... and I'm not there yet.

Side note: I picked up Scott Meyers Effective C++, I love it. Very approachable style, not nearly as dry as Koenig & Moo.

Edit: I know that's a c-style ungreppable cast, and I hate it too. But the C++ static_cast wouldn't even compile. I'm so ashamed, so very
ashamed.

Edit 2: Reading Salem's recommended link
[36] Serialization and Unserialization Updated! , C++ FAQnow... =)

[M.Richard Tober]

Until my brain gets bigger, this text mode solution will have to do:

// Serializing a vector -
// Why I shouldn't read Reddit while watching youtube CompSci videos

#include <iostream>
#include <fstream>
#include <vector>
#include <string>

using namespace std;

struct vec3d
{
    double x;
    double y;
    double z;
};

typedef vector<vec3d> PointList;
PointList::iterator it;

int main()
{
    /// Blab
    cout << "Structure" << endl;
    cout << "---------" << endl;

    /// Data
    PointList Points;
    vec3d P1 = {1.99, 5.0,  9.0   };    Points.push_back(P1);
    vec3d P2 = {2.0,  6.0,  10.555};    Points.push_back(P2);
    vec3d P3 = {3.0,  7.55, 11.0  };    Points.push_back(P3);
    vec3d P4 = {4.5,  8.0,  12.0  };    Points.push_back(P4);

    /// Open File
    ofstream os("points.dat");
    if(!os.is_open()) {
        cout << "Error opening points.dat for write" << endl;
        return 1;
    }
    else {
        cout << "Opened points.dat for write" << endl;
        /// Write Data
        for(it = Points.begin(); it != Points.end(); ++it) {
            os << " " << it->x << " " << it->y << " " << it->z;
        }
        os.close();
    }

    /// Open File
    PointList NewPoints;
    ifstream is("points.dat");
    if(!is.is_open()) {
        cout << "Error opening points.dat for read" << endl;
        return 1;
    }
    else {
        cout << "Opened points.dat for read" << endl;
        vec3d *vec_ptr = new vec3d;
        while(!is.eof()) {
            is >> vec_ptr->x; is >> vec_ptr->y; is >> vec_ptr->z;
            NewPoints.push_back(*vec_ptr);
        }
        is.close();
    }

    /// Display
    cout << "Output:" << endl;
    for(it = NewPoints.begin(); it != NewPoints.end(); ++it) {
        cout << it->x << " " << it->y << " " << it->z << endl;
    }
    return 0;
}

[jimblumberg]

One of the things I noticed in your first program was:

os.write((char*)&Points, sizeof(Points));

This is incorrect, on my machine sizeof(Points) evaluates to 12 which is not actually the correct size for your class. You should probably be using sizeof(vec3d) for one element, this evaluates to 28 on my machine. So to write the whole vector you would need to write (sizeof(vec3d) * Points.size()) bytes.

However when I tried to write out the entire vector and then read in the entire vector I got the same error you received. When I wrote each element of the vector separately and the read each element separately it worked.

// Serializing a vector -
// Why I shouldn't read Reddit while watching youtube CompSci videos

#include <iostream>
#include <fstream>
#include <vector>

using namespace std;

struct vec3d
{
    vec3d() : x(0.0), y(0.0), z(0.0) {};
    vec3d(double xa, double ya, double za) : x(xa), y(ya), z(za) {}
    virtual ~vec3d() {}
    double x;
    double y;
    double z;
};

typedef vector<vec3d> PointList;

int main()
{
    cout << "Structure" << endl;
    cout << "---------" << endl;

    PointList Points;

    vec3d P1(1.99, 5.0,  9.0);      Points.push_back(P1);
    vec3d P2(2.0,  6.0,  10.555);   Points.push_back(P2);
    vec3d P3(3.0,  7.55, 11.0);     Points.push_back(P3);
    vec3d P4(4.5,  8.0,  12.0);     Points.push_back(P4);

    // Searializing struct to point.dat

    ofstream os("points.dat", ios::binary);
    if(!os.is_open()) {
        cout << "Error opening points.dat for write" << endl;
        return 1;
    }
   for(size_t i = 0; i < Points.size(); ++i)
   {
      os.write((char*)&Points[i], sizeof(vec3d));
   }

    os.close();


    PointList NewPoints;

    int file_sz;

    // Reading from it
    ifstream is("points.dat", ios::binary);
    if(!is.is_open()) {
        cout << "Error opening points.dat for read" << endl;
        return 1;
    }


    is.seekg(0, ios::end);
    file_sz = is.tellg();
    is.seekg(0, ios::beg);
    NewPoints.resize(file_sz/sizeof(vec3d));

    for(size_t i = 0; i < NewPoints.size();++i)
      is.read((char*)&NewPoints[i], sizeof(vec3d));



    // Display
    cout << "Output:" << endl;
    PointList::const_iterator it;

    for(it = NewPoints.begin(); it != NewPoints.end(); ++it)
    {
        cout << it->x << " " << it->y << " " << it->z <<  endl;
    }

    return 0;
}

Jim

[hk_mp5kpdw]

One option is that you can overload the stream insertion/extraction operators (>>/<<) for both the vec3d struct and the PointList container:

#include <iostream>
#include <fstream>
#include <vector>
#include <algorithm>
#include <iterator>

using namespace std;

struct vec3d
{
    vec3d() : x(0.0), y(0.0), z(0.0) {};
    vec3d(double xa, double ya, double za) : x(xa), y(ya), z(za) {}
    virtual ~vec3d() {}
    double x;
    double y;
    double z;
};

// Overload vec3d stream insertion operator
ostream& operator<<(ostream& lhs, const vec3d& rhs)
{
    return lhs << rhs.x << ' ' << rhs.y << ' ' << rhs.z;
}

// Overload vec3d stream extraction operator
istream& operator>>(istream& lhs, vec3d& rhs)
{
    return lhs >> rhs.x >> rhs.y >> rhs.z;
}

typedef vector<vec3d> PointList;

// Overload PointList stream insertion operator
ostream& operator<<(ostream& lhs, const PointList& rhs)
{
    copy(rhs.begin(),rhs.end(),ostream_iterator<vec3d>(lhs,"\n"));
    return lhs;
}

// Overload PointList stream extraction operator
istream& operator>>(istream& lhs, PointList& rhs)
{
    copy(istream_iterator<vec3d>(lhs),istream_iterator<vec3d>(),
         back_inserter(rhs));
    return lhs;
}

int main()
{
    /////////////////////////////////////
    // Test output of PointList container
    PointList points_out;
    ofstream outfile("test.txt");

    // Fill points_out container
    points_out.push_back( vec3d(1.99, 5.0,  9.0)    );
    points_out.push_back( vec3d(2.0,  6.0,  10.555) );
    points_out.push_back( vec3d(3.0,  7.55, 11.0)   );
    points_out.push_back( vec3d(4.5,  8.0,  12.0)   );

    // Write all the vec3d objects in the points_out container to outfile
    outfile << points_out;
    outfile.close();

    ////////////////////////////////////
    // Test input of PointList container
    PointList points_in;
    ifstream infile("test.txt");

    // Read from infile into our points_in container
    infile >> points_in;

    // Now spit what we just read into points_in back to console
    cout << points_in;

    return 0;
}

[gardhr]

A wholesale binary write would only work IFF the object and all of it's sub-objects contained no pointers. An std::vector, for example, is basically composed of a pointer to a chunk of bytes and a length parameter. Writing it out in binary is useless because the actual data itself is untouched in the serialization process. Iterating through the vector to binary-write each element indeed works in this case because a vec3d contains no pointers, but otherwise you'd run into the same issue. The easiest and most portable way to serialize data is to use text IO. I'd stick with that, if I were you.

[M.Richard Tober]

Thank you jim, hk, and gardhr - excellent solutions and advice. Functional, and educational!