Thread: Nested/recursive template specialization

I'm trying to write some naive binary serialization code and wanted to cut down on repetition of logic for serializing/deserializing nested vectors or other STL containers to reduce the chance of typos etc, and thought templates might hold a solution for me.

Apologies if some of my naming/conventions look like bastardized C#/Java, it's been a while since I've used C++.

Code:

template <typename T> void serializeField(IWriter& writer, const T& val) {
   writer.write((char*)&val, sizeof(T));
}

template<typename U, typename V>
template <> void serializeField(IWriter& writer, const U<V>& collection)
{
   serializeField(writer, (unsigned long)col.size());
   for (typename U<typename V>::const_iterator
           i = collection.begin();
           i != collection.end();
           ++i)
   {
      serializeField(writer, *i);
   }
}

With the goal of being able to do something like:

Code:

// main, or wherever
StlWriter myWriter(my_ofstream);

int val;
serializeField(myWriter, val);

vector<int> valVector;
serializeField(myWriter, valVector);

list<int> valList;
serializeField(myWriter, valList);

vector<vector<int> > nestedVector;
serializeField(myWriter, nestedVector);

Is there a way to do something like this? It isn't a big deal for me to just manually write code to serialize my vectors to the needed depth, but it sure would be nice to get this working.

I'm not sure you can quite get away with that, but you can come fairly close - instead of trying to use U, just call it a vector. Downside is you'd have to specialize for other containers, but by just specializing for e.g. vectors and list it'll (i.e. should) automagically do lists of vectors of lists (etc.) of stuff. Here's a working example:

Code:

#include <cstdio>
#include <vector>

using namespace std;

class IWriter
{
public:
	void write(char *ptr, size_t bytes) {
		printf("Write %d bytes at %p\n", bytes, ptr);
	}
};

template <typename T>
void serializeField(IWriter& writer, const T& val)
{
	puts("generic case");
	writer.write((char *)&val, sizeof(T));
}

template <typename U>
void serializeField(IWriter& writer, const vector<U>& vec)
{
	puts("in vector variant");
	for (typename vector<U>::const_iterator
		i = vec.begin() ; i != vec.end() ; i++) {
		serializeField(writer, *i);
	}
}

int main(void)
{
	IWriter writer;

	puts("\nvector<int>");
	vector<int> vect1(2);
	serializeField(writer, vect1);

	puts("\nvector< vector<int> >");
	vector< vector<int> > vect2(2);
	vect2[0] = vector<int>(2);
	vect2[1] = vector<int>(2);
	serializeField(writer, vect2);
}

Output:

Code:

vector<int>
in vector variant
generic case
Write 4 bytes at 0x99e7008
generic case
Write 4 bytes at 0x99e700c

vector< vector<int> >
in vector variant
in vector variant
generic case
Write 4 bytes at 0x99e7048
generic case
Write 4 bytes at 0x99e704c
in vector variant
generic case
Write 4 bytes at 0x99e7058
generic case
Write 4 bytes at 0x99e705c

Many thanks!

So it looks to me like the vector variant isn't actually a template specialization but a... templated overload of the template function?

You probably meant to overload the function instead of trying to partially specialize it (which is not allowed):

Code:

template<typename<typename> Container, typename V>
void serializeField(IWriter& writer, const Container<V>& col)
{
   serializeField(writer, col.size());
   for (auto & elem : col)
      serializeField(writer, elem);
}

Next you probably want to think about how to distinguish between collections and non-nollections. Right now, any template type will match (e.g. on some compilers, std::shared_ptr). But that's probably not what you want.
Also, I discourage you from using write/read. Use the stream operators to ensure it can be read and written properly.

Thanks Elysia,
>>Use the stream operators to ensure it can be read and written properly.
Is there something preventing read/write from operating correctly? I wrote it in this way because I need to adapt it to multiple platforms which don't all support fstreams.

I wasn't able to actually compile "template<typename<typename> Container, typename V>". Did you mean this:

Code:

template <template<typename> class Container, typename T>

My test code doesn't seem to invoke the container variant in this case.


In any case, JohnGraham's solution seems general enough for my purposes and also avoids the over-matching issue you mentioned, so I'll consider that part of the problem solved for now.

I'm now trying to meet a second goal, which is to make this work with classes derived from ISerializable to just call .serialize(writer). My initial thought was to just add another specialization or overload:

Code:

void serializeField(IWriter& writer, const ISerializable& ser) {
   ser.serialize(writer);
}

// or

template <> void serializeField(IWriter& writer, const ISerializable& ser) {
   ser.serialize(writer);
}

However, the compiler prefers the generic version over my overload/specialization, I'm assuming because it isn't an exact type match.

My second idea was to try and dynamic_cast<ISerializable*>(&val) in the generic version, but that causes compilation errors with POD types.

After that, I thought to use std::is_pod and an extra bool template parameter, and then specialize for T and is_pod=false, but that requires function partial specialization which as has been pointed out is not allowed.

Any other ideas?

Originally Posted by Hunter2

Is there something preventing read/write from operating correctly? I wrote it in this way because I need to adapt it to multiple platforms which don't all support fstreams.

fstreams is a C++ standard library container! What possible platform does not support this? That would mean that the compiler wouldn't actually be C++ compliant!
Using streams is more portable. You don't have to worry about endianess, etc.

I wasn't able to actually compile "template<typename<typename> Container, typename V>". Did you mean this:

You are correct. I didn't compile it, so it was an error on my part.

My test code doesn't seem to invoke the container variant in this case.

It did for me, though? Can you replicate a small example?

Any other ideas?

Code:

#include <type_traits>

struct IWriter {};
template<typename T> struct foo {};
struct BaseBar {};
struct DerivedBar: BaseBar {};

template <typename T> void SerializeFieldGeneric(IWriter& writer, const T& val);
template <typename T> void SerializeFieldDispatcher(IWriter& writer, const T& val, std::true_type);
template <typename T> void SerializeFieldDispatcher(IWriter& writer, const T& val, std::false_type);
template <typename T> void SerializeField(IWriter& writer, const T& val);
template<template<typename> class Container, typename V> void SerializeField(IWriter& writer, const Container<V>& col);
void SerializeFieldDerived(IWriter& writer, const BaseBar& val);

template <typename T>
void SerializeFieldGeneric(IWriter&, const T&) { std::cout << "SerializeFieldGeneric<T>\n"; }

template <typename T>
void SerializeFieldDispatcher(IWriter& writer, const T& val, std::true_type) { SerializeFieldDerived(writer, val); } // Is derived type

template <typename T>
void SerializeFieldDispatcher(IWriter& writer, const T& val, std::false_type) { SerializeFieldGeneric(writer, val); } // Is not derived type

template <typename T>
void SerializeField(IWriter& writer, const T& val) { SerializeFieldDispatcher(writer, val, std::is_base_of<BaseBar, T>()); }

template<template<typename> class Container, typename V>
void SerializeField(IWriter&, const Container<V>&) { std::cout << "SerializeField<Container<V>>\n"; }

void SerializeFieldDerived(IWriter&, const BaseBar&) { std::cout << "SerializeField<BaseBar>\n"; }

int main()
{
	int t1;
	foo<int> t2;
	IWriter writer;
	BaseBar Base;
	DerivedBar Derived;

	SerializeField(writer, t1);
	SerializeField(writer, t2);
	SerializeField(writer, Base);
	SerializeField(writer, Derived);
}

Output:
SerializeFieldGeneric<T>
SerializeField<Container<V>>
SerializeField<BaseBar>
SerializeField<BaseBar>

fstreams is a C++ standard library container! What possible platform does not support this? That would mean that the compiler wouldn't actually be C++ compliant!

I was imprecise Plat A = iOS, which is fine; Plat B = Android, where app assets are exposed via a read interface on AAssetManager. I could write my own streambuf or whatnot, but this seemed like the more straightforward solution.

Using streams is more portable. You don't have to worry about endianess, etc.

Are you referring to "<<", i.e. serializing as text?.. That seems like it would come with its own set of potential issues. Granted, your point of endianness hits a bit too close to home, but for now that hasn't been an issue and I'm looking for a quick and dirty solution to get 'something' running.

Code:

void SerializeFieldDispatcher(IWriter& writer, const T& val, std::true_type)

Ahh, I see what you did there. Except, isn't that pretty much just partial specialization, without the generic template? Not that I'm complaining.
**EDIT - oops, I see that I didn't actually see what was going on after all. Argument overload, not template - brilliant!

stuff

TBD after more poking around

Code:

#include <type_traits>
#include <iostream>
#include <vector>

struct IWriter {};
template<typename T> struct foo {};
template<typename T> class foo2 {};
struct BaseBar {};
struct DerivedBar: BaseBar {};

template <typename T> void SerializeFieldGeneric(IWriter& writer, const T& val);
template <typename T> void SerializeFieldDispatcher(IWriter& writer, const T& val, std::true_type);
template <typename T> void SerializeFieldDispatcher(IWriter& writer, const T& val, std::false_type);
template <typename T> void SerializeField(IWriter& writer, const T& val);
template<template<typename> class Container, typename V> void SerializeField(IWriter& writer, const Container<V>& col);
void SerializeFieldDerived(IWriter& writer, const BaseBar& val);

template <typename T>
void SerializeFieldGeneric(IWriter&, const T&) { std::cout << "SerializeFieldGeneric<T>\n"; }

template <typename T>
void SerializeFieldDispatcher(IWriter& writer, const T& val, std::true_type) { SerializeFieldDerived(writer, val); } // Is derived type

template <typename T>
void SerializeFieldDispatcher(IWriter& writer, const T& val, std::false_type) { SerializeFieldGeneric(writer, val); } // Is not derived type

template <typename T>
void SerializeField(IWriter& writer, const T& val) { SerializeFieldDispatcher(writer, val, std::is_base_of<BaseBar, T>()); }

template<template<typename> class Container, typename V>
void SerializeField(IWriter&, const Container<V>&) { std::cout << "SerializeField<Container<V>>\n"; }

void SerializeFieldDerived(IWriter&, const BaseBar&) { std::cout << "SerializeField<BaseBar>\n"; }

int main()
{
    int t1;
    foo<int> t2;
    std::vector<int> t3;
    foo2<int> t4;
    IWriter writer;
    BaseBar Base;
    DerivedBar Derived;
    
    SerializeField(writer, t1);
    SerializeField(writer, t2);
    SerializeField(writer, t3);
    SerializeField(writer, t4);
    SerializeField(writer, Base);
    SerializeField(writer, Derived);
}

Output:
SerializeFieldGeneric<T>
SerializeField<Container<V>>
SerializeFieldGeneric<T>
SerializeField<Container<V>>
SerializeField<BaseBar>
SerializeField<BaseBar>

Anyway, I have a working solution now via the explicit overloads for vector and list, and the magickery of std::is_base_of.
Thanks a bunch to both of you!

Containers are tricky because they do not take one template argument. They do tend to take at least two: the type and the allocator type. Because of this, they do not match Container<V>.
I don't know if it's specified in the standard exactly how many parameters they take and how the signature looks like, though.

EDIT:

Code:

template<template<typename, typename> class Container, typename V, typename A>
void SerializeField(IWriter&, const Container<V, A>&) { std::cout << "SerializeField<Container<V, A>>\n"; }

This will capture vector on Visual C++. May or may not work on other compilers.

Originally Posted by Elysia

Containers are tricky because they do not take one template argument. They do tend to take at least two: the type and the allocator type. Because of this, they do not match Container<V>.

Idea: Variadic templates might be of use here:

Code:

template<template<typename...> class Container, typename T, typename Ts...>
void SerializeField(IWriter&, const Container<T, Ts...>&) { std::cout << "SerializeField<Container<T>>\n"; }

For reference, Phantomotap had some great comments on this in a thread of mine a few weeks back.

This approach would require a C++11 compliant compiler for each of the target platforms, which might be a tall order.

For reference, Phantomotap had some great comments on this in a thread of mine a few weeks back.

^_^

I'm glad you said something; you disappeared after the post and I wasn't sure if you knew of my comments.

I don't know if it's specified in the standard exactly how many parameters they take and how the signature looks like, though.

You are, in simply checking for "containerness", expecting the type to provide a specific interface.

You don't know how many, if any, parameters a "container" expects; you only need to inspect the availability of the interface you intend to use.

Once you know you can use the interface you expect, you code the implementation according to that interface.

[Edit]
I updated the code; a simple bug, a poorly copied example, from an earlier thread.
[/Edit]

Soma

Code:

/******************************************************************************/

struct IWriter; // The interface for `IWriter' may be whatever you require.

/******************************************************************************/

template
<
    typename FWriter
>
struct SWriterAdapter:
    public IWriter
{
    // You would container a `FWriter'.
    // You would stabilize any `FWriter' to a `IWriter' interface.
};

/******************************************************************************/

// The next few tools are written "out-of-order" to show the layers of implementation.

template
<
    typename FWriter // We don't need any inheritance; many things may model a `Writer'/`Reader'.
  , typename FValue
>
void Serialize
(
    FWriter & fWriter
  , const FValue & fValue
)
{
    // The `is_collection' metafunction is omitted for brevity.
    // The `contain_writer' metafunction is omitted for brevity.
    // This does not require dynamic allocation.
    // The `contain_writer' metafunction "returns" a `IWriter &'
    // if and only if `fWriter' already "is-constrainable-as"
    // an `IWriter &'; otherwise `contain_writer' "returns" a
    // `SWriterAdapter' which itself "is-constrainable-as" an
    // `IWriter &'.
    contain_writer<FWriter>::type sWriter(fWriter);
    Serialize(sWriter, fValue, is_container<FValue>());
}

/******************************************************************************/

/******************************************************************************/

template
<
    typename FValue
>
void Serialize
(
    IWriter & fWriter // The `Serialize' template interface adapts the `FWriter'.
  , const FValue & fValue
  , std::true_type // We leave this "unnamed" to prevent compiler warnings.
)
{
    // The `iterator_for' metafunction is omitted for brevity.
    // The `begin'/`end' functions are omitted for brevity.
    // The `begin'/`end' functions are standard C++11 conforming.
    // We know we have a "collection" which conforms to our expections.
    // We can just use the interface regardless of the actual type of `FValue'.
    const_iterator_for<FValue> sOrigin(begin(fValue));
    const_iterator_for<FValue> sTerminus(end(fValue));
    while(sOrigin != sTerminus)
    {
        // This is where the "magic" happens.
        // The function `Serialize' calls an overload that resolves
        // to `Serialize(IWriter &, const FValue &);' for any `FValue'
        // even if that happens to be the `Serialize' template interface.
        Serialize(fWriter, *sOrigin);
        ++sOrigin;
    }
}

/******************************************************************************/

struct MyClass{}; // This object requires special handling.

struct MyClass2: public MyClass{};

void Serialize // I provide that behavior here.
(
    IWriter & fWriter
  , const MyClass & fValue // A call for `MyClass' or `MyClass2' "just works".
  , std::false_type = false_type();
)
{
    // Implement whatever behavior is required for `MyClass'.
    // This function is called directly by overload resolution
    // when `Serialize(IWriter &, const MyClass &);' is called;
    // otherwise the `Serialize' template interface adapts
    // the behavior of the first parameter and calls this function.
}

/******************************************************************************/