Good way to construct packets

A lot of communication is done by creating packets (for sockets/IPC etc). Often you end up with variable sized packets for example strings in it. For example.

Code:

struct StringPacketBase
{
protected:
int stringLen;
};

Then the name comes after the struct. If you do a class of this

Code:

class StringPacket : public StringPacketBase
{
public:
uint getLength { return stringLen; }
char *GetString { return (char*)this + sizeof(StringPacketBase) }
};

I wonder if there is a better way to automate packet creation classes. Also a problem is that the string will never show up in debugging information.

I've seen this:

Code:

struct StringPacketBase
{
protected:
int stringLen;
char string[];
};

Now string is a part of the struct but what happens if you want two strings after each other? Then this method only work for the first string. Also if you use sizeof with StringPacketBase you have to be more careful now.

The question is how you can create packets so that it becomes easy to write. Right now you do a lot of calculations with offsets and I wonder if there is a method make writing all these packets (which is a lot packets) more conveniently.

So what is it that you really need to do in order for you to have packet classes?
Typical network protocols (well, TCP, at least) will chunk up the data for you.

It's simple - you don't create a structure to represent the data that is sent over the wire. Look into serialisation / deserialisation instead.

I think I'm missing the plot here; are you deliberately not storing the data in your Packet class, but instead relying on declaring it directly after and hoping that the compiler puts those two variables next to each other and in the right order? Why not -- and this is crazy I'm sure -- why not put the string inside the class, and then provide a create_packet method that creates a new string of length+actual data (on the assumption that that is the point of the exercise) for you to send, or write a function that takes an object and then sends first the length then the string?

I also don't know what you mean when you say: "what happens if you want two strings after each other?" In what way is this not covered by what you have? (Either you want two \0-terminated strings, in which case there is no alternative to making two packets, or you don't want the \0 in the middle, in which case you have one string with a larger length. And sizeof is irrelevant, because sizeof has not worked with any of your samples yet.) Your empty-array-trick was legalized in C99, but I don't think it made it into C++, even the brand-new C++.

And why not use a std::string instead of char[]?

Originally Posted by iMalc

It's simple - you don't create a structure to represent the data that is sent over the wire. Look into serialisation / deserialisation instead.

I cannot use that in this case for performance reasons. Otherwise it would have been a good idea. I must send all the data in packet at once and I need to know the size from the beginning. The receiver side must be able to do random access on the members.

Originally Posted by TotalTurd

I cannot use that in this case for performance reasons.

That is a completely different story then. Do you want to say that you have already tried serialization/deserialization? What were the results then? Premature optimizations may often lead to a less readable, less scalable, and sometimes the worst - less efficient code. Not to mention, that constructing/analysing packet is usually way faster than transmitting it through the network.

Originally Posted by kmdv

That is a completely different story then. Do you want to say that you have already tried serialization/deserialization? What were the results then? Premature optimizations may often lead to a less readable, less scalable, and sometimes the worst - less efficient code. Not to mention, that constructing/analysing packet is usually way faster than transmitting it through the network.

This is for Mach OS/QNX type message IPC and I cannot just serialize like that. I realize the socket word in the original text was misleading. The structures are in memory and will end up in just like before they were sent, there will be no serialization.

Originally Posted by TotalTurd

This is for Mach OS/QNX type message IPC and I cannot just serialize like that.

So, all in all, you have not used serialization before? Is your software ready to test (can send packets etc)? Profile it if it is.

It is just a mechanism. Noones tells you to use a general-purpose serialization library. You can do all the serialization/deserialization on your own (providing two methods and a buffer). Get rid of references and advanced structures, and the whole process will be similar to plain copying (that is what you probably want to hear).

Random access of a variable-length packet will result in analysing. I (and probably anybody here) do not know what answer you want, since you seem to know all the internals.

Code:

char *GetString { return (char*)this + sizeof(StringPacketBase) }

Now string is a part of the struct but what happens if you want two strings after each other?

Then this method only work for the first string. Also if you use sizeof with StringPacketBase you have to be more careful now.

O_o

I cannot use that in this case for performance reasons.

o_O

Right, well, when you get tired of bashing your head against the wall be sure to look around and see how proper serialization of C++ classes is done for targetting "IPC".

You'll be amazed at how proper primitives to serialize types like `std::string' relieves those headaches.

Soma

Originally Posted by kmdv

So, all in all, you have not used serialization before? Is your software ready to test (can send packets etc)? Profile it if it is.

It is just a mechanism. Noones tells you to use a general-purpose serialization library. You can do all the serialization/deserialization on your own (providing two methods and a buffer). Get rid of references and advanced structures, and the whole process will be similar to plain copying (that is what you probably want to hear).

Random access of a variable-length packet will result in analysing. I (and probably anybody here) do not know what answer you want, since you seem to know all the internals.

No I have not used serialization and maybe that's what I'm looking for. On receiver side I guess this require parsing the entire packet once first in order to find the members, basically unwinding it. Doesn't lead to that I must access the members twice in some cases, for example packets with several strings. First the strings must be parsed in order to find the pointers to the first characters. Then they must be accessed again when they are actually used.

Anyway, I will look at serialization of structures. Thank you for your help.

Originally Posted by TotalTurd

The structures are in memory and will end up in just like before they were sent, there will be no serialization.

That's not true.

You have a specification of your data structure (the struct or class definition). You have not specified how it is to be transmitted over a network. Serialization is the necessary step of converting data into a form that it can be stored or (applicable in your case) transmitted over a network. Once the data has been transmitted over the network, the process needs to be reversed. That means a deserialization step, in order to faithfully recreate a representation of the original structure in memory that can be used by the recipient.

These things don't happen by magic.

Originally Posted by TotalTurd

I cannot use that in this case for performance reasons. Otherwise it would have been a good idea. I must send all the data in packet at once and I need to know the size from the beginning. The receiver side must be able to do random access on the members.

That's a misconception. This is not an issue of performance.
The process is serialisation / deserialisation whether you try and fudge it by dreaming up an impossible and unusable struct to dump verbatim onto the wire and call it by a different name, or whether you plan out a sensible on-the-wire representation and explicitly write code to serialise and deserialise it.

See Serializing classes (especially Salem's link).