Thread: Binary data handling

  1. #1
    Registered User
    Join Date
    Jul 2006

    Binary data handling

    I have a stream of binary data that I am pushing into structs and handling. The data sometimes, though, can contain variable length fields in between other fixed fields. I'm having trouble trying to decide the best way to handle structuring my structs to handle this. For example, lets say the "event1" data comes as:

    - int block_type
    - int block_length // includes type & length length
    - char *block
    - int id
    - int other_data
    - int block_type2
    - int block_length2 // includes type & length length
    - char *block2
    - int another_value

    The way I've handled it to this point in a piece by piece approach. As I'm dealing with a binary chunk of data and multiple variable length fields, I need know the sizeof() each section before moving on to the next. It works but is less organized and easy to look at than I'd like. For example, I've implemented the above as such:

    struct _Block {
        int block_type;
        int block_length;
        char block[MAXBUFSIZE];
    } __attribute__((__packed__));
    typedef struct _Block Block;
    and handling would be something like this:

    // Handle event data chunk and return number of bytes processed so I can
    // move onto the next one
    int handle_data(char *rawdata) {
        Block *block, *block2;
        int *id;
        int *other_data;
        int *another_value;
        int offset = 0;
        block = (Block*)rawdata;
        // process block
        offset += block->block_length;
        id = (int*)(rawdata + offset);
        // process id
        offset += sizeof(int);
        other_data = (int*)(rawdata + offset);
        // process other_data
        offset += sizeof(int);
        block2 = (Block*)(rawdata + offset);
        // process block2
        offset += block2->block_length;
        another_value = (int*)(rawdata + offset);
        // process another_value
        offset += sizeof(int);
        return offset;
    Size can slightly be reduced by by bundling a section like the id and other_data into another struct. Basically, is there a more elegant way to handle this? Is there a way to design my initial struct such that it contains all the fields? If so, I assume the variable length char* fields would have to be dynamically allocated?

  2. #2
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Writing a structure to a file which contains pointers to dynamically allocated memory would not work. You'd have to set those members to something else when you read in the file, since the pointers would be invalid next time you read the data into memory (unless the same instance of the program was still running), which would defeat the purpose.

    You could, however, do something like this:
    struct string_t {
        size_t len;
        char *data;
    Then you'd write that structure to the file followed by the data in data. When reading that structure in, you'd look at len and read that many more bytes into data.

    Something like this:
    struct string_t str = {3, "abc"};
    fwrite(&str, sizeof(str), 1, fp);
    fputs(, fp);
    /* ... */
    fread(&str, sizeof(str), 1, fp);
    read_bytes(str.len, &;
    where read_bytes() would read str.len bytes into, which would be dynamically allocated. Maybe like this, without error checking.
    void read_bytes(size_t len, char **data) {
        size_t x;
        *data = malloc(len + 1);
        for(x = 0; x < len; x ++) {
            data[x] = fgetc(fp);
        data[x] = 0;

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell

    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ:

    My website:
    Projects: codeform, xuni, atlantis, nort, etc.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Changing header data in binary file
    By neruocomp in forum C Programming
    Replies: 8
    Last Post: 11-14-2008, 07:30 PM
  2. How to write image data to binary PGM file format(P5)?
    By tommy_chai in forum C Programming
    Replies: 6
    Last Post: 11-03-2007, 10:52 PM
  3. Binary data -> istream
    By Magos in forum C++ Programming
    Replies: 8
    Last Post: 01-24-2005, 02:57 AM
  4. Help with Reading and writing binary data
    By Yasir_Malik in forum C Programming
    Replies: 3
    Last Post: 12-12-2004, 09:24 AM