Portable binary file I/O library attempt

Hi everyone. This my first post here. I'm a bit of novice C programmer and am very insecure about my programming skills. I needed to write a small wrapper for handling binary file I/O for use by a couple of programs that I use on 32-bit and 64-bit Windows and Linux systems. I'm feeling a bit uneasy about my design skills and general coding practices and was wondering if any folks here could give it a once over.

Thank you for your time.

Header file:

Code:

---

/*
 * File: binio.h
 * Namespace: binio_ / BINIO_
 *
 * Binary file I/O library
 *
 * Note that 'binio' assumes that the user will never read or write values
 * outside of the minimum sizes guaranteed by the C Standard. For example, a
 * char is expected to be an 8-bit value and a long is expected to be a 32-bit
 * value.
 */

#ifndef BINIO_H
#define BINIO_H

#include <stdio.h>



/*
 * Binary file handle returned by 'binio_from*' and freed by 'binio_close'.
 */
struct binio_file {
        FILE *fp;                /* Private */
        const char *name;        /* Read-only */
        int eof;                /* Read-only */
        int error;                /* Read-only */
};



/*
 * Opens the file 'file' for use. The string 'mode' is passed to 'fopen' and
 * should always include the 'b' option.
 *
 * Returns: If the file was opened successfully, a new file handle is returned.
 * Otherwise, NULL returned and an error message is printed to 'stderr'.
 *
 * Asserts:
 *        file != NULL
 *        mode != NULL
 *        Returned 'binio_file' is valid
 */
struct binio_file *binio_from_file(const char *file, const char *mode);

/*
 * Closes the binary file 'bf'. Note that the file handle is freed by this
 * function. If an error occurred while closing the file, an error message is
 * printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 */
void binio_close(struct binio_file *bf);

/*
 * Reads the entire file 'file' into a newly allocated buffer.
 *
 * Returns: If the buffer was successfully created and read into, it is
 * returned and 'size' is set to the size of the buffer. Otherwise, NULL is
 * returned and an error message is printed to 'stderr'.
 *
 * Asserts:
 *        file != NULL
 *        size != NULL
 */
unsigned char *binio_read_file(const char *file, size_t *size);

/*
 * Reads 'size' bytes from the binary file 'bf' from its current position
 * forward into the buffer 'data'. If the end of the file was reached before
 * reading is complete, the file handle's 'eof' member will be set to a
 * non-zero value. If an error occurred while reading, the file handle's
 * 'error' member will be set to a non-zero value and an error message will be
 * printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 *        data != NULL
 *        size > 0
 */
void binio_read(struct binio_file *bf, unsigned char *data, size_t size);

/*
 * The following input functions read binary values from the binary file 'bf'.
 * Functions with names that end with 'le' read data in little-endian format.
 * Functions with names that end with 'be' read data in big-endian format. If
 * the end of the input file is reached, the file handle's 'eof' member will be
 * set to a non-zero value. If an error occurs while reading, the file handle's
 * 'error' member will be set to a non-zero value and an error message will be
 * printed to 'stderr'.
 *
 * Returns: If successful, the value read from file is returned, otherwise
 * zero is returned.
 *
 * Asserts:
 *        bf != NULL
 */
unsigned char binio_read_u8(struct binio_file *bf);
signed char binio_read_s8(struct binio_file *bf);
unsigned short binio_read_u16le(struct binio_file *bf);
short binio_read_s16le(struct binio_file *bf);
unsigned short binio_read_u16be(struct binio_file *bf);
short binio_read_s16be(struct binio_file *bf);
unsigned long binio_read_u32le(struct binio_file *bf);
long binio_read_s32le(struct binio_file *bf);
unsigned long binio_read_u32be(struct binio_file *bf);
long binio_read_s32be(struct binio_file *bf);

/*
 * Writes the binary data in 'data' with the size 'size' to the file 'file'.
 *
 * Returns: If the binary data was successfully written, zero is returned.
 * Otherwise, a non-zero value is returned and an error message is printed to
 * 'stderr'.
 *
 * Asserts:
 *        file != NULL
 *        data != NULL
 *        size > 0
 */
int binio_write_file(const char *file, const unsigned char *data, size_t size);

/*
 * Writes 'size' bytes to the binary file 'bf' from its current position
 * forward from the buffer 'data'. If an error occurs while writing, the file
 * handle's 'error' member will be set to a non-zero value and an error message
 * will be printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 *        data != NULL
 *        size > 0
 */
void binio_write(struct binio_file *bf, const unsigned char *data,
        size_t size);

/*
 * The following output functions write binary values to the binary file 'bf'.
 * Functions with names that end with 'le' write data in little-endian format.
 * Functions with names that end with 'be' write data in big-endian format. If
 * an error occurs while writing, the file handle's 'error' member will be set
 * to a non-zero value and an error message will be printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 */
void binio_write_u8(struct binio_file *bf, unsigned char u8);
void binio_write_s8(struct binio_file *bf, signed char s8);
void binio_write_u16le(struct binio_file *bf, unsigned short u16);
void binio_write_s16le(struct binio_file *bf, short s16);
void binio_write_u16be(struct binio_file *bf, unsigned short u16);
void binio_write_s16be(struct binio_file *bf, short s16);
void binio_write_u32le(struct binio_file *bf, unsigned long u32);
void binio_write_s32le(struct binio_file *bf, long s32);
void binio_write_u32be(struct binio_file *bf, unsigned long u32);
void binio_write_s32be(struct binio_file *bf, long s32);

/*
 * Resets the current position of the binary file 'bf' to its beginning. If an
 * error occurs while rewinding, the file handle's 'error' member will be set
 * to a non-zero value and an error message will be printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 */
void binio_rewind(struct binio_file *bf);

/*
 * Gets the number of bytes left to be read in the binary file 'bf', then
 * resets the binary file's position back to the beginning of the file.
 *
 * Returns: If successful, the number of bytes left in the file is returned.
 * Otherwise, if an error occurs or there where no bytes left to read, zero is
 * returned, the file handle's 'error' member is set to a non-zero value, and
 * an error message will be printed to 'stderr'.
 *
 * Asserts:
 *        bf != NULL
 */
size_t binio_get_size(struct binio_file *bf);

#endif /* BINIO_H */

---

Source:

Code:

---

/*
 * File: binio.c
 *
 * Binary file I/O library implementation
 */

#include <assert.h>
#include <errno.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "binio.h"



#if UCHAR_MAX > 0xFFU
#        error 'binio' requires 8-bit chars
#endif

typedef unsigned char uint8;
typedef signed char int8;

#if USHRT_MAX > 0xFFFFU
#        error 'binio' requires 16-bit shorts
#endif

typedef unsigned short uint16;
typedef short int16;

#if ULONG_MAX == 0xFFFFFFFFUL

typedef unsigned long uint32;
typedef long int32;

#elif UINT_MAX == 0xFFFFFFFFUL

typedef unsigned int uint32;
typedef int int32;

#else
#        error 'binio' requires either 32-bit longs or 32-bit ints
#endif



struct binio_file *binio_from_file(const char *file, const char *mode)
{
        struct binio_file *bf;

        assert(file != NULL);
        assert(mode != NULL);

        if ((bf = malloc(sizeof *bf)) == NULL) {
                fprintf(stderr, "Unable to allocate new file handle\n");
                return NULL;
        }

        if ((bf->fp = fopen(file, mode)) == NULL) {
                fprintf(stderr, "Error opening \"%s\", %s\n", file,
                        strerror(errno));
                return NULL;
        }

        bf->name = file;
        bf->eof = bf->error = 0;

        assert(bf->fp != NULL);
        assert(bf->name == file);
        assert(bf->eof == 0);
        assert(bf->error == 0);

        return bf;
}



void binio_close(struct binio_file *bf)
{
        assert(bf != NULL);

        if (fclose(bf->fp) == EOF) {
                fprintf(stderr, "Error closing \"%s\", %s\n", bf->name,
                        strerror(errno));
        }

        free(bf);
}



unsigned char *binio_read_file(const char *file, size_t *size)
{
        struct binio_file *bf;
        unsigned char *data;
        int success;

        assert(file != NULL);
        assert(size != NULL);

        if ((bf = binio_from_file(file, "rb")) == NULL) {
                return NULL;
        }

        if ((*size = binio_get_size(bf)) == 0) {
                binio_close(bf);
                return NULL;
        }

        if ((data = malloc(sizeof *data * *size)) == NULL) {
                fprintf(stderr, "Unable to allocate new binary buffer\n");
                binio_close(bf);
                return NULL;
        }

        binio_read(bf, data, *size);
        success = !(bf->error || bf->eof);

        if (bf->eof) {
                fprintf(stderr, "Error reading \"%s\", Unexpected EOF\n",
                        bf->name);
        }

        binio_close(bf);

        if (!success) {
                free(data);
                return NULL;
        }

        return data;
}



void binio_read(struct binio_file *bf, unsigned char *data, size_t size)
{
        size_t i;

        assert(bf != NULL);
        assert(data != NULL);
        assert(size > 0);

        for (i = 0; i < size; i++) {
                data[i] = binio_read_u8(bf);
        }
}



unsigned char binio_read_u8(struct binio_file *bf)
{
        uint8 u8;

        assert(bf != NULL);

        if (bf->eof || bf->error) {
                return 0;
        }

        if (fread(&u8, 1, 1, bf->fp) == 1) {
                return u8;
        }

        if (feof(bf->fp)) {
                bf->eof = 1;
        } else if (ferror(bf->fp)) {
                bf->error = 1;
                fprintf(stderr, "Error reading \"%s\", %s\n", bf->name,
                        strerror(errno));
        }

        return 0;
}



signed char binio_read_s8(struct binio_file *bf)
{
        return (int8)binio_read_u8(bf);
}



unsigned short binio_read_u16le(struct binio_file *bf)
{
        uint16 u16;

        u16 = binio_read_u8(bf);
        u16 |= (uint16)binio_read_u8(bf) << 8;

        return u16;
}



short binio_read_s16le(struct binio_file *bf)
{
        return (int16)binio_read_u16le(bf);
}



unsigned short binio_read_u16be(struct binio_file *bf)
{
        uint16 u16;

        u16 = (uint16)binio_read_u8(bf) << 8;
        u16 |= binio_read_u8(bf);

        return u16;
}



short binio_read_s16be(struct binio_file *bf)
{
        return (int16)binio_read_u16be(bf);
}



unsigned long binio_read_u32le(struct binio_file *bf)
{
        uint32 u32;

        u32 = binio_read_u8(bf);
        u32 |= (uint32)binio_read_u8(bf) << 8;
        u32 |= (uint32)binio_read_u8(bf) << 16;
        u32 |= (uint32)binio_read_u8(bf) << 24;

        return u32;
}



long binio_read_s32le(struct binio_file *bf)
{
        return (int32)binio_read_u32le(bf);
}



unsigned long binio_read_u32be(struct binio_file *bf)
{
        uint32 u32;

        u32 = (uint32)binio_read_u8(bf) << 24;
        u32 |= (uint32)binio_read_u8(bf) << 16;
        u32 |= (uint32)binio_read_u8(bf) << 8;
        u32 |= binio_read_u8(bf);

        return u32;
}



long binio_read_s32be(struct binio_file *bf)
{
        return (int32)binio_read_u32be(bf);
}



int binio_write_file(const char *file, const unsigned char *data, size_t size)
{
        struct binio_file *bf;
        int success;

        if ((bf = binio_from_file(file, "wb")) == NULL) {
                return 0;
        }

        binio_write(bf, data, size);
        success = !bf->error;
        binio_close(bf);

        return success;
}



void binio_write(struct binio_file *bf, const unsigned char *data, size_t size)
{
        size_t i;

        assert(bf != NULL);
        assert(data != NULL);
        assert(size > 0);

        for (i = 0; i < size; i++) {
                binio_write_u8(bf, data[i]);
        }
}



void binio_write_u8(struct binio_file *bf, unsigned char u8)
{
        assert(bf != NULL);

        if (bf->error) {
                return;
        }

        if (fwrite(&u8, 1, 1, bf->fp) == 1) {
                return;
        }

        bf->error = 1;

        if (ferror(bf->fp)) {
                fprintf(stderr, "Error writing \"%s\", %s\n", bf->name,
                        strerror(errno));
        }
}



void binio_write_s8(struct binio_file *bf, signed char s8)
{
        binio_write_u8(bf, (uint8)s8);
}



void binio_write_u16le(struct binio_file *bf, unsigned short u16)
{
        binio_write_u8(bf, (uint16)(u16 & 0xFF));
        binio_write_u8(bf, (uint16)(u16 >> 8 & 0xFF));
}



void binio_write_s16le(struct binio_file *bf, short s16)
{
        binio_write_u16le(bf, (uint16)s16);
}



void binio_write_u16be(struct binio_file *bf, unsigned short u16)
{
        binio_write_u8(bf, (uint16)(u16 >> 8 & 0xFF));
        binio_write_u8(bf, (uint16)(u16 & 0xFF));
}



void binio_write_s16be(struct binio_file *bf, short s16)
{
        binio_write_u16be(bf, (uint16)s16);
}



void binio_write_u32le(struct binio_file *bf, unsigned long u32)
{
        binio_write_u8(bf, (uint32)(u32 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 >> 8 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 >> 16 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 >> 24 & 0xFF));
}



void binio_write_s32le(struct binio_file *bf, long s32)
{
        binio_write_u32le(bf, (uint32)s32);
}



void binio_write_u32be(struct binio_file *bf, unsigned long u32)
{
        binio_write_u8(bf, (uint32)(u32 >> 24 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 >> 16 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 >> 8 & 0xFF));
        binio_write_u8(bf, (uint32)(u32 & 0xFF));
}



void binio_write_s32be(struct binio_file *bf, long s32)
{
        binio_write_u32be(bf, (uint32)s32);
}



void binio_rewind(struct binio_file *bf)
{
        assert(bf != NULL);

        if (fseek(bf->fp, 0L, SEEK_SET) == -1) {
                bf->error = 1;
                fprintf(stderr, "Error rewinding \"%s\", %s\n", bf->name,
                        strerror(errno));
        } else {
                bf->eof = 0;
        }
}



size_t binio_get_size(struct binio_file *bf)
{
        size_t size = 0;

        assert(bf != NULL);

        binio_read_u8(bf);

        while (!bf->eof) {
                binio_read_u8(bf);
                size++;
        }

        binio_rewind(bf);

        if (bf->error) {
                return 0;
        }

        return size;
}

---

It might be best to use "int8_t" and "int16_t" and "int32_t" rather than hoping that char and short and int are the right size (I mean, they probably will be the right size, but if you mean "eight-bit integer" you might as well say "eight-bit integer").

TBH I'm not sure what you're binio struct is buying you, other than having the name associated with the file. I guess C isn't as strict about C++ as "thou shalt not read from a file if there was an error", but that's probably because scanf sort of thrives on errors. Depending on your situation, you may want to have a way to clear an error state (or I might have missed it above).

First off I don't see how you can establish private anything in C. Of course you could use static to try an lock down some variables to the source file itself.

Secondly, why use all those nasty big endian and little endian functions? Why not encapsulate them? Make a single function which has two cases (one for big endian and one for little endian). You can write a simple checking function:

isLittleEndian()

If it returns true then do the little endian case, else do the big endian case. This way you can simplify the library.

Quote:

---

It might be best to use "int8_t" and "int16_t" and "int32_t" rather than hoping that char and short and int are the right size (I mean, they probably will be the right size, but if you mean "eight-bit integer" you might as well say "eight-bit integer").

---

I thought using "unsigned char" and "short" would be adequate, as long as I assume that they can hold at least an 8-bit unsigned value and 16-bit signed value respectively.

Quote:

---

TBH I'm not sure what you're binio struct is buying you, other than having the name associated with the file. I guess C isn't as strict about C++ as "thou shalt not read from a file if there was an error", but that's probably because scanf sort of thrives on errors. Depending on your situation, you may want to have a way to clear an error state (or I might have missed it above).

---

I'm not sure I follow the last part, but the error state does stop the called functions from reading/writing to file. That way, the user can do several reads, than check the error flag if needed to see if the data read is going to be valid.

Quote:

---

First off I don't see how you can establish private anything in C. Of course you could use static to try an lock down some variables to the source file itself.

---

It's more of a suggestion to the user than anything, it's C after all :).

Quote:

---

Secondly, why use all those nasty big endian and little endian functions? Why not encapsulate them? Make a single function which has two cases (one for big endian and one for little endian). You can write a simple checking function:

isLittleEndian()

If it returns true then do the little endian case, else do the big endian case. This way you can simplify the library.

---

You mean associating endianness with the file when its handle is created? That might actually be a good idea.

Quote:

---

Originally Posted by lost_in_code

I thought using "unsigned char" and "short" would be adequate, as long as I assume that they can hold at least an 8-bit unsigned value and 16-bit signed value respectively.

---

If you were making that assumption, you'd be okay. But:

Code:

---

#if UCHAR_MAX > 0xFFU
#        error 'binio' requires 8-bit chars
#endif

---

is a whole different assumption -- that char is exactly eight bits.

Quote:

---

Originally Posted by lost_in_code

I'm not sure I follow the last part, but the error state does stop the called functions from reading/writing to file. That way, the user can do several reads, than check the error flag if needed to see if the data read is going to be valid.

---

As I say, I've not fiddled with ferror enough to know whether or not you're reduplicating what's already there or actually providing something. Then again, I don't think I've ever had ferror trigger on me anyway.

Quote:

---

If you were making that assumption, you'd be okay. But:

---

Yeesh.... Why on earth did I even write that. Thanks for pointing that out.

EDIT: I remember actually why I did that. I realized that in order to get correct behavior out of the functions that read signed values, I needed to ensure that they would cast correctly. So those limits at the top actually define what platforms the routines will work on.

Quote:

---

As I say, I've not fiddled with ferror enough to know whether or not you're reduplicating what's already there or actually providing something. Then again, I don't think I've ever had ferror trigger on me anyway.

---

I think your actually right. I did all of this so I wouldn't call fread/fwrite after an error. I assumed that was the "proper" thing to do. But that's exactly what ferror is for; in case you want to do error checking after several fread/fwrite calls. So it seems that calling fread/fwrite after an error can't be to bad, just as long as you handle the error in your code eventually.

Boy, this is providing a lot of good insight, thanks to the both of you for helping me with this.

I think fread and fwrite will happily go on no matter what the state of ferror, because I have this sneaking suspicion that going on would be the Right Thing if our FILE was actually something volatile (i.e. some sort of physical device that provides data as it is generated), and the standard only says that fread may set the error state, not what it will do if the error state is already set. So I think you're actually doing something useful there, but as I say I've not really got a way to check.

Casting is a bit problematic anyway, as casting from unsigned to signed also requires information about how the system stores negative numbers. Again, all the platforms you're interested in do it the same way, so that's not an actual issue, but a theoretical issue.

I guess that's my final question then. Is it "safe" to continue using fread/fwrite after an error, or is it "proper" to make sure you never call it for a file whose error state is set. If any experts out there know or have advice as to how to find out, feel free to post :D.

Quote:

---

Originally Posted by lost_in_code

I guess that's my final question then. Is it "safe" to continue using fread/fwrite after an error, or is it "proper" to make sure you never call it for a file whose error state is set. If any experts out there know or have advice as to how to find out, feel free to post :D.

---

That depends how many errors caused by previous errors you're willing to tolerate...

One might reasonably think the first error should signal a need to either A) correct the problem or B) abort the procedure.

IOW... ignoring errors is never _safe_

Take for example, I need to read a small binary data file. This might consist of several calls to fread. It would be darn inconvenient to check the result of each one instead of calling fread several times, storing the values read, then checking for errors with ferror. That was the reasoning behind the code I wrote, so I could just do something like this:

Code:

---

    struct binio_file *bf;

    ...

    val[0] = binio_read_u8(bf);
    val[1] = binio_read_u8(bf);
    val[2] = binio_read_u8(bf);
    val[3] = binio_read_u8(bf);

    if (bf->error) {
        /* Report error and abort */
    }

---

What I want to know is, is it safe and proper to do this with plain old fread:

Code:

---

    FILE *fp;
    ...

    fread(&val[0], 1, 1, fp);
    fread(&val[1], 1, 1, fp);
    fread(&val[2], 1, 1, fp);
    fread(&val[3], 1, 1, fp);

    if (ferror(fp)) {
        /* Report error and abort */
    }

---

The library code I wrote uses its error member to prevent binio_read_u8 from actually calling fread if there has been an error.

Sorry to double post, but after doing some research, I've come to the conclusion that it is generally OK to call fwrite/fread after a preceding call causes an error. Granted you would want to do sanity checks on values read from file so you don't end up doing things like exceeding the bounds of an array, but there doesn't seem to be to much of problem with deferring error checking when its really needed. Also, I can't imagine Linux/Windows allowing extra fwrite/fread calls after an error occurs to do anything harmful.

Anyways, thanks guys for your help.

Don't we already have portable binary in C in the form of htonl, htons, ntohl, ntohs?


Quzah.

Quote:

---

Originally Posted by lost_in_code

Also, I can't imagine Linux/Windows allowing extra fwrite/fread calls after an error occurs to do anything harmful.

---

Imagine it. C does absolutely nothing to baby sit or protect you from yourself... If you tell it to fill a file with garbage, that's exactly what it's going to do.