Thread: Structs and unions in shared libraries

I want to make a shared library that exposes structs and unions that aren't just opaque pointers. Is this portable at all between compilers on the same platform or should I change the design completely? A lot of libraries do this but I'm not sure how if it's a good idea since the padding in structs and unions may change depending on the compiler used.

Originally Posted by Elkvis

It depends on what you mean by "all compilers on the same platform." If you mean "all versions of GCC on linux," then you might be ok, but the Intel compiler is likely to do it different, and Clang might do it a bit different yet.

I was hoping for a solution that would work with at least all the major compilers like GCC, Clang and VC++. I looked at the #pragma pack directive but that doesn't seem 100% reliable since the integer types could potentially vary with the compiler. This is what the Microsoft documentation says:

The alignment of a member will be on a boundary that is either a multiple of n or a multiple of the size of the member, whichever is smaller.

I may be wrong but given that wording it seems like the packing would not lead to a consistent result if for example a short is 2 bytes in one compiler and 4 in another.

In practice I'm guessing that all the compilers do the same thing since a lot of major libraries like the Win32 API expose transparent structs and unions.

I thought about this some more and now I have even more doubts. What happens if the basic data types aren't the same size as in the compiler that was used to compile the shared library? For example, let's say you have a function like this in a library:

Code:

void foo(short *x);

This function foo copies a short into the memory location pointed to by x. What happens if sizeof(short) was 4 bytes in the compiler used to compile the library while sizeof(short) is only 2 bytes in the compiler that the calling code uses?

Could this ever happen or do you always compile to an ABI where the data types are the same size?

Originally Posted by Elkvis

It's reasonable to expect that you'd have to recompile any time you switch compilers or platforms.

What if you're providing a shared library or a DLL for a particular platform though? You have no control over what compiler the user might be using. I've never seen any requirements that you have to use a certain compiler when I've used DLLs myself.

Originally Posted by MOS-6581

Is this portable at all between compilers on the same platform

Just provide the sources to your users, and save all that hassle.

For a better overall picture, see the Application Binary Interface, Binary code compatibility, and x86 calling conventions Wikipedia pages.

Structure padding and alignment is typically the same across all compilers (and if not, only the defaults vary); it's the function calling conventions that tend to bite you, if anything.

If you compile your dynamic library using the standard compiler for that platform -- each platform has a standard compiler its OS is compiled with --, then you can be pretty sure every other compiler for that platform can be made to compile compatible code. It all comes down to just using the suitable compiler options.

For static libraries, just provide a copy for each compiler.

Document the compiler options used in each case exactly.

What will actually happen if I compile my library with GCC (64 bit) and then use MSVC to compile the calling code? Let's replace short with long in the above function and assume that we're compiling for a 64 bit target on Windows:

Code:

void foo(long *x);

sizeof(long) is 4 bytes in MSVC while it's 8 in GCC. Will the foo function write 4 more bytes than it's supposed to since the library was compiled with the assumption that a long is 8 bytes?

vart is right.

You could also look at how e.g. ffmpeg has solved this exact problem. I don't use Windows, so I don't know the details there.

I have used ICC, Pathscale, Portland Group, Open64, and GCC to compile high-performance applications to be run on Linux clusters, but there the issues are a bit different: typically, the code quality is quite low, and you need to use specific optimizations and avoid others to have the code work correctly; otherwise, the code sometimes crashes or produces garbage results. Due to the desire for high performance, libraries like MPI tend to provide binaries compiled for each compiler separately.

From my own experience, I cannot emphasize enough how useful and worthwhile it is to write a test suite you can compile and run to verify the library works correctly. It should only bother to report the failures, and if at all possible, the details of the failure. Tests that detect e.g. incorrect padding, incorrect or missing function parameters etc. (testing the function call interface parameter passing) are pure gold.

Are there any compilers where you can over align members in a struct (other than the first member)? GCC has __attribute__(aligned(x)) but I'm not sure if that can be applied to the struct members. It's not possible in the MSVC compiler as far as I understand (you can make the alignment smaller but not larger).

I've noticed that some libraries add padding themselves to end up on 8 byte alignment for the members but can you break code like that by changing the compiler settings?

Code:

struct foo
{
  int32_t x;
  int32_t padding0;
  int16_t y;
  int8_t padding1[6];
};

Can you rely on sizeof(foo) to be 16 bytes? The sockaddr_storage struct in the sockets library uses a construct like this but it doesn't feel rock solid.

Originally Posted by MOS-6581

Can you rely on sizeof(foo) to be 16 bytes?

Depends on what you mean by rely on.

sizeof (struct foo) == 16 using GCC on all Linux architectures. I believe, but have not verified, that it is also true on all Mac OS (not just X, but older versions too), Windows, and MS-DOS architectures and compilers (that have the requisite types defined).

The C standards do not require sizeof (struct foo) to be 16, but just about every current C compiler can be told (via compile-time flags et cetera) to provide that, because such options are needed to compile a lot of existing C code.

In other words, you can rely on any practical C compiler to be able to provide such an environment, but you cannot rely on users/developers getting their compiler options right.

Add easy-to-use compile-time checks, both as part of the library build process, and as part of the documentation, so that those who compile their own programs against the library can easily verify that their compiler settings are correct too.