Increasing The Size of Memory Allocated to a Struct via Malloc

Hi,

I just learned that it's possible to increase the size of the memory you'll allocate to a struct when using the malloc function. For example, you can have a struct like this:

Code:

---

struct test{
        char c;
        int v[1];
};

---

Which clearly has space for only 1 char and 1 int. But you could call malloc in such a way to make it hold 1 char and up to 10 ints:

Code:

---

int main(){
        struct test *ptr;
        ptr = malloc (sizeof(struct test)+sizeof(int)*9);
        ptr->v[9]=50;
        printf("%d\n",ptr->v[9]);       
return 0;
}

---

The output here would be "50" printed on the screen.

My questions for the experienced C programmers out there:

1. What is happening behind the scenes here? Does the computer allocate 1+4 (char+int) bytes for the standard "struct test", and then 4*9 more bytes of memory and let the pointer "ptr" put whatever kind of data it wants on those extra bytes?

2. Does this trick only works when there is an array inside the struct?

In the most recent C standard (C99), the last member of a structure may be a variable-sized array, so maybe that's what's going on. Try compiling under strict ANSI rules (-ansi on gcc) and see what happens.

Malloc just returns a block of bytes, it doesn't know what type of object you're trying to allocate. So you get a block of size ( sizeof(struct test) + sizeof(int) * 9 ) bytes. You tell the compiler to point a struct test pointer at it. So you're correct, you're using the last member of the struct as a point from which to access you're "extra" bytes.

Curiously the trick works when using ANSI standard as well.

But yeah doesn't work when the array is not last member.

This would do well in a code obfuscation contest, not in the real world.

All this does is that you asked for a chunk of memory that was of a certain size and then proceeded to access parts of it up to that size.

It's not really much different than if there were no struct involved and you were just allocating an array. All the struct contributes is its size (a number at runtime) to help you allocate the correct amount of memory, and something that tells the compiler that accessing certain variables within that struct just involves accessing memory at the base address of the memory allocated for it plus some offset.

The following should also access your 50, demonstrating that it's all just a chunk of RAM, and also that the struct contains padding around the char, bringing it up to the size taken by an int.

Code:

---

        int *ptr2 = (int*)ptr;
        printf("%d\n",ptr2[10]);

---

Quote:

---

Originally Posted by envec83

It also works when the array is not the last member of the struct.

---

No, it most certainly does not!
You'd see that it does not if you wrote to the array contents and then examined any members listed after the array would have been overwritten.
Agreed that it would not crash, but that isn't really a definition of "works".

@iMalc, good points, and it made things more clear.

Thanks.

Quote:

---

Originally Posted by envec83

My questions for the experienced C programmers out there:

1. What is happening behind the scenes here?

---

Pure and unmitigated dumb luck. Try this in a more complex program and it's likely to do all kinds of strange stuff. You are playing with undefined behavior and the outcome will depend on the compiler, usage of the variables and the order in which the struct is stored in memory.

Earlier versions of windows used to to this (WNet network enumerator still does). You would get back a memory buffer with an array of structs and all the strings are stored behind the array in memory... It does work, but it's not exactly the safest way to do things, buffer overruns and struct misalignment were common problems.

Quote:

---

2. Does this trick only works when there is an array inside the struct?

---

AND only when the array is stored at the end of the stuct... anywhere else, as iMalc points out, all you're going to do is overwrite the other struct members.

I believe this concept is addressed by C-Faq 2.6 (struct hack)

O_o

It is undefined behavior according to the standards committee by virtue of not being address, but this "struct hack" method is as common as dirt and unless you have a bad compiler or a platform that violates parts of the standard you'll be fine.

Compilers that attempt to implement C99 are more free in this regard, but it so happens that "flexible array members" are a required part of that standard.

It works basically the same as having these three structures:

Code:

---

struct base
{
    int size;
};

struct core
{
    int size;
    int i[1];
};

struct big
{
    int size;
    int i[100];
};

---

The standard demands that the `size' member be correctly addressable if you allocate any of these structures and cast the result to a pointer to any of these structures. Further, the one element of the `i' array must also be addressable in the case of `core' and `big'. Finally, the standard demands a regular approach to indirectly addressing elements of an array. Basically, if your compiler conforms to all of those parts of the standard, and doesn't implement bounds checking, your walking on undefined but highly reliable behavior.

Ultimately, you may find a compiler that produces code that doesn't work or emits a warning/error, but in practice it is widely available with well defined semantics.

Oh, as for it being "dumb luck" or a problem in "complex programs", you'll find it used a lot in the "Linux" kernel, "Oracles" database systems, and a lot of "FreeBSD" bits.

Soma