Thread: Increasing The Size of Memory Allocated to a Struct via Malloc

  1. #1
    Registered User
    Join Date
    Apr 2011
    Posts
    33

    Increasing The Size of Memory Allocated to a Struct via Malloc

    Hi,

    I just learned that it's possible to increase the size of the memory you'll allocate to a struct when using the malloc function. For example, you can have a struct like this:

    Code:
    struct test{
    	char c;
    	int v[1];
    };
    Which clearly has space for only 1 char and 1 int. But you could call malloc in such a way to make it hold 1 char and up to 10 ints:

    Code:
    int main(){
    	struct test *ptr;
    	ptr = malloc (sizeof(struct test)+sizeof(int)*9);
    	ptr->v[9]=50;
    	printf("%d\n",ptr->v[9]);	
    return 0;
    }
    The output here would be "50" printed on the screen.

    My questions for the experienced C programmers out there:

    1. What is happening behind the scenes here? Does the computer allocate 1+4 (char+int) bytes for the standard "struct test", and then 4*9 more bytes of memory and let the pointer "ptr" put whatever kind of data it wants on those extra bytes?

    2. Does this trick only works when there is an array inside the struct?
    Last edited by envec83; 09-15-2011 at 12:45 PM.

  2. #2
    Registered User MacNilly's Avatar
    Join Date
    Oct 2005
    Location
    CA, USA
    Posts
    466
    In the most recent C standard (C99), the last member of a structure may be a variable-sized array, so maybe that's what's going on. Try compiling under strict ANSI rules (-ansi on gcc) and see what happens.

    Malloc just returns a block of bytes, it doesn't know what type of object you're trying to allocate. So you get a block of size ( sizeof(struct test) + sizeof(int) * 9 ) bytes. You tell the compiler to point a struct test pointer at it. So you're correct, you're using the last member of the struct as a point from which to access you're "extra" bytes.

  3. #3
    Registered User
    Join Date
    Apr 2011
    Posts
    33
    Curiously the trick works when using ANSI standard as well.

    But yeah doesn't work when the array is not last member.
    Last edited by envec83; 09-15-2011 at 01:32 PM.

  4. #4
    Registered User
    Join Date
    Oct 2008
    Location
    TX
    Posts
    2,059
    This would do well in a code obfuscation contest, not in the real world.

  5. #5
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    All this does is that you asked for a chunk of memory that was of a certain size and then proceeded to access parts of it up to that size.

    It's not really much different than if there were no struct involved and you were just allocating an array. All the struct contributes is its size (a number at runtime) to help you allocate the correct amount of memory, and something that tells the compiler that accessing certain variables within that struct just involves accessing memory at the base address of the memory allocated for it plus some offset.

    The following should also access your 50, demonstrating that it's all just a chunk of RAM, and also that the struct contains padding around the char, bringing it up to the size taken by an int.
    Code:
    	int *ptr2 = (int*)ptr;
    	printf("%d\n",ptr2[10]);
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  6. #6
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    Quote Originally Posted by envec83 View Post
    It also works when the array is not the last member of the struct.
    No, it most certainly does not!
    You'd see that it does not if you wrote to the array contents and then examined any members listed after the array would have been overwritten.
    Agreed that it would not crash, but that isn't really a definition of "works".
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  7. #7
    Registered User
    Join Date
    Apr 2011
    Posts
    33
    @iMalc, good points, and it made things more clear.

    Thanks.

  8. #8
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by envec83 View Post
    My questions for the experienced C programmers out there:

    1. What is happening behind the scenes here?
    Pure and unmitigated dumb luck. Try this in a more complex program and it's likely to do all kinds of strange stuff. You are playing with undefined behavior and the outcome will depend on the compiler, usage of the variables and the order in which the struct is stored in memory.

    Earlier versions of windows used to to this (WNet network enumerator still does). You would get back a memory buffer with an array of structs and all the strings are stored behind the array in memory... It does work, but it's not exactly the safest way to do things, buffer overruns and struct misalignment were common problems.

    2. Does this trick only works when there is an array inside the struct?
    AND only when the array is stored at the end of the stuct... anywhere else, as iMalc points out, all you're going to do is overwrite the other struct members.

  9. #9
    Registered User
    Join Date
    May 2011
    Location
    Around 8.3 light-minutes from the Sun
    Posts
    1,949
    I believe this concept is addressed by C-Faq 2.6 (struct hack)
    Quote Originally Posted by anduril462 View Post
    Now, please, for the love of all things good and holy, think about what you're doing! Don't just run around willy-nilly, coding like a drunk two-year-old....
    Quote Originally Posted by quzah View Post
    ..... Just don't be surprised when I say you aren't using standard C anymore, and as such,are off in your own little universe that I will completely disregard.
    Warning: Some or all of my posted code may be non-standard and as such should not be used and in no case looked at.

  10. #10
    Master Apprentice phantomotap's Avatar
    Join Date
    Jan 2008
    Posts
    5,108
    O_o

    It is undefined behavior according to the standards committee by virtue of not being address, but this "struct hack" method is as common as dirt and unless you have a bad compiler or a platform that violates parts of the standard you'll be fine.

    Compilers that attempt to implement C99 are more free in this regard, but it so happens that "flexible array members" are a required part of that standard.

    It works basically the same as having these three structures:

    Code:
    struct base
    {
        int size;
    };
    
    struct core
    {
        int size;
        int i[1];
    };
    
    struct big
    {
        int size;
        int i[100];
    };
    The standard demands that the `size' member be correctly addressable if you allocate any of these structures and cast the result to a pointer to any of these structures. Further, the one element of the `i' array must also be addressable in the case of `core' and `big'. Finally, the standard demands a regular approach to indirectly addressing elements of an array. Basically, if your compiler conforms to all of those parts of the standard, and doesn't implement bounds checking, your walking on undefined but highly reliable behavior.

    Ultimately, you may find a compiler that produces code that doesn't work or emits a warning/error, but in practice it is widely available with well defined semantics.

    Oh, as for it being "dumb luck" or a problem in "complex programs", you'll find it used a lot in the "Linux" kernel, "Oracles" database systems, and a lot of "FreeBSD" bits.

    Soma

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Allocated memory size is not correct.
    By ramtinraji in forum C Programming
    Replies: 8
    Last Post: 02-22-2011, 11:16 PM
  2. Replies: 2
    Last Post: 06-25-2010, 04:04 AM
  3. Malloc & Calloc difference in terms of memory allocated
    By swapnaoe in forum C Programming
    Replies: 2
    Last Post: 04-19-2007, 12:57 AM
  4. Size of allocated memory
    By TriKri in forum C++ Programming
    Replies: 8
    Last Post: 11-27-2006, 01:22 PM
  5. Replies: 4
    Last Post: 12-01-2005, 11:16 PM