free portions of a buffer

[dayalsoap]

Hi,
Question about malloc and free. Suppose I malloc a buffer of size, say 100 bytes, but end up only needing 90 of those bytes. Is there a way at all to free the remaining 10 bytes?

Should I just have a pointer point to the 91st byte and then free it? How would the compiler know where the 2nd pointer ends (i.e., how much to free?) I'm guessing the kernel must maintain some understanding of this as it already manages the virtual to physical memory mappings.

Thanks.

[ZuK]

Use realloc().
Kurt

[dayalsoap]

Moving the 2nd pointer to the appropriate location and then freeing the 2nd pointer doesn't work.

[dayalsoap]

realloc seems like it would do the trick.
Thanks.

[phantomotap]

O_o

Be aware that `realloc' may not necessarily attempt to honor a request for less memory, when it does honor such a request the memory will not be returned to the system unless it just happens to begin at certain location or the difference is at least as large as one of the system native memory blocks, and may give you a pointer to a new location with the new size instead of using the old location.

Soma

[Click_here]

"realloc" is made for this purpose. As phantomotap said, check to see that it doesn't return NULL and remember that it may be in a different location to the original pointer.

[ledow]

Also, while you're thinking about this problem you've stated, why would you not know how much you needed to allocate until after you've allocated it and used it? Surely if you only needed ten bytes, you could have only allocated 10 in the first place, or allocated nothing and grown as you've needed.

realloc, in practice, rarely releases memory below the page-size of your architecture. If you have a 4k page size, you'll get memory in blocks of 4kbytes and nothing less. Even a one-byte malloc might take up significantly more in memory - maybe even the entire 4kbytes! So growing from tiny to large buffers often needs no work at all. realloc may only shift things around or release the memory when you actually cross a 4k boundary, in that instance. So you won't "save" anything in practice at all, but will make realloc do a lot of work.

Personally, if I'm handling a totally unknown quantity, I start it as NULL. The first realloc (or actually malloc because realloc'ing a NULL pointer is basically the same as malloc) will put it to a size I estimate will cope with it (and a bit more). If it doesn't, the next realloc I just double the current size of the allocation. Still not big enough after lots of hard work? Double again. And again. This means that small buffers get only a single realloc big enough to cope with them and have a realloc or two done to them, but large buffers get to a size that will hold them (and quite a lot more because it will probably be almost-double what I actually NEED) very quickly.

There are memory libraries that have optimised alloc routines that tuck small allocations into single pages and other tricks (which is why you ALWAYS need to pay attention to realloc's return value), but they also do a lot of work to manage all that. So minimising realloc calls and not worrying about a few lost bytes may be better for you.

Personally, 90 bytes of over-allocation isn't enough to worry about. You'll probably find it actually takes up 1k of memory or something anyway and unless you're designed for embedded systems it's hardly worth tracking. Also, the chances are that the initial allocation would fail if you really were THAT tight on memory anyway, so blatting past the limit only to save 90% of it later on is a little pointless. And if you're allocating thousands of things like that, you should really just be allocating a huge block of memory in advance anyway, to cut down on the number of *-alloc calls anyway.

I spent a long time on my first big project worrying about every byte of memory, and then realised that just the sheer volume of *-alloc calls was actually more of a performance hit that if I'd just allocated more memory anyway. Your program probably has at least 1,073,741,824 bytes (1Gb) available to it anyway and won't run 24/7, and won't allocate more than 100Mb anyway. Let's say that 100Mb is a reasonable executable size - the largest program on my computer at the moment is 250Mb and that's an Opera session that's been open for about a year now, and second is a full Eclipse C/C++ environment that's been open for longer. So 90 bytes is really just 0.0000009% of the memory a reasonable program would take. Sure, that can multiply up, but it's not a huge problem for the majority of programs unless you're really tight on memory. And, there, you can employ other techniques (allocating in advance, in-page allocators, etc.) that would probably get you better gains than trying to slim everything down to its theoretical minimum all the time.