memory not released after running program

[lehe]

Hi,
I am programming in Emacs and Ubuntu8.10.

I have finished running my C++ program from Emacs, even exited Emacs, however the memory used by my program is still held by the system. System monitor shows this in the memory and swap charts under "Resources" tab, but does not show the process that takes that large amount of memory in its "Processes" tab, so I have no clue how to release the memory. Is there a bash command that can release memory? Also even if there is no deallocation of memory in the code, why after the program finishes its running it does not release the memory used?

Thanks and regards!

[brewbuck]

You are misinterpreting something. Either your program is actually still running, or you are looking at the buffer/cache usage, which is meaningless for measuring actual memory pressure.

[lehe]

How to measure the actual memory pressure? In Ubuntu's system monitor, does the two circular charts, one for memory and one for swap, not measure the actual memory pressure? The result shown in system monitor is:

Memory
471.0 MiB (49.8%) of 944.9 Mib
Swap
269.1 MiB (27.4%) of 949.1 MiB

However it is different from the result by top:

Mem: 967588k total, 788012k used, 179576k free, 11320k buffers
Swap: 971892k total, 266308k used, 705584k free, 303604k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6381 root 20 0 162m 81m 8200 R 18.8 8.6 190:45.85 Xorg
23322 ting 20 0 43472 20m 11m R 15.5 2.2 32:15.48 gnome-system-mo
3338 ting 20 0 260m 102m 30m S 2.3 10.8 7:13.74 firefox
14391 ting 20 0 208m 48m 22m S 2.0 5.1 0:42.75 python
24781 ting 20 0 111m 25m 13m R 2.0 2.8 0:30.38 gnome-terminal
15756 ting 20 0 2416 1148 876 R 0.7 0.1 0:06.62 top
20538 ting 20 0 5280 1696 1364 S 0.7 0.2 0:04.76 wget
2603 root 15 -5 0 0 0 S 0.3 0.0 1:41.31 b43
6801 ting 20 0 21344 4820 4156 S 0.3 0.5 6:50.97 gnome-screensav
6852 ting 20 0 54800 15m 9284 S 0.3 1.6 1:51.14 nm-applet
1 root 20 0 3056 516 460 S 0.0 0.1 0:01.94 init
2 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 15 -5 0 0 0 S 0.0 0.0 0:17.66 ksoftirqd/0
5 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
6 root 15 -5 0 0 0 S 0.0 0.0 0:03.52 events/0
7 root 15 -5 0 0 0 S 0.0 0.0 0:00.00 khelper

BTW, I cannot find my process in the output of top or system monitor.

[brewbuck]

If you look at the output of top, you'll notice that if you take "788012k used" from the first line, then subtract the "11320k buffers" from the first line and also subtract the "303604k cached" from the second line, you get 473088, which is extremely close to the number that Ubuntu is reporting.

So I do believe the ~470 megabyte number. I was worried that you were looking at top's "179576k free" statistic, which is misleading and not a true indication of available RAM.

BTW, I cannot find my process in the output of top or system monitor.

Then it's not running, and it's impossible for it to be taking up usable memory, unless your program is doing something like allocating a SysV shared memory segment and then forgetting to deallocate it before exiting.

[lehe]

Thanks, brewbuck!

Will "new" allocate a SysV shared memory segment? "new" is the only function I used to allocate memory. What is "allocating a SysV shared memory segment" like?

The OS seems unable to deallocate the memory. Is there some way to do that without restarting OS?

Then it's not running, and it's impossible for it to be taking up usable memory, unless your program is doing something like allocating a SysV shared memory segment and then forgetting to deallocate it before exiting.

[bithub]

Will "new" allocate a SysV shared memory segment?

No, it will allocate memory off the application's heap.

[lehe]

So the OS still can handle the deallocation even if I forgot to call delete.

What else could possibly cause the memory being held? Does it have something to do with the fact that I run the C++ program from Emacs sometimes within GDB ?

Anyway to release the memory?

How can I release the cached part in swap and buffer part in Memory?

[bithub]

There is a way to clear the cached memory, but I'm not going to bother looking up how to do it. I can't think of a valid reason why someone would ever want to do this.

I'm willing to bet that the OS is not holding any of the application's memory after it exits. If your process is not on "top", then it is not using any memory.

[matsp]

So the OS still can handle the deallocation even if I forgot to call delete.

What else could possibly cause the memory being held? Does it have something to do with the fact that I run the C++ program from Emacs sometimes within GDB ?

Anyway to release the memory?

How can I release the cached part in swap and buffer part in Memory?

Are you actually seeing a problem, or are you simply trying to make sense out of some numbers that you do not understand.

You must understand that OS's do a lot of work "under the covers" and use free memory as cache for various purpose. When you first run your application, shared libraries may be loaded for the first time. If there is plenty of free memory (e.g. you have 1GB and only run a basic Linux setup), the OS will not remove the shared libraries from memory just because the application exits.

Likewise, if your application reads from files (including loading shared libraries and other such things), the file-system data that the OS uses to find the files needed will be loaded into memory, and some or all of the file-content as well. Again, if there is plenty of free memory, why should the OS "throw away" that data, when there is nothing else that requires that memory to be used?

If you are running an ordinary standard distribution of Linux, I guarantee that unless your program does really unusual things, the memory it uses will be available to be used.

Caches and such are flushed when the OS needs to use that memory (e.g. if you load a 400MB file into Emacs). But if there is nothing else needing the memory, it remains in use as cache - it will not serve a better purpose just sitting empty, right?

--
Mats

[lehe]

Thanks!
Even after I exit Emacs from where I run my program, the percentage of used memory does not change. Can we say the memory it uses will be available to be used?

[matsp]

Thanks!
Even after I exit Emacs from where I run my program, the percentage of used memory does not change. Can we say the memory it uses will be available to be used?

Unless you are running into problems where starting emacs again (for the umpteenth time, perhaps), yes, I'd say so. Programs only take up memory when they are in use, but the OS doesn't like "free" memory, so it tries to fill all of the memory with something that makes sense. Caching the file-system is one such thing, leaving shared libraries loaded after the app exits is another example. There are probably other things too that are cached, but those are the obvious ones.

Free memory is of no use (unless you have applications that need a lot of memory, but even then caches can be disposed of, especially those that do not need writing to disk first).

--
Mats

[brewbuck]

We have to make a distinction between "unused" memory and "available" memory. As matsp said, the OS tries to completely fill memory with data in a way that will improve system performance. For instance, by caching filesystem blocks. This memory is "in use," but it is also "available" because if an application demands memory, the OS can simply discard the cached data and turn the memory over to the application.

It's notoriously difficult to define how much free memory exists on a system. In my mind, memory is "free" if it can be immediately assigned to an application without needing to copy its contents to backing store. In other words, shared memory pages which are not currently mapped to any process, and are not dirty, are fair game. File system buffers which aren't dirty are fair game. Everything else needs to either be written back to the filesystem or to swap before the OS can hand over the pages to an application.

[Kennedy]

There is a way to clear the cached memory, but I'm not going to bother looking up how to do it. I can't think of a valid reason why someone would ever want to do this.

I'm willing to bet that the OS is not holding any of the application's memory after it exits. If your process is not on "top", then it is not using any memory.

In fact, there is no way to dump the cache (in Linux). The kernel will dump each cache for each device AFTER it is unmounted, however, there is currently no system call that allow a user-space app to clear the system cache. -- I had this problem in an embedded system one time and had to do some really fancy footwork to get a tar command to work properly twice (c then x). Had to umount the FS several times as it was a large data store.

[bithub]

In fact, there is no way to dump the cache (in Linux). The kernel will dump each cache for each device AFTER it is unmounted, however, there is currently no system call that allow a user-space app to clear the system cache. -- I had this problem in an embedded system one time and had to do some really fancy footwork to get a tar command to work properly twice (c then x). Had to umount the FS several times as it was a large data store.

There definitely is a way to dump the cache. It's not a system call, you have to echo something into a /proc. I can look it up if needed (or ask someone at work that knows how to do this). We messed around with it for a little while when we were having cache problems on one of our products.

[brewbuck]

In fact, there is no way to dump the cache (in Linux). The kernel will dump each cache for each device AFTER it is unmounted, however, there is currently no system call that allow a user-space app to clear the system cache. -- I had this problem in an embedded system one time and had to do some really fancy footwork to get a tar command to work properly twice (c then x). Had to umount the FS several times as it was a large data store.

One way to clear the cache is to run this:

/* eat_all_memory.c */

#include <stdlib.h>

int main()
{
    char *page;
    while( page = malloc( 4096 ) )
        *page = 1;
    return 0;
}

Looks sucky, but is portable to pretty much any OS you want