leaked memory not freed after process is killed?

nantonop · 09-03-2007, 07:04 AM

hi,

i'm trying to verify and track down what seems to be a memleak in a C program that runs in daemon fashion 24/7. I'm using Ubuntu. The process crashes every few days due to lack of memory apparently...

My doubt is that if there was a memleak wouldn't the memory be freed after the process was killed by the OS? Is there a chance that the memory is not released to the OS even if the allocator process is dead?

thanks

matsp · 09-03-2007, 07:14 AM

Yes, if the process is killed, then the memory that process owns will be released. There are some forms of memory that may not be freed (until ALL processes owning it are killed), such as shared memory between processes. But normal memory allocated with malloc or related functions should be freed.

You may want to investigate one of these two methods:
1. Use a library with "memory leak detection".
2. Add your own extended malloc/free functions that keep track of where and how much memory is allocated, and every now and again run through to see how memory is allocated. Print any difference between two runs to a log-file or some such.

--
Mats

nantonop · 09-03-2007, 08:07 AM

i've actually been using valgrind to get an indication of what's going on since the application is rather big... It get's a bit messy since there are external libs used and to this point i can't be 100% sure who's causing the leak... thanks for your suggestion!

matsp · 09-03-2007, 08:17 AM

by defining your own allocation, you could use a macro like:

Code:

#define malloc(x) mymalloc(x, __LINE__, __FILE__)
#define free(p) myfree(p, __LINE__, __FILE__)

Then implement malloc() and free() to allocate a bit of extra memory, where you store [before the actual data] a structure like this:

Code:

struct memnode {
   struct memnode *next; 
   size_t size;
   char *file;
   int      line;
   int      seen;
   
};

Every time you "scan" the list, which means printing the info for "not yet seen" and updating "seen" to say "this is 'old'", [you'll obviously have to find a place where this is done now and again - if you have lots of calls to mymalloc, you may want to scan every 100 calls to mymalloc, for example]. If there are lots of new ones EVERY time, it may be an indication that there's something amiss. But some new items is probably expected.

--
Mats

**Salem** · 09-03-2007, 11:02 AM

Quote:

Originally Posted by gnu ld manual

--wrap symbol
Use a wrapper function for symbol. Any undefined reference to symbol will be resolved to __wrap_symbol. Any undefined reference to __real_symbol will be resolved to symbol. This can be used to provide a wrapper for a system function. The wrapper function should be called __wrap_symbol. If it wishes to call the system function, it should call __real_symbol. Here is a trivial example:

Code:

    void *
    __wrap_malloc (int c)
    {
      printf ("malloc called with %ld\n", c);
      return __real_malloc (c);
    }

If you link other code with this file using --wrap malloc, then all calls to malloc will call the function __wrap_malloc instead. The call to __real_malloc in __wrap_malloc will call the real malloc function. You may wish to provide a __real_malloc function as well, so that links without the --wrap option will succeed. If you do this, you should not put the definition of __real_malloc in the same file as __wrap_malloc; if you do, the assembler may resolve the call before the linker has a chance to wrap it to malloc.

Hopefully, you'll have an easy way of making the program run in 'non-daemon' mode for the purposes of debugging and testing.

Another option is to use Electric Fence (gcc prog.c -lefence) to also diagnose mis-use of allocated memory.

Use the 'top' (or 'ps') command periodically to see how much memory is allocated to each process. If you see this creeping up for your program, then it is definitely leaking memory.

nantonop · 09-04-2007, 02:48 AM

Thanks for your suggestions. i shall be having a go at the malloc wrapper asap. However check this out: This is a 'top' for my situation. I'm running the app thorugh valgrind (appears as 'memcheck'). The server has now been up for 5 days and the app is still ok. But what you can make of this is that the total memory usage report (which is that about all mem is used and a small bit of swap too) has no correlation to the mem allocated to running processes. memcheck takes up 7.4% and no other process takes up more than 1% (cut short for brevity - take my word...)... How can this be? I do have the feeling that info from 'top' is not very reliable...

Code:

top - 12:12:18 up 5 days, 49 min,  2 users,  load average: 2.56, 2.98, 2.86
Tasks:  92 total,   2 running,  89 sleeping,   0 stopped,   1 zombie
Cpu(s): 43.8% us,  1.5% sy,  0.0% ni, 54.1% id,  0.0% wa,  0.2% hi,  0.5% si
Mem:   2575056k total,  2556844k used,    18212k free,   174936k buffers
Swap:  2000084k total,      176k used,  1999908k free,  1860680k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 5560 root      15   0  574m 185m 2884 S   91  7.4   5865:02 memcheck
10176 backuppc  20   0  105m  78m 1308 S    0  3.1  51:58.58 BackupPC_dump
 4868 mysql     15   0  143m  22m 4952 S    0  0.9   0:28.49 mysqld
 5402 gdm       16   0 67508  14m 6992 S    0  0.6   0:01.97 gdmgreeter
 4633 root      16   0 53172 9764 3992 S    0  0.4   0:14.70 Xorg

**Salem** · 09-04-2007, 02:56 AM

Yeah, perhaps top was not the thing to use - by default, it measures the CPU time. Perhaps there is a way to get top to display other parameters as being the top-most.

Which your daemon may not use that much of.

So perhaps 'ps', restricted to the PIDs of interest would be a better command.

matsp · 09-04-2007, 02:59 AM

Quote:

Originally Posted by Salem

Yeah, perhaps top was not the thing to use - by default, it measures the CPU time. Perhaps there is a way to get top to display other parameters as being the top-most.

Run top, then press M (can't remember if it's upper or lower case M) and it will show the top memory users [I think that is what was posted].

@nantonop: Could you edit your post to have code-tags around the data you posted, so that the columns line up - it's so much easier to read that way.

--
Mats

nantonop · 09-04-2007, 03:07 AM

sorry found it! see previous post. top sorted by 'M' and text alligned. thanks

matsp · 09-04-2007, 03:25 AM

From what I can determine:
This is essenitally free memory 1860680k cached
See: http://gentoo-wiki.com/FAQ_Linux_Memory_Management

So you have plenty of free memory available.

--
Mats

brewbuck · 09-04-2007, 11:11 AM

Quote:

Originally Posted by Salem

Yeah, perhaps top was not the thing to use - by default, it measures the CPU time. Perhaps there is a way to get top to display other parameters as being the top-most.

Hit the 'm' key to sort by memory usage.

brewbuck · 09-04-2007, 11:13 AM

Quote:

Originally Posted by nantonop

Thanks for your suggestions. i shall be having a go at the malloc wrapper asap. However check this out: This is a 'top' for my situation. I'm running the app thorugh valgrind (appears as 'memcheck'). The server has now been up for 5 days and the app is still ok. But what you can make of this is that the total memory usage report (which is that about all mem is used and a small bit of swap too) has no correlation to the mem allocated to running processes. memcheck takes up 7.4% and no other process takes up more than 1% (cut short for brevity - take my word...)... How can this be? I do have the feeling that info from 'top' is not very reliable...

Typical misunderstanding of the output of "top."

Your memory usage is fine. Just because "Free" says only 18 megs doesn't mean you've only got 18 megs free. That's not what "free" means.

QuantumPete · 09-04-2007, 11:43 AM

Quote:

Originally Posted by nantonop

The process crashes every few days due to lack of memory apparently...

You'd be leaking a huge amount of memory if that's true. I'd check whether you free all memory when you encounter an error condition, it's easy to return from a function early and forget to free all allocated memory.

QuantumPete

matsp · 09-04-2007, 01:34 PM

Quote:

Originally Posted by brewbuck

Typical misunderstanding of the output of "top."

Your memory usage is fine. Just because "Free" says only 18 megs doesn't mean you've only got 18 megs free. That's not what "free" means.

Do I have it right that it tells us how much memory is IMMEDIATELY available, and the cached number as I mentioned earlier is there to be shrunk if need be, to allow more memory to be used.

I agree that leaking enough memory to use up 2GB in a matter of days is serious leakage.

--
Mats

brewbuck · 09-04-2007, 02:30 PM

Quote:

Originally Posted by matsp

Do I have it right that it tells us how much memory is IMMEDIATELY available, and the cached number as I mentioned earlier is there to be shrunk if need be, to allow more memory to be used.

Pretty much. The cache can shrink instantly, so there's no real difference between cache and "free" memory.

Of course, just adding cache + free doesn't give you the right number either. It's actually pretty hard to define exactly how much memory is "free" at any one moment.

09-03-2007, 07:04 AM	#1
nantonop Registered User Join Date: Apr 2007 Posts: 13	leaked memory not freed after process is killed? hi, i'm trying to verify and track down what seems to be a memleak in a C program that runs in daemon fashion 24/7. I'm using Ubuntu. The process crashes every few days due to lack of memory apparently... My doubt is that if there was a memleak wouldn't the memory be freed after the process was killed by the OS? Is there a chance that the memory is not released to the OS even if the allocator process is dead? thanks
nantonop is offline

09-03-2007, 07:14 AM	#2
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	Yes, if the process is killed, then the memory that process owns will be released. There are some forms of memory that may not be freed (until ALL processes owning it are killed), such as shared memory between processes. But normal memory allocated with malloc or related functions should be freed. You may want to investigate one of these two methods: 1. Use a library with "memory leak detection". 2. Add your own extended malloc/free functions that keep track of where and how much memory is allocated, and every now and again run through to see how memory is allocated. Print any difference between two runs to a log-file or some such. -- Mats
matsp is offline

09-03-2007, 08:07 AM	#3
nantonop Registered User Join Date: Apr 2007 Posts: 13	i've actually been using valgrind to get an indication of what's going on since the application is rather big... It get's a bit messy since there are external libs used and to this point i can't be 100% sure who's causing the leak... thanks for your suggestion!
nantonop is offline

09-03-2007, 08:17 AM	#4
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	by defining your own allocation, you could use a macro like: Code: #define malloc(x) mymalloc(x, __LINE__, __FILE__) #define free(p) myfree(p, __LINE__, __FILE__) Then implement malloc() and free() to allocate a bit of extra memory, where you store [before the actual data] a structure like this: Code: struct memnode { struct memnode next; size_t size; char file; int line; int seen; }; Every time you "scan" the list, which means printing the info for "not yet seen" and updating "seen" to say "this is 'old'", [you'll obviously have to find a place where this is done now and again - if you have lots of calls to mymalloc, you may want to scan every 100 calls to mymalloc, for example]. If there are lots of new ones EVERY time, it may be an indication that there's something amiss. But some new items is probably expected. -- Mats
matsp is offline

09-04-2007, 02:48 AM	#6
nantonop Registered User Join Date: Apr 2007 Posts: 13	Thanks for your suggestions. i shall be having a go at the malloc wrapper asap. However check this out: This is a 'top' for my situation. I'm running the app thorugh valgrind (appears as 'memcheck'). The server has now been up for 5 days and the app is still ok. But what you can make of this is that the total memory usage report (which is that about all mem is used and a small bit of swap too) has no correlation to the mem allocated to running processes. memcheck takes up 7.4% and no other process takes up more than 1% (cut short for brevity - take my word...)... How can this be? I do have the feeling that info from 'top' is not very reliable... Code: top - 12:12:18 up 5 days, 49 min, 2 users, load average: 2.56, 2.98, 2.86 Tasks: 92 total, 2 running, 89 sleeping, 0 stopped, 1 zombie Cpu(s): 43.8% us, 1.5% sy, 0.0% ni, 54.1% id, 0.0% wa, 0.2% hi, 0.5% si Mem: 2575056k total, 2556844k used, 18212k free, 174936k buffers Swap: 2000084k total, 176k used, 1999908k free, 1860680k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5560 root 15 0 574m 185m 2884 S 91 7.4 5865:02 memcheck 10176 backuppc 20 0 105m 78m 1308 S 0 3.1 51:58.58 BackupPC_dump 4868 mysql 15 0 143m 22m 4952 S 0 0.9 0:28.49 mysqld 5402 gdm 16 0 67508 14m 6992 S 0 0.6 0:01.97 gdmgreeter 4633 root 16 0 53172 9764 3992 S 0 0.4 0:14.70 Xorg Last edited by nantonop; 09-04-2007 at 03:12 AM.
nantonop is offline

09-04-2007, 02:56 AM	#7
Salem and the hat of Jobseeking Join Date: Aug 2001 Location: The edge of the known universe Posts: 21,680	Yeah, perhaps top was not the thing to use - by default, it measures the CPU time. Perhaps there is a way to get top to display other parameters as being the top-most. Which your daemon may not use that much of. So perhaps 'ps', restricted to the PIDs of interest would be a better command. __________________ If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
Salem is offline

09-04-2007, 03:07 AM	#9
nantonop Registered User Join Date: Apr 2007 Posts: 13	sorry found it! see previous post. top sorted by 'M' and text alligned. thanks Last edited by nantonop; 09-04-2007 at 03:16 AM.
nantonop is offline

09-04-2007, 03:25 AM	#10
matsp Kernel hacker Join Date: Jul 2007 Location: Farncombe, Surrey, England Posts: 15,686	From what I can determine: This is essenitally free memory 1860680k cached See: http://gentoo-wiki.com/FAQ_Linux_Memory_Management So you have plenty of free memory available. -- Mats
matsp is offline

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How can you make a parent process wait for a child? I'm gettin a seg fault.	mr_coffee	C Programming	3	10-15-2008 09:24 AM
To find the memory leaks without using any tools	asadullah	C Programming	2	05-12-2008 07:54 AM
Problem with forking a process	Unitedroad	C Programming	10	10-04-2007 01:43 AM
process programming	St0rM-MaN	Linux Programming	2	09-15-2007 07:53 AM
Suggestions on this C style code	Joelito	C Programming	11	06-07-2007 03:22 AM