Really Stumped - Help :)

[brett]

Hi,
I written a rather large program in C that runs on Web Servers, with probably about 50 different functions, some very long others short. For a long time all I used were arrays for opening files, storing data etc. Since some files lately were getting too large for normal arrays I used malloc() dynamic memory allocation.

I've come across a strange segmentation fault on only 1 server so far - not any other servers yet. I have used malloc to define a number of memory pointers in the main function and then the main function calls a smaller sub function. This sub function has its own malloc calls and at the end of this sub function free(pointer); is used to free up the memory. Now for some reason upon return to the main function again the segmentation fault happens. For the life of me I can't figure out why, and it works fine on other servers, its just this one server so far that keeps giving segmentation faults at this point.

I tried renaming the malloc pointers, shuffling things around etc, it always fails on the "return" -> end of the sub function.

Is there some memory issue you need to know about using malloc? This server in question has 2GB ram, plenty, my functions on use up about 1 or 2 MBs.

Anyone got any ideas? Thoughts? Little tricks of the trade?

Many thanks to anyone who responds

[maven]

Make a check when u r doing malloc.

Hint : Malloc returns Null if it is not able to allocate memory.

[brett]

Thanks for response

Yes I have a NULL check after the malloc declarations.

I know there is a 10MB limit for arrays in C programs. I thought the reason for using malloc was to solve this memory limitation problem. When using malloc over multiple functions does that have the same 10MB memory limit? Does it accumulate? For example if I use malloc to allocate 5MB in main() then call a sub function that allocates 6MB via malloc - will that cause problems? I thought not?

[maven]

Write some debug statement and check, exacltly where it is failing.
I think it might help to understand the problem.

[vart]

seg fault on return is mainly cause by the corrupted stack, which can occure when the buffer overrun is performed by some operation.

I know there is a 10MB limit for arrays in C programs.

Where do you get this strange idea?

[brett]

Thanks again for your response

I have done this very carefully. I have an array called 'debug', I use the following to write to the array all during the code:

strcat(debug, "End of generate_tasks function\n");
save_debug();

the save_debug function saves it to file each time. So I know exactly where it fails. As I said above it fails right on "return;". In the main() function I have a debug statement right after the function call and this debug statement never saves - so the return never happens - seems on closure of the function it crashes as the last comment in debug file is "End of generate_tasks function\n" - I can't figure out why this is happening.

[vart]

Thanks again for your response

I have done this very carefully. I have an array called 'debug', I use the following to write to the array all during the code:

strcat(debug, "End of generate_tasks function\n");
save_debug();

the save_debug function saves it to file each time. So I know exactly where it fails. As I said above it fails right on "return;". In the main() function I have a debug statement right after the function call and this debug statement never saves - so the return never happens - seems on closure of the function it crashes as the last comment in debug file is "End of generate_tasks function\n" - I can't figure out why this is happening.

for example if your strcat overruns a buffer

[brett]

Not sure exactly what you mean but very much appreciate your involvement, welcome and thanks

The debug array declaration is:

char debug[500000]={'\x00'};

Its very large and the debug file thats save is les than 10KB in length - so the debug array is not full at all. Is that what you meant?

[brett]

Heres the malloc declarations in the sub function that I beleive could be causing the problem:

char *trueuserowners1 = malloc(200000*sizeof(char));
char *rawresellerlist1 = malloc(800000*sizeof(char));
char *activeuserlist1 = malloc(400000*sizeof(char));
char *passwd1 = malloc(500000*sizeof(char));
char *subresellers1 = malloc(400000*sizeof(char));
char *allresellerlist1 = malloc(400000*sizeof(char));
                                                                                                                             
if(rawresellerlist1 == NULL || activeuserlist1 == NULL || passwd1 == NULL || trueuserowners1 == NULL || subresellers1 == NULL || allresellerlist1 == NULL)
        {
        strcat(debug, "MEMORY ALLOCATION FAILURE\n");
        save_debug();
        fprintf(stderr, "Couldn't allocate memory\n");
        exit(EXIT_FAILURE);
        }

[brett]

Actually I just took all the malloc statements out of this function and went back to simple arrays. Same problem still happens....hmmm strange. Something about this server I think. How can you check a server memory limits for running programs? Anyone know?

[vart]

Could you post the whole code of the function that crashes on the return? or it is too long?

[brett]

Not really possible - too long. the thing is the program works fine on every other server I've tested ???? So I don't know??? Driving me nuts

[Salem]

> char *trueuserowners1 = malloc(200000*sizeof(char));
You really don't know how much memory you really need, so you just allocate big numbers and hope that it's enough ?
That's just asking for trouble in a web server environment.

> Now for some reason upon return to the main function again the segmentation fault happens.
That could be anything, but the usual symptom is trashing memory somewhere.

> and it works fine on other servers.
Which in itself proves nothing, except that the code is broken.
"Working" != "Bug Free" no matter how many machines it seems to work on, and how few it crashes on.

Two approaches
1. Use electric fence and a debugger
gcc prog.c -lefence
gdb a.out
With any luck, it will drop you into the debugger the first time you do something iffy with any dynamically allocated memory.

2. Use valgrind
valgrind a.out
Produces copious output about all suspicious references to memory.
It can also make the program significantly slower to run, so if you have large data files which take a long time to run, then prepare smaller ones for testing with.

Try both with really simple test programs with known faults, so you can recognise the fault when you see it in your main program.
Eg.

#include <stdlib.h>

void foo ( void ) {
    char *p = malloc(10); /* this one leaks */
    p = malloc(10);
    p[10] = '\0'; /* out of bound */
    free(p);
    free(p);  /* "Britney" oops, I freed it again */
}
int main ( ) {
    foo();
    return 0;
}

[brett]

Thank you heaps

I really appreciate that - this is exactly what I will do tomorrow, its very late now where I live so I will be sleeping now. But thank you very much for explaing all of that, I now know what approaches to take to find the root cause, narrow it down.