Thread: How to may a REALLY big array?

Hi,

I want to make a BIG array with about 10M elements, but it seems like I'm overflowing the stack, if I do an array of 500 000 elements its ok, but when I go to 5 000 000 I get segmentation fault.

basically this is all I do:

Code:

#define SAMPLES 5000000;

int
main(void)
{
struct block_data bd[SAMPLES];
}

the I save a lot of data in bd, but the seg fault comes before that. tried valgrind and it points to where I initialize variables the lines under I defined bd, so I think I'm overflowing the stack ... i've also try to allocate memory like this:

Code:

    struct block_data *bd[SAMPLES];
    uint64_t i = 0;

    while(i<SAMPLES) {
        bd[i] = (struct block_data *)malloc(sizeof(struct block_data));
        ++i;
    }

but still segementationfault.


So how can I do this?

Originally Posted by matsp

I doubt your first example even compiles, but you are trying to create a local array that is quite large, so you probably run out of stack-space.

Likewise if you create a local pointer array.

If you need really large amounts of data, global variable, or allocating the memory through a pointer will avoid stack overflow.

Most systems have a limit of around 2MB of stack-space.

--
Mats

so how do I "allocating the memory through a pointer", thats what I'm trying to do in my second code section, what is wrong?

Originally Posted by Elysia

You're creating too many pointers. The stack still overflows.

Ahh, ok, but how to allocate memory the correct way then?

Originally Posted by Elysia

You are doing it correct. You're just using too much of it.
You can increase the stack size, make it global or avoid it by using lesser pointers. You can, for example, group several pointers into a struct and allocate it on the heap, thus reducing the amount of pointers you use.

Ok, thanks you for your answers, but I'm kind of new on the memory management in C, so how do I increase the stack size? how do I allocate memory on the heap?

Don't malloc use the heap? Are the pointers at the heap but the array of pointers at the stack which I'm overflowing? But how do I put the array of pointers at the heap? and how do I increase the size of the heap if I need to?

Originally Posted by matsp

The code I posted allocates the array on the heap. And yes, malloc() and it's close relatives give you space on the heap. Since the heap can easily be 100-1000 times larger than the stack, you should be OK.

You should, however check if bd == NULL or bd[i] == NULL after the respective calls to malloc, and if so print something and exit the program, as this indicates that there wasn't enough memory.

Increasing the stack is something you need to tell the compiler as part of teh building of the application.

--
Mats

Ahh, ok thanks will try your code, must have been writing my post same time as you did. just saw it.

thanks matsp, your code works perfect, thanks, problem solved.