Large Arrays

This is a discussion on Large Arrays within the C++ Programming forums, part of the General Programming Boards category; I'm currently writing a program that uses several large arrays to store data. It's important that the data is quickly ...

  1. #1
    Registered User
    Join Date
    Aug 2003
    Posts
    3

    Question Large Arrays

    I'm currently writing a program that uses several large arrays to store data. It's important that the data is quickly accessible throughout the program, but I need to store a lot of it.

    The problem is, if I try to allocate more than about 500000 memory slots, the program crashes when I run it. What is the reason for this? My program only uses about 500 kB of memory when it is just small enough to run. Is there a trick to allocating large chunks of memory? I have tried writing programs that do nothing but declare arrays, and they still crash when I allocate around 500000 slots.

    To edit and compile my code I use Dev-C++ on a Windows 2000 machine.

    Thanks.
    XuRumin

  2. #2
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    Are you allocating it dynamically or statically?
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  3. #3
    *******argv[] - hu? darksaidin's Avatar
    Join Date
    Jul 2003
    Posts
    314
    You should use pointers and dynamic memory. Static arrays are created at compile time and have direct impact on the size of your executable. I can only guess that your compiler limited your array size and that causes a crash.

    Do something like...

    Code:
    byte *pByte;
    pByte= new byte[4294967296];
    
    
    ....
    
    pByte[<position>] = <value>;
    and...
    Code:
    delete [] pByte;
    when you're done with it.

    edit: those semicolons always...
    Last edited by darksaidin; 08-26-2003 at 04:38 PM.
    [code]

    your code here....

    [/code]

  4. #4
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    >>Static arrays are created at compile time and have direct impact on the size of your executable.
    No they don't. They just suck up stack space, which is very limited.
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  5. #5
    *******argv[] - hu? darksaidin's Avatar
    Join Date
    Jul 2003
    Posts
    314
    Originally posted by XSquared
    >>Static arrays are created at compile time and have direct impact on the size of your executable.
    No they don't. They just suck up stack space, which is very limited.
    edit: I guess I was wrong then. Maybe that was the way it was done in Delphi. Anyway, doesn't matter....
    Last edited by darksaidin; 08-26-2003 at 04:54 PM.
    [code]

    your code here....

    [/code]

  6. #6
    Registered User
    Join Date
    Aug 2003
    Posts
    3

    hmm...

    Thanks for the replies-- that was fast!

    I have thought about dynamic memory allocation, and I'll try it, but I don't know if it will help. Like I said, I need all of the data to be available throughout the program. In other words, there is no point where I can just delete one of my arrays, because I need to save them all while it's running.

    If it's true that the stack size limits my array space that severely, is there another way of storing data without slowing down my program significantly? What is the usual way that programmers implement large look-up tables, for example?

  7. #7
    *******argv[] - hu? darksaidin's Avatar
    Join Date
    Jul 2003
    Posts
    314
    You can create the array in one of the first lines of your code and delete it when the program terminates.

    Instead of new and delete you could use malloc/calloc/realloc and free, but it's just another way to allocate dynamic memory and it is not safe for objects.


    Another way would be to implement a class that does the allocation and deletion for you. You could than statically create an instance of the class and wouldn't need to take care of new/delete. That class would delete the memory in it's destructor which will be automatically called on program termination.

    edit: A vector (#include <vector>) is a templated class that does something very similar. You can use it almost like an array, but I don't know how fast it is. I guess sequential appending to a vector with push_back could be pretty slow, but it also allows you to set it's size and then assign elements.

    Have a look here
    Last edited by darksaidin; 08-26-2003 at 05:30 PM.
    [code]

    your code here....

    [/code]

  8. #8
    Registered User
    Join Date
    Aug 2003
    Posts
    3

    Thumbs up dynamic allocation

    Dynamic allocation totally solved the problem. Thanks guys!

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Large 2D Array's Problem
    By kas2002 in forum C Programming
    Replies: 4
    Last Post: 05-25-2009, 01:42 PM
  2. Allocating a number of large 3d arrays in C
    By Surrender in forum C Programming
    Replies: 22
    Last Post: 08-19-2008, 09:36 AM
  3. Large arrays, malloc, and strcmp
    By k2712 in forum C Programming
    Replies: 1
    Last Post: 09-24-2007, 09:22 PM
  4. Problem declaring large arrays
    By mattAU in forum C Programming
    Replies: 5
    Last Post: 09-28-2006, 06:47 AM
  5. Replies: 4
    Last Post: 01-30-2005, 04:50 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21