Problem with multidimensional dynamic array

This is a discussion on Problem with multidimensional dynamic array within the C Programming forums, part of the General Programming Boards category; I declare a two-dimensional dynamic array of size 65536 * 65536, but I can't initialize the entire array[65535][65535]. Code: int** ...

  1. #1
    Registered User
    Join Date
    Sep 2011
    Posts
    78

    Problem with multidimensional dynamic array

    I declare a two-dimensional dynamic array of size 65536 * 65536, but I can't initialize the entire array[65535][65535].

    Code:
    int** array = (int**)malloc(sizeof(int*) * 65536);
    
    for(int i = 0; i < 65536; i++)
    {
        array[i] = (int*)malloc(sizeof(int) * 65536);
    }
    
    for(int x = 0; x < 65536; x++)
    {
        for(int y = 0; y < 65536; y++)
        {
            array[x][y] = 0   //my value, 0 for example
        }
    }
    Any suggestions?
    Thanks in advance for your answers.
    Last edited by drew99; 10-28-2012 at 06:03 AM.

  2. #2
    qny
    qny is offline
    Registered User
    Join Date
    Sep 2012
    Posts
    355
    If your ints are 4-bytes long, that's 16G of memory.

    Suggestion: if you really really really need that many integers, use a disk-based approach
    Suggestion #2: re-think your solution to whatever problem you're dealing with to not need that many values.

  3. #3
    TEIAM - problem solved
    Join Date
    Apr 2012
    Location
    Melbourne Australia
    Posts
    1,375
    Can you process a smaller amount of memory at a time?
    Fact - Beethoven wrote his first symphony in C

  4. #4
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,304
    What is the array for?
    Exactly what do you do with the data in it?
    Where does the data come from?
    How often is it accessed?
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  5. #5
    Registered User
    Join Date
    Jun 2005
    Posts
    6,290
    Check each malloc() call to see if it succeeds. malloc() returns NULL if it fails. NULL should not be dereferenced, so your initialisation loop exhibits undefined behaviour if any of the malloc() calls have failed.

    At least one of the malloc() calls will fail if your host environment is 32 bits or less.

    Generally, it is a rare program that justifiably needs to create arrays (dynamically or statically) of such a size.
    Right 98% of the time, and don't care about the other 3%.

  6. #6
    Registered User
    Join Date
    Oct 2011
    Posts
    847
    If you happen to work with enormous data sets, much larger than what can fit in your RAM, make sure you have a 64-bit machine and use memory-mapping techniques.

    Some time ago, I wrote an example here that creates an ephemeral (unlinked while open) sparse file to hold a terabyte-sized memory structure. In other words, you do not need a terabyte of swap, or disk space; just enough to hold the pages you modify.

    In most cases you get better performance if you can stream the data, not hold the entire data structure in memory. If the data structure is truly that large, then you can always use one or more smaller mappings to the file, so you won't waste your RAM for the virtual memory overhead (kernel structures needed for the mappings).

    That said, there is nothing wrong in doing
    Code:
    size_t  rows = 65536;
    size_t  cols = 65536;
    double *matrix = malloc(rows * cols * sizeof (double));
    to allocate a 65536x65536-element matrix, if you really do need one. You just need a 64-bit machine with over 32G of RAM, or it will either fail or trash badly (swap a lot, slowing everything down to snail speed).

    Even on 32-bit architectures you can usually have a 65536x65536 matrix, if you limit to one bit (or maybe just a few bits) per element, as that's just 512MB per bit per matrix ( rows * (cols / CHAR_BIT) chars). Using accessor macros or inline functions makes it quite straightforward, too.

  7. #7
    Registered User
    Join Date
    Sep 2011
    Posts
    78
    Solved.
    Thank tou for your help.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. dynamic allocation of multidimensional array
    By lancehumiston in forum C++ Programming
    Replies: 5
    Last Post: 02-21-2012, 05:06 AM
  2. Question on dynamic multidimensional arrays
    By TheLegend219 in forum C++ Programming
    Replies: 2
    Last Post: 09-28-2010, 10:30 AM
  3. Passing multidimensional/dynamic array to function
    By epyfathom in forum C Programming
    Replies: 2
    Last Post: 04-02-2009, 05:39 PM
  4. dynamic multidimensional arrays as function arguments
    By magda_k in forum C++ Programming
    Replies: 1
    Last Post: 03-20-2006, 03:00 PM
  5. dynamic allocation of multidimensional arrays in c++
    By Unregistered in forum C++ Programming
    Replies: 2
    Last Post: 02-15-2002, 03:18 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21