An error about size of an array

This is a discussion on An error about size of an array within the C Programming forums, part of the General Programming Boards category; I have a question about an error in a C program. I need to define a float array as follow: ...

  1. #1
    Registered User
    Join Date
    Jun 2007
    Posts
    6

    An error about size of an array

    I have a question about an error in a C program.
    I need to define a float array as follow:

    float rad[1000][1000000];

    But this error is occured:
    size of variable 'rad' is too large

    I need to define an array with those sizes. So, what should I do in order to solve the problem? Please help me if anyone knows the solution.

    Thanks so much

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    I'm not surprised. You want 1000 * 1000000 * sizeof(float) bytes. That is about 4GB, which for a 32-bit application is "too large".

    If you can use a 64-bit OS and a 64-bit compiler, you could probably solve the problem that way.

    However, I would seriously consider your needs - because even if you manage to build an application that "works" for this size of an array, you will probably find that it's quite slow unless your machine also has a huge amount of memory [at least 5GB].

    For most problems like this [that is, where you have a "need" for a large array], there are alternative solutions, where you only actually store "useful" data, and "standard values are ignored".

    If you can't use a 64-bit OS for some reason, you will DEFINITELY have to rethink the design of your software.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    You'll need to acquire the memory from the heap (general processor RAM) instead of using the stack. Look to malloc().

    Todd

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Todd Burch View Post
    You'll need to acquire the memory from the heap (general processor RAM) instead of using the stack. Look to malloc().

    Todd
    In a 32-bit machine, that won't make ANY difference.

    Of course, using a approx 4GB stack based variable would be complete lunacy - even in a 64-bit system.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  5. #5
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    Quote Originally Posted by matsp View Post
    In a 32-bit machine, that won't make ANY difference.
    1,000 * 1,000,000 * 4 (length of float)= 4,000,000,000 = 0xEE6B 2800 (ala, < 4GB.)

    Why won't it work? Are lengths treated as signed values?

    Todd

  6. #6
    Registered User
    Join Date
    Jun 2007
    Posts
    6
    Could you please tell me what is a malloc() code program which is equivalent to rad[1000][1000000]

  7. #7
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    p.s. And yes, it's lunacy to request this much storage.

  8. #8
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Todd Burch View Post
    1,000 * 1,000,000 * 4 (length of float)= 4,000,000,000 = 0xEE6B 2800 (ala, < 4GB.)

    Why won't it work? Are lengths treated as signed values?

    Todd
    It is less than 4GB by a bit more than 256MB. But for any 32-bit OS that has any memory management at all, the > 3GB is sufficient to make it not work.

    Quote Originally Posted by yazdan59
    Could you please tell me what is a malloc() code program which is equivalent to rad[1000][1000000]
    Code:
    rad = malloc(sizeof(float) * 1000 * 1000000)
    But that will not work any better if you don't have a machine that supports more than 3GB for application address spaces, so you need a 64-bit OS [or a custom built OS].

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  9. #9
    Registered User
    Join Date
    Nov 2007
    Posts
    73
    you can do one thing....

    just create a memory space when it is needed usin maloc and cear of the space usig free if you r done with it....
    why do you want to allocate so much of space at a time...
    allocate it before you take an input.......

    more details read dma

  10. #10
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,627
    Quote Originally Posted by Todd Burch View Post
    1,000 * 1,000,000 * 4 (length of float)= 4,000,000,000 = 0xEE6B 2800 (ala, < 4GB.)

    Why won't it work? Are lengths treated as signed values?

    Todd
    I believe each application is limited to 2 GB of (virtual) memory, aren't they (on x86)?
    But even so, good luck allocating much more than 1 GB of memory - it just won't work, because you'll find that there's no such thing as 1 GB of contiguous virtual memory available! It's all fragmented a little bit here and there (why it doesn't allocate in a contiguous manner beats me).
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #11
    Ex scientia vera
    Join Date
    Sep 2007
    Posts
    478
    Quote Originally Posted by Elysia View Post
    I believe each application is limited to 2 GB of (virtual) memory, aren't they (on x86)?
    But even so, good luck allocating much more than 1 GB of memory - it just won't work, because you'll find that there's no such thing as 1 GB of contiguous virtual memory available! It's all fragmented a little bit here and there (why it doesn't allocate in a contiguous manner beats me).
    No, all 32-bit operating systems can handle up to 4gb of memory(Eg, 2^32). 64-bit operating systems are thus limited to 2^64 bytes, which I think is something like 16-18 exabytes.

  12. #12
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,627
    Please differentiate between can handle and limited/reserved. Yes, 32-bit can address 4 GB of memory, but a lot of virtual memory for each process is reserved (I believe it's the lower half of the memory space) plus Windows reserves a bit of memory space for itself, so it can't address all 4 GB.

    EDIT:
    Operating systems based on Microsoft Windows NT technologies have always provided applications with a flat 32-bit virtual address space that describes 4 gigabytes (GB) of virtual memory. The address space is usually split so that 2 GB of address space is directly accessible to the application and the other 2 GB is only accessible to the Windows executive software.
    Each application only has access to 2 GB virtual memory.
    Last edited by Elysia; 01-02-2008 at 12:56 PM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  13. #13
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,243
    Quote Originally Posted by Elysia View Post
    I believe each application is limited to 2 GB of (virtual) memory, aren't they (on x86)?
    The limitation has nothing to do with the architecture. The OS is implemented with a certain chunk of virtual address space reserved for its own purposes while in kernel mode. If all 4 gigabytes were reserved for user space, how would the kernel be able to access its internal data structures and user application data at the same time? To make things easier the kernel just says that the top gig (or two gig) of virtual address space is reserved for the kernel, period.

    An OS could be written or modified to allow the user application access to more memory, but at that point it seems preferable to go to a 64-bit architecture anyway.

    Please differentiate between can handle and limited/reserved. Yes, 32-bit can address 4 GB of memory, but a lot of virtual memory for each process is reserved (I believe it's the lower half of the memory space)
    It's reserved because the OS says it is. There's no reason it couldn't be made available, it just isn't.

  14. #14
    C++まいる!Cをこわせ! Elysia's Avatar
    Join Date
    Oct 2007
    Posts
    22,627
    Yeah, point being, just because the OS is 32-bit doesn't mean you can access and use all that 4 GB of virtual memory. On Windows (NT), each application only has 2 GB to play around with.
    So all calls with > 2GB automatically fails.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  15. #15
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    On IBM mainframes (my day job), prior to the 64 bit OS, we had (actually still have) 31 (not 32) bit addressing. An address space (aka, a logged on user for simplicity sake) was 2GB-1 bytes long. Reducing out the system resident code and the common areas of memory available to all address spaces, the rest is available for the user (or application). If you needed more memory than 2GB, you simply told the operating system you needed another address space, and thus you got another "2GB", and so on and so on. In theory, you can have 32,767 address spaces (minus the address spaces required to keep the system running.

    Now with the 64-bit OS, you get 16 exabytes (10**18) virtual in each address space. It's obscene.

    Todd

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. size of array
    By goran00 in forum C Programming
    Replies: 38
    Last Post: 04-02-2008, 09:57 AM
  2. Invalid conversion from 'void*' to 'BYTE' help
    By bikr692002 in forum C++ Programming
    Replies: 9
    Last Post: 02-22-2006, 10:27 AM
  3. Unknown Memory Leak in Init() Function
    By CodeHacker in forum Windows Programming
    Replies: 3
    Last Post: 07-09-2004, 09:54 AM
  4. Type and nontype parameters w/overloading
    By Mr_LJ in forum C++ Programming
    Replies: 3
    Last Post: 01-02-2004, 12:01 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21