Thread: Allocation of large arrays C++ style crashes program but not C style, why?

  1. #1
    Registered User
    Join Date
    Nov 2015
    Posts
    72

    Allocation of large arrays C++ style crashes program but not C style, why?

    I'm experimenting with large arrays in VisualStudio2013 IDE and I'm noticing that when trying to allocate arrays larger than about 1GB, C++ style, i.e. by using:

    Code:
    double *myArray = new double[numBytes];
    the program crashes. I have traced the crash to this line so it is at the call of the 'new' function it happens.

    But when I allocate C-style, i.e.

    Code:
    double *myArray = (double *)malloc(numBytes);
    the program runs just fine.

  2. #2
    Registered User
    Join Date
    May 2010
    Posts
    4,465
    What is the value of numBytes?

    Is it the same with both snippets?

    How much memory does your system have, what operating system are you using, what compiler, compiler version?

    Do you realize that your first snippet may be allocating more memory than your second snippet?

    Jim

  3. #3
    Registered User
    Join Date
    Nov 2015
    Posts
    72
    Well, I have experimented with many different values of numBytes but it is around 1GB that the crash happens. The value if numBytes is of course the same, try for example '12000*12000*sizeof(double)'.

    I had about 8GB of free RAM according to task manager at the time of executions, according to task manager.

    Operating system: Windows 7 x64 SP1.

    Given the same numBytes, I cannot see how the first code snippet allocates more memory than the second snippet. If that's the case then something is horribly wrong with C++, or the C++ part of the Visual compiler.

    Microsoft (R) C/C++ Optimizing Compiler Version 18.00.31101 for x64


    So what you're saying is that this is a bug within the compiler?

  4. #4
    Registered User
    Join Date
    May 2010
    Posts
    4,465
    Given the same numBytes, I cannot see how the first code snippet allocates more memory than the second snippet. If that's the case then something is horribly wrong with C++, or the C++ part of the Visual compiler.
    No there's nothing wrong with your compiler, it's your understanding of that nasty C function malloc() that is lacking. Your first snippet is allocating doubles, your second snippet is only allocating bytes. Doubles are usually about 8 bytes each.

    Jim

  5. #5
    Informer -Adrian's Avatar
    Join Date
    Jan 2013
    Posts
    820
    ^ Or rather, the understanding of new, since numBytes is misplaced there, not in malloc.

  6. #6
    Registered User
    Join Date
    May 2010
    Posts
    4,465
    Perhaps, but since this is the C++ forum I lean towards misunderstanding malloc(), especially since the OP was under the impression that his malloc() call was allocating enough space for "numBytes" doubles.

    Also since this is the C++ forum the OP should start using the safer C++ style casts.

    Jim

  7. #7
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    The OP should be using std::vector, not new.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  8. #8
    Registered User
    Join Date
    Nov 2015
    Posts
    72
    Yes, I have misunderstood "new", not malloc(). I thought that it wanted number of bytes as input argument whereas the number of elements should be used as input argument. After making the correction my programs run fine with "new" and "delete[]".

    But still, if it fails to allocate memory, why crash the program? Why not just terminate without allocating the memory or, at the very least, throw an exception?

    Yes, std::vector looks really nice and powerful, will use it in the future if it doesn't consume too much of performance.
    Last edited by bot; 11-23-2015 at 01:49 PM.

  9. #9
    Lurking whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    9,612
    Quote Originally Posted by bot View Post
    But still, if it fails to allocate memory, why crash the program? Why not just terminate without allocating the memory or, at the very least, throw an exception?
    new does throw an exception called std::bad_alloc, but any exception that goes uncaught will cause termination eventually.

  10. #10
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Operator new DOES throw an exception if it fails to allocate the memory. If you never catch that exception, it WILL crash your program.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #11
    Registered User
    Join Date
    Nov 2015
    Posts
    72
    I have played around with try and catch(std::bad_alloc &errH) blocks and that seemed to prevent the crash.

    I tried using

    Code:
    int x(999999), y(999999);
    int nBytes(x*y);

    which yielded a large number. But when I used


    Code:
    int x(1<<16), y(1<<16);
    int nBytes(x*y);

    I found that x*y was 0! So


    Code:
    array = new double[0];

    is what caused the crash. I solved it by using


    Code:
    int x(1<<16), y(1<<16);
    int64 nBytes = static_cast<int64>(x)*y;
    Now I realize that allocating 32GB of memory won't crash the system, only create a very big page file. Not at the point of allocation of course but when filling it with data.

    I also realize that 'nBytes' is not the number of bytes but number of elements in this context.
    Last edited by bot; 11-23-2015 at 02:53 PM.

  12. #12
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Quote Originally Posted by bot View Post
    I tried using

    Code:
    int x(999999), y(999999);
    int nBytes(x*y);

    which yielded a large number. But when I used
    Because the lower 32 bits yield 1101 0100 1000 0110 1000 1011 1000 0001 -- i.e. a very large number.


    Code:
    int x(1<<16), y(1<<16);
    int nBytes(x*y);

    I found that x*y was 0! So
    In this case, the lower 32 bits are only 0s.

    Now I realize that allocating 32GB of memory won't crash the system, only create a very big page file. Not at the point of allocation of course but when filling it with data.
    Disable or reduce the page file to a very small file. The page file is evil™.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Proper coding style on large projects
    By Verneg in forum C Programming
    Replies: 0
    Last Post: 03-27-2010, 06:13 PM
  2. C style arrays question
    By Raigne in forum C++ Programming
    Replies: 3
    Last Post: 04-12-2008, 12:06 AM
  3. vectors vs c style arrays
    By markucd in forum C++ Programming
    Replies: 6
    Last Post: 04-20-2006, 11:11 AM
  4. c-style string vs. c++-style strings
    By Mbrio in forum C++ Programming
    Replies: 3
    Last Post: 02-10-2002, 12:26 PM
  5. how do I extract a style from a DWORD style
    By zMan in forum Windows Programming
    Replies: 1
    Last Post: 01-17-2002, 10:09 AM