Can I malloc a space larger than 3G?

This is a discussion on Can I malloc a space larger than 3G? within the C++ Programming forums, part of the General Programming Boards category; We know that a process memory space is limited by 4G (1G for kernel). I am thinking if I can ...

  1. #1
    Registered User
    Join Date
    Apr 2007
    Posts
    284

    Can I malloc a space larger than 3G?

    We know that a process memory space is limited by 4G (1G for kernel).
    I am thinking if I can malloc a space larger than 3G. So I tried the following:

    Code:
    int main()
    {
    	char* x = (char*)malloc(3*1024*1024*1024);
    	memset(x,'m',3*1024*1024*1024);
    	printf("%s",x);
    	return 0;
    }
    AND It works! But the print out result isn't that nice.....
    Anybody knows why?

    I also tried
    Code:
    char x[0x7fffffff];
    This time compiler complains for its size. So I guess the implementations of malloc and char[] are different, right?

  2. #2
    Registered User whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    7,759
    >> why?

    printf( "%s", x ); prints x as a C string. A C string must be zero terminated.

    >> This time compiler complains for its size. So I guess the implementations of malloc and char[]
    >> are different, right?

    Stack memory is much smaller than the heap.

  3. #3
    Registered User
    Join Date
    Apr 2007
    Posts
    284
    Thanks.
    Put aside the stack and printf. let's focus on
    Code:
    char* x = (char*)malloc(3*1024*1024*1024);
    How come I can allocate a 3G space for a process?!

    Quote Originally Posted by citizen View Post
    >> why?

    printf( "%s", x ); prints x as a C string. A C string must be zero terminated.

    >> This time compiler complains for its size. So I guess the implementations of malloc and char[]
    >> are different, right?

    Stack memory is much smaller than the heap.

  4. #4
    Registered User whiteflags's Avatar
    Join Date
    Apr 2006
    Location
    United States
    Posts
    7,759
    Because malloc returns memory from a much larger pool maintained by the operating system. As you are aware though, a ceiling (though a generous one) for your process is imposed.

  5. #5
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    I'm still surprised you can do it. Is this a 64-bit sytem?
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  6. #6
    Deathray Engineer MacGyver's Avatar
    Join Date
    Mar 2007
    Posts
    3,211
    I believe someone wrote a simple program on this forum awhile back to find out how much memory one could get out of the *alloc() functions. I think it was about 512 MB for me on a 32-bit install of Windows XP with MinGW.

    There are obviously other ways of getting more memory, but that was just what occurred at the time. If you really wanted to use massive amounts of memory, I don't think the *alloc() family of functions are the proper way to go about it... at least not on Windows.

  7. #7
    Registered User
    Join Date
    Oct 2001
    Posts
    2,129
    Why wouldn't you be able to malloc 3G? It would just allocate some swap if you didn't have the memory, right?

  8. #8
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by robwhit View Post
    Why wouldn't you be able to malloc 3G? It would just allocate some swap if you didn't have the memory, right?
    Yes, but there is a limit within the OS that is arbitrarily set to be able to tell the difference between user-mode memory and kernel memory. On Linux, it's either at 1GB or 3GB, on Windows it's either 2GB or 3GB (both applies to 32-bit OS's).

    This limit is HARD - you can not allocate past it, no matter how much RAM you have, or any other parameters (swap-space). From that hard limit, you can only go down by other restricting factors such as amount of actual RAM, swap space, space already used in process space (for example, Linux "uses" the first 128MB as "no touch area" to capture use of NULL-pointers, and both OS's put a stack area of a few megabytes somewhere, and of course, you need at least a few kilobytes of code for your application and probably some system library code to interface to the OS, etc).

    I'm not aware of any "common" 32-bit OS that gives you more than 3GB of memory-space for an application.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  9. #9
    Registered User
    Join Date
    Oct 2001
    Posts
    2,934
    >So I tried the following
    >AND It works!
    Probably not, since you never checked for success.

  10. #10
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,001
    Not to mention even if 3 GB was the limit, there's probably a ton of other stuff in your virtual memory, as well, taking up space. At least your main stack would take up 1 MB place, which would leave you with 2.99 GB of memory left.
    And another big factor is memory fragmentation. There's just unlikely to be 3 GB of contiguous space free. There's probably some bytes here and there that are allocated for other things, which would instantly make the request fail.
    So even if the OS did allow you to allocate that much, it's unlikely it would succeed.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #11
    Registered User
    Join Date
    Oct 2001
    Posts
    2,129
    Thanks Mats.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. malloc doesn't do anything?
    By EUJeffrey in forum C Programming
    Replies: 7
    Last Post: 03-31-2008, 01:19 PM
  2. Replies: 4
    Last Post: 01-13-2008, 02:14 AM
  3. Multiplying Two Polynomials
    By CaptainMorgan in forum C++ Programming
    Replies: 6
    Last Post: 10-30-2006, 02:34 PM
  4. malloc always setting length of 8?
    By Zarkhalar in forum C++ Programming
    Replies: 7
    Last Post: 08-02-2004, 12:36 AM
  5. help with malloc
    By geetee in forum C Programming
    Replies: 1
    Last Post: 10-23-2001, 01:07 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21