Thread: Why address space 0-0x08000000 in a process is unused?

  1. #1
    Registered User
    Join Date
    Apr 2007
    Posts
    284

    Why address space 0-0x08000000 in a process is unused?

    Why address space 0-0x08000000 in a process is unused?

    This article (http://lwn.net/Articles/91829/) said:

    The very bottom part of the address space is unused; it is there to catch NULL pointers and such.
    But I don't understand what is catching NULL pointers? Why such a large block (128M) is used to catch NULL pointers?

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Because someone decided to do that. It's a pretty arbitrary decision, balancing between catching NULL pointers and making most of the memory available.

    Here's an example of when this comes in handy [albeit a bit contrived].
    Code:
    struct S
    {
       int x;
       int y;
       int size;
       int arr[250000];   // Approx 1M
    };
    
    #define COUNT 128
    
    S *p;
    
    int main() 
    {
        p = malloc(sizeof S * COUNT);
        for(i = COUNT-1; i >= 0; i--)
        {
           p[i].x = 0;
           p[i].y = 0;
           p[i].size = 0;
        }
        return 0;
    }
    Now imagine that malloc fails - since we don't check for NULL, and we start from "the back" of the large block of memory, we would be overwriting whatever is at just under 128M in the memory.

    Linux has a 3GB user-space, so using 128MB (0.125GB leave 2.875GB for user-space use).

    If you want more than that, it's very easy to upgrade most modern machines to use 64-bit Linux, and there the limit for user-space is some ridiculously large amount, very much more than the amount of memory you can actually fit in one of those machines. [Sorry, I can't remember exactly how much it is, but it would be in the order of 2^47 bytes - which is 32768 * 4GB].

    Note also that this area is UNUSED, it does not occupy space in your machine - it is a hole in the virtual to physical mapping, so whenever a read or write to/from that area is made, it causes a page-fault. It does NOT take up physical memory at all.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    Registered User
    Join Date
    Jun 2005
    Posts
    6,815
    This is specific to a particular machine/architecture. It is not related to C/C++.

    The basic reasoning would be that a NULL pointer has a zero value, and a common misuse of pointers is to write a large buffer to that address. A zero address would be correspond to the "bottom" of the address space, and allowing 128M of buffer space allows the program to survive should 128M or less of data be written to a NULL pointer.

    Personally, I consider such practices are only of some limited value when debugging programs (limited because there are plenty of other common forms of pointer molestation for which such buffering techniques are ineffective). Relying on such a memory buffer to compensate for programmer error would be extremely poor form in a production program as well.

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by grumpy View Post
    This is specific to a particular machine/architecture. It is not related to C/C++.

    The basic reasoning would be that a NULL pointer has a zero value, and a common misuse of pointers is to write a large buffer to that address. A zero address would be correspond to the "bottom" of the address space, and allowing 128M of buffer space allows the program to survive should 128M or less of data be written to a NULL pointer.

    Personally, I consider such practices are only of some limited value when debugging programs (limited because there are plenty of other common forms of pointer molestation for which such buffering techniques are ineffective). Relying on such a memory buffer to compensate for programmer error would be extremely poor form in a production program as well.
    I agree that it doesn't solve many of the potential pointer problems - however, NULL pointers from out of memory is probably one of the more common problems in a system when the system is running low on memory - and whilst it actually fixes nothing to catch the write to low memory, it's still better than overwriting some other data and then crashing LATER ON.

    And the other pointer molestations are much harder for the OS to do anything about.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  5. #5
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Another thing is that the OS typically reserves some of the Virtual Memory Space for its private use (kernel). An application will never be able to use the entire 4 GB memory space in the virtual memory on a 32-bit machine.
    Or at least I think this holds true for most operating systems...
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  6. #6
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Elysia View Post
    Another thing is that the OS typically reserves some of the Virtual Memory Space for its private use (kernel). An application will never be able to use the entire 4 GB memory space in the virtual memory on a 32-bit machine.
    Or at least I think this holds true for most operating systems...
    Yes, it's normally reserving the upper range for OS kernel space. The reason for this is that we want the code and data used by for example interrupt services and OS system calls to be available at all times. By mapping this into a separate chunk of memory, it can be shared between all processes in the system as one memory mapping [which is only accessible when in kernel mode, so it's not allowing the user-mode application access to the code or data other than through the appropriate system calls].

    Linux by default reserves 1GB kernel space and 3GB of user-space [in recent versions of Linux, previously it was 1GB each, I seem to remember], Windows defaults to 2GB user and 2GB kernel space.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  7. #7
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by meili100 View Post
    Why address space 0-0x08000000 in a process is unused?
    It's only unused until you use it. The linker decided to place the program at that address. If you wanted it somewhere else, you could have told the linker to put it somewhere else -- within the limits of what the OS will allow.

    Dynamic libraries might be relocated to those addresses, you could use them to map files, or shared memory regions, or anything else you want. You can use addresses above this region as well, although that's where the heap lives, and you probably shouldn't mess with that.

    The OS will completely reserve certain pages so that you can never allocate them. The page containing address 0x0 is always reserved, so that NULL pointer references will cause a violation. The region close to, and below, 0xC0000000 is the stack, and the OS treats that slightly differently as well. Addresses above 0xC0000000 belong to the kernel, and are inaccessible from user space.

    If you read the article carefully, you see that it doesn't say that the whole area from 0x0 to 0x08000000 is reserved for the NULL pointer -- only that the program code and data begins there.

  8. #8
    Reverse Engineer maxorator's Avatar
    Join Date
    Aug 2005
    Location
    Estonia
    Posts
    2,318
    Quote Originally Posted by brewbuck View Post
    The OS will completely reserve certain pages so that you can never allocate them. The page containing address 0x0 is always reserved, so that NULL pointer references will cause a violation. The region close to, and below, 0xC0000000 is the stack, and the OS treats that slightly differently as well. Addresses above 0xC0000000 belong to the kernel, and are inaccessible from user space.
    Umm, when we're talking about Windows, then it's not true.

    In Windows the 00000000h-7FFFFFFFh is the usermode virtual memory of the current process. 80000000h-0FFFFFFFFh is the kernel memory.

    The program itself is usually situated at 00400000h. The stack of the main thread is usually at 0013E000h. Stack is a piece of normal memory except for the cause that it is used for. Basic Windows DLLs are usually loaded from 73000000h-7FFFFFFFh.

    Other DLLs are often relocated to different places.

    Of course the /3G switch changes the location of kernel memory...
    "The Internet treats censorship as damage and routes around it." - John Gilmore

  9. #9
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,396
    Quote Originally Posted by maxorator View Post
    Umm, when we're talking about Windows, then it's not true.
    It's not exactly that different, either. Also, on Windows, thread stacks are finite in size, so yes, the stack is treated like any other area of memory. With a growable stack, the OS will automatically allocate more pages for the stack as it grows down.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Process sending file descriptors to another process
    By Yasir_Malik in forum C Programming
    Replies: 4
    Last Post: 04-07-2005, 07:36 PM
  2. Contest Results - May 27, 2002
    By ygfperson in forum A Brief History of Cprogramming.com
    Replies: 18
    Last Post: 06-18-2002, 01:27 PM
  3. Relocatable code and Symbol Table
    By Unregistered in forum C Programming
    Replies: 2
    Last Post: 06-10-2002, 11:05 AM
  4. Im so lost at . .
    By hermit in forum C Programming
    Replies: 18
    Last Post: 05-15-2002, 01:26 AM
  5. doubly linked lists
    By cworld in forum C++ Programming
    Replies: 2
    Last Post: 04-21-2002, 09:33 AM