Thread: Where exactly are things located in memory ?

Where can I find reliable information about memory management in terms of programming. Especially I would like to know in which memory location are things like: global variables stored, where are functions stored, pointers and so on. I know that function local variables, interrupts and temporary arithmetic are all stored on stack. Dynamically allocated memory lies on heap. But what about the rest ?

Thanks!

It depends on the operating system, and underlying architecture.

In Linux, for example, you can examine the /proc/self/maps pseudofile, which is actually a kernel view of the memory mappings for the current process. There's documentation here. You can also use dl_iterate_phdr() if you're only interested about shared libraries. For individual dynamic symbols (functions, global variables), there is the dlsym() function (provided by the dl library that interfaces to the dynamic linker). For all symbols, you can use the library names (or /proc/self/exe pseudo-symlink for the current executable) and examine them (even in run time) via binutils utilities, especially objdump and readelf.

(Unlike some other operating systems, in Linux and POSIXy systems in general, you can open executables (read-only) even when they are being executed. You can also rename or delete an executable file even if it is being executed, as the underlying inode (file data and metadata) is only released after the final process accessing it exits. Replacing an executable with another file thus only affects those processes that execute it after the replacement. Other systems behave differently.)

Here's what /proc/self/maps looks for the bash shell on my system:

Code:

00400000-004ef000 r-xp 00000000 08:05 1181680                            /bin/bash
006ef000-006f0000 r--p 000ef000 08:05 1181680                            /bin/bash
006f0000-006f9000 rw-p 000f0000 08:05 1181680                            /bin/bash
006f9000-006ff000 rw-p 00000000 00:00 0 
02414000-026e3000 rw-p 00000000 00:00 0                                  [heap]
7f3751237000-7f3751242000 r-xp 00000000 08:05 1437385                    /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f3751242000-7f3751441000 ---p 0000b000 08:05 1437385                    /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f3751441000-7f3751442000 r--p 0000a000 08:05 1437385                    /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f3751442000-7f3751443000 rw-p 0000b000 08:05 1437385                    /lib/x86_64-linux-gnu/libnss_files-2.19.so
7f3751443000-7f375144e000 r-xp 00000000 08:05 1437413                    /lib/x86_64-linux-gnu/libnss_nis-2.19.so
7f375144e000-7f375164d000 ---p 0000b000 08:05 1437413                    /lib/x86_64-linux-gnu/libnss_nis-2.19.so
7f375164d000-7f375164e000 r--p 0000a000 08:05 1437413                    /lib/x86_64-linux-gnu/libnss_nis-2.19.so
7f375164e000-7f375164f000 rw-p 0000b000 08:05 1437413                    /lib/x86_64-linux-gnu/libnss_nis-2.19.so
7f375164f000-7f3751666000 r-xp 00000000 08:05 1437397                    /lib/x86_64-linux-gnu/libnsl-2.19.so
7f3751666000-7f3751865000 ---p 00017000 08:05 1437397                    /lib/x86_64-linux-gnu/libnsl-2.19.so
7f3751865000-7f3751866000 r--p 00016000 08:05 1437397                    /lib/x86_64-linux-gnu/libnsl-2.19.so
7f3751866000-7f3751867000 rw-p 00017000 08:05 1437397                    /lib/x86_64-linux-gnu/libnsl-2.19.so
7f3751867000-7f3751869000 rw-p 00000000 00:00 0 
7f3751869000-7f3751872000 r-xp 00000000 08:05 1437396                    /lib/x86_64-linux-gnu/libnss_compat-2.19.so
7f3751872000-7f3751a71000 ---p 00009000 08:05 1437396                    /lib/x86_64-linux-gnu/libnss_compat-2.19.so
7f3751a71000-7f3751a72000 r--p 00008000 08:05 1437396                    /lib/x86_64-linux-gnu/libnss_compat-2.19.so
7f3751a72000-7f3751a73000 rw-p 00009000 08:05 1437396                    /lib/x86_64-linux-gnu/libnss_compat-2.19.so
7f3751a73000-7f3752079000 r--p 00000000 08:05 1044632                    /usr/lib/locale/locale-archive
7f3752079000-7f3752234000 r-xp 00000000 08:05 1437402                    /lib/x86_64-linux-gnu/libc-2.19.so
7f3752234000-7f3752433000 ---p 001bb000 08:05 1437402                    /lib/x86_64-linux-gnu/libc-2.19.so
7f3752433000-7f3752437000 r--p 001ba000 08:05 1437402                    /lib/x86_64-linux-gnu/libc-2.19.so
7f3752437000-7f3752439000 rw-p 001be000 08:05 1437402                    /lib/x86_64-linux-gnu/libc-2.19.so
7f3752439000-7f375243e000 rw-p 00000000 00:00 0 
7f375243e000-7f3752441000 r-xp 00000000 08:05 1437389                    /lib/x86_64-linux-gnu/libdl-2.19.so
7f3752441000-7f3752640000 ---p 00003000 08:05 1437389                    /lib/x86_64-linux-gnu/libdl-2.19.so
7f3752640000-7f3752641000 r--p 00002000 08:05 1437389                    /lib/x86_64-linux-gnu/libdl-2.19.so
7f3752641000-7f3752642000 rw-p 00003000 08:05 1437389                    /lib/x86_64-linux-gnu/libdl-2.19.so
7f3752642000-7f3752667000 r-xp 00000000 08:05 1436281                    /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f3752667000-7f3752866000 ---p 00025000 08:05 1436281                    /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f3752866000-7f375286a000 r--p 00024000 08:05 1436281                    /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f375286a000-7f375286b000 rw-p 00028000 08:05 1436281                    /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f375286b000-7f375288e000 r-xp 00000000 08:05 1437399                    /lib/x86_64-linux-gnu/ld-2.19.so
7f3752a62000-7f3752a65000 rw-p 00000000 00:00 0 
7f3752a7c000-7f3752a84000 r--p 00000000 08:05 1312704                    /usr/share/locale-langpack/en_GB/LC_MESSAGES/bash.mo
7f3752a84000-7f3752a8b000 r--s 00000000 08:05 1062844                    /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7f3752a8b000-7f3752a8d000 rw-p 00000000 00:00 0 
7f3752a8d000-7f3752a8e000 r--p 00022000 08:05 1437399                    /lib/x86_64-linux-gnu/ld-2.19.so
7f3752a8e000-7f3752a8f000 rw-p 00023000 08:05 1437399                    /lib/x86_64-linux-gnu/ld-2.19.so
7f3752a8f000-7f3752a90000 rw-p 00000000 00:00 0 
7ffe2fa79000-7ffe2fa9a000 rw-p 00000000 00:00 0                          [stack]
7ffe2fabf000-7ffe2fac1000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

Because it is a kernel-generated pseudofile (generated by the kernel directly, whenever accessed), it is never localized, and the format is stable; it will not change in incompatible ways. (The Linux kernel developers are strict about that, and that's why there are a number of pseudofiles in the /proc/PID/ directory with overlapping contents.)

The entries with no filename/description attached are memory the process allocated via mmap(). [heap] refers to initial heap only, the one that can be grown via sbrk(). (The GNU C library, for example, uses the latter for small allocations, but mmap() for larger allocations. Other POSIXy systems have different heuristics, but in general behave roughly similarly.)

To track individual dynamic allocations, the malloc() et al. memory allocation functions can be interposed (by a dynamic library, using LD_PRELOAD_LIB environment variable to load an extra dynamic library to the process, which replaces/interposes the original functions), and tracked that way.

Originally Posted by Salem

GeeksforGeeks

As you can see from my post above, it is not really an up to date view anymore, especially on 64-bit architectures. The address space is so large, treating it as a single contiguous block does not reflect the current reality anymore.

On my system, heap tends to get really low addresses, stack high addresses, with dynamic libraries and memory maps somewhere in between. There is no single text segment; all the r--xp segments contain executable code. (The ---p are guard pages, inaccessible but remappable, and r--p are read-only, non-executable data.)

Trying to access any addresses not within the ranges listed will kill the process with a SIGSEGV signal. If the region is mapped, but inaccessible (unreadable if trying to read, or read-only if trying to write to it), a SIGBUS signal is generated; it can be caught and handled by the process. This is very useful if you want to create e.g. copy-on-write mappings et cetera.

Thanks for answers. But what about memory allocation on different systems, let's say I allocated 4 bytes of memory on Linux 64bit and 8bit microcontroller like below:

Code:

int i = malloc(4);

We know that these 4 bytes will be allocated from memory segment called heap, in both 64 bit linux and 8 bit uC (i.e Atmega16a), right ? Now, what really happens when we compile such instruction ? How the system is requested (what instructions, etc) to allocate 4 bytes ? Then, how those systems (linux, 8-bit uC) actually allocate that memory ?

Where and how the memory is allocated is implementation defined, especially when you're talking about non-hosted systems. There are systems that don't have a stack, or heap. Many 8-bit uC have several different segments where the memory could be allocated.

You really shouldn't worry about where malloc() is getting the memory. And if you need to worry about the where, you really shouldn't be using malloc() in the first place.

Jim

Agreed. The details are interesting, but vary.

The tools I named are those that one can use in Linux, but it only gets truly interesting if you compile and run the same self-examining program on multiple architectures, say on 32-bit ARM (Raspberry Pi), and 64-bit AMD64/x86-64. It not only shows how things differ, but gives you an idea how much more flexible these new systems are compared to old ones.

On microcontrollers, the memory allocation policy tends to depend on how much memory is available. If just a few thousand bytes, it is common to have the stack and heap use whatever RAM is available after statically allocated variables and necessary library internal structures have been catered for; with stack growing from one end, and heap allocations from the other, and if they cross, you're hosed. But even this is just the typical case, not a rule you can assume to hold.

I think I should have emphasized the importance on not relying on such implementation details.

The information is interesting, and probably can help you build a better mental picture of the memory structures involving processes, and it's useful for interposing libraries and debugging, but there is no underlying rules there that would apply universally; it all varies from system to system. There is no universal memory layout for C programs.