the reason the memory never got over 8-9 megabytes is because the system never attempted to actually allocate physical memory pages for the virtual memory space you were allocating. modern OS kernels only allocate physical pages upon a write to a given region of memory. otherwise, it's just entries in a page table somewhere, and the present bit is not set, meaning the space is not actually allocated in physical memory.
consider the following code:
this will allocate (and write to) memory until it cannot anymore. either the system will crash, or the kernel will terminate your program because it cannot allocate any more memory.
int pageSize = getpagesize();
long totalMemory = 0;
char* foo = new char[pageSize];
totalMemory += pageSize;
foo = 0;
std::cout << "Allocated " << totalMemory << " bytes (" << totalMemory / 1024 << "KB) of memory." << std::endl;