Thread: Available memory size

  1. #1
    Registered User
    Join Date
    Oct 2008
    Posts
    77

    Available memory size

    Hi,
    I'm building an application which needs to dynamicall create memory entities - each of 10MB in size. In the heap, not using virtual memory.
    I'd like to evaluate, before the memory objects creation starts, what would be the max (just evaluation) number that can be created, thus I need to get the process' available heap at the instant.
    How can I get such info, if possible, for Windows environments (XP, Vista, Server 2008 ...). I have some hints but I'd like to have your expert opinion.
    Many thanks!

  2. #2
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,195
    System Information Functions (Windows)

    AFAIK theres no way to get the amount of physical ram that is 'free' vs the amount that will be virtual, because any number returned would be immediately stale, the OS pages memory on an as needed basis, so there is no way of determining how much s allocated to your application right at that moment short of a kernel hack.

    Then again I could be wrong, the API is pretty big.

    http://msdn.microsoft.com/en-us/libr...89(VS.85).aspx

    Code:
    //  Sample output:
    //  There is       51 percent of memory in use.
    //  There are 2029968 total Kbytes of physical memory.
    //  There are  987388 free Kbytes of physical memory.
    //  There are 3884620 total Kbytes of paging file.
    //  There are 2799776 free Kbytes of paging file.
    //  There are 2097024 total Kbytes of virtual memory.
    //  There are 2084876 free Kbytes of virtual memory.
    //  There are       0 free Kbytes of extended memory.
    
    #include <windows.h>
    #include <stdio.h>
    
    // Use to convert bytes to KB
    #define DIV 1024
    
    // Specify the width of the field in which to print the numbers. 
    // The asterisk in the format specifier "%*I64d" takes an integer 
    // argument and uses it to pad and right justify the number.
    #define WIDTH 7
    
    void main(int argc, char *argv[])
    {
      MEMORYSTATUSEX statex;
    
      statex.dwLength = sizeof (statex);
    
      GlobalMemoryStatusEx (&statex);
    
      printf ("There is  %*ld percent of memory in use.\n",
              WIDTH, statex.dwMemoryLoad);
      printf ("There are %*I64d total Kbytes of physical memory.\n",
              WIDTH, statex.ullTotalPhys/DIV);
      printf ("There are %*I64d free Kbytes of physical memory.\n",
              WIDTH, statex.ullAvailPhys/DIV);
      printf ("There are %*I64d total Kbytes of paging file.\n",
              WIDTH, statex.ullTotalPageFile/DIV);
      printf ("There are %*I64d free Kbytes of paging file.\n",
              WIDTH, statex.ullAvailPageFile/DIV);
      printf ("There are %*I64d total Kbytes of virtual memory.\n",
              WIDTH, statex.ullTotalVirtual/DIV);
      printf ("There are %*I64d free Kbytes of virtual memory.\n",
              WIDTH, statex.ullAvailVirtual/DIV);
    
      // Show the amount of extended memory available.
    
      printf ("There are %*I64d free Kbytes of extended memory.\n",
              WIDTH, statex.ullAvailExtendedVirtual/DIV);
    }
    Last edited by abachler; 10-01-2009 at 06:40 AM.

  3. #3
    Registered User
    Join Date
    Oct 2008
    Posts
    77
    Yeah, that's system one.
    What I need is info about the process I'm calling from.
    I found a "way around" i.e. looping with malloc increasing the size and exiting when getting out of memory (NULL), but I wonder there's no process info structure available. The system should know it so that to manage the process exec space. So where is it ;-) ?

  4. #4
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,195
    the OS uses the global descriptor table and the local descriptor tables to manage physical and virtual memory. Applications do not have access to those structures as they are only available in ring 0. The OS doesn't know how much of your specific program is in memory or not as such, it just knows globally which memory locations are in physical memory and which are on disk, It would have to specifically do a page walk to collect that information, which would be of no use to it, so its not something the kernel would do, but it is something only the kernel can do.

    There is no reason you would need to know this information. The kernel handles paging transparently. Whatever it reports as free system wide physical memory is what is available to your process as physical memory. Each process has access to all available physical memory up to its per process limit, which is 2GB normally or 3GB if you use the /3gb switch. However, just because it reports, e.g. 2GB free physical memory, doesn't mean if you allocate a 2GB object that the entire object will be in physical ram, the OS will handle what is and is not paged to optimize system wide performance. I believe there is a way of marking heap memory as being non-paged though, check out the heap functions.

  5. #5
    Registered User
    Join Date
    Oct 2008
    Posts
    77
    Ok fine now, your thorough comments confirms what I found on msdn. FYI, I just want to anticipate the apps limits before the crash test, so that's the reason.
    Thanks!

  6. #6
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Quote Originally Posted by abachler View Post
    the OS uses the global descriptor table and the local descriptor tables to manage physical and virtual memory. Applications do not have access to those structures as they are only available in ring 0. The OS doesn't know how much of your specific program is in memory or not as such, it just knows globally which memory locations are in physical memory and which are on disk, It would have to specifically do a page walk to collect that information, which would be of no use to it, so its not something the kernel would do, but it is something only the kernel can do.

    There is no reason you would need to know this information. The kernel handles paging transparently. Whatever it reports as free system wide physical memory is what is available to your process as physical memory. Each process has access to all available physical memory up to its per process limit, which is 2GB normally or 3GB if you use the /3gb switch. However, just because it reports, e.g. 2GB free physical memory, doesn't mean if you allocate a 2GB object that the entire object will be in physical ram, the OS will handle what is and is not paged to optimize system wide performance. I believe there is a way of marking heap memory as being non-paged though, check out the heap functions.
    2 GB for an x86 OS, but that is not required for an x64 OS.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  7. #7
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,195
    Quote Originally Posted by Elysia View Post
    2 GB for an x86 OS, but that is not required for an x64 OS.
    No theres a switch you can add to system.ini that lets you use 3GB per process on an x86.

  8. #8
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    That was beside the point. The point was that a process can use a maximum of 2/3 GB in an x86 OS, but in an x64 OS, such a limit does not exist. It may not be unlimited, but it is way more than 2/3 GB.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  9. #9
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,195
    Quote Originally Posted by Elysia View Post
    That was beside the point. The point was that a process can use a maximum of 2/3 GB in an x86 OS, but in an x64 OS, such a limit does not exist. It may not be unlimited, but it is way more than 2/3 GB.
    its 64GB, thanks to Microsoft.

  10. #10
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    64 GB is plenty for today. No doubt it will be increased in the future.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  11. #11
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,229
    I thought most supercomputers pack A LOT more than 64GB RAM already. But then they don't run Windows anyways.

    Even some consumer enthusiast systems already have 16GB RAM (4x4GB, 4GB sticks are widely available now).

    64GB will run out in no time (3 years if we assume it doubles every 1.5 year).

    But of course, Windows 8 will support 128GB RAM... so if you need that, you'll have to upgrade. Then Windows 9 will support 256GB RAM .
    Last edited by cyberfish; 10-05-2009 at 02:26 PM.

  12. #12
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,229
    As for the original question, I don't think malloc till failure will work. It will always give you close to the amount of virtual memory you have.

  13. #13
    Registered User
    Join Date
    Sep 2004
    Location
    California
    Posts
    3,268
    Quote Originally Posted by cyberfish View Post
    As for the original question, I don't think malloc till failure will work. It will always give you close to the amount of virtual memory you have.
    While that's true on Linux, I don't think it is true on Windows. I believe that Windows will always check to ensure there is heap space available before returning a valid pointer from malloc().
    bit∙hub [bit-huhb] n. A source and destination for information.

  14. #14
    Registered User
    Join Date
    Dec 2006
    Location
    Canada
    Posts
    3,229
    Ah ok. Stand corrected.

    I have never done much Windows programming.

  15. #15
    'Allo, 'Allo, Allo
    Join Date
    Apr 2008
    Posts
    639
    Memory limits in Windows releases

    Summary:

    If program isn't linked with LargeAddressAware flag, no more than 2GB address space regardless of program bittage or machine configuration.

    32-bit program = max of 4 GB of user address space on 64-bit machines, max of 3GB with 4GT on 32-bit.

    32-bit machine = up to 4 GB physical memory in workstation editions, no more than 64GB for server versions

    64-bit program = max of 7TB (IPF) / 8TB (x64) user address space.

    64-bit machine = up to 192 GB physical memory in workstation editions, no more than 2TB for server versions.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. tools for finding memory leaks
    By stanlvw in forum C++ Programming
    Replies: 4
    Last Post: 04-03-2009, 11:41 AM
  2. memory leaks
    By TehOne in forum C Programming
    Replies: 4
    Last Post: 10-10-2008, 09:33 PM
  3. Find Size Of Dynamic Memory Allocation?
    By appleGuy in forum C++ Programming
    Replies: 7
    Last Post: 06-17-2007, 09:34 AM
  4. Pointer's
    By xlordt in forum C Programming
    Replies: 13
    Last Post: 10-14-2003, 02:15 PM
  5. How do I check memory size after malloc and realloc?
    By electrolove in forum C Programming
    Replies: 3
    Last Post: 02-06-2003, 10:12 AM