Thread: the confusing term page file

  1. #1
    Registered User
    Join Date
    May 2006
    Posts
    1,579

    the confusing term page file

    Hello everyone,


    On Windows Server 2003, task manager there is a term called PF (Page File) and it has a related curve. Task manager also has another term called commit charge.

    1. I think PF is the same as commit charge. Previously, I think Page File only includes the paged-out pages which physical memory can not contain, and the pages are not in memory are counted as PF. Now I think I am wrong and PF includes all committed pages, including the ones in memory and the ones on disk;

    2. Even if the pages are marked as COMMIT, it is not always counted as a part of PF in some situations before it is actually used (request for zero and copy-on-write scenarios);

    Are my understandings of (1) and (2) correct?


    thanks in advance,
    George

  2. #2
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    You asked this earlier, remember? The page file contains the pages not present in physical memory at this time (plus some other stuff like memory mapped files).
    It's just that the task manager is confusing by incorrectly specifying the page file size.

    And non-used (but reserved pages) are not inside the page file. What would be the use? Reserved pages are reserved in name only. The physical memory itself is never reserved and is committed upon use when a reserved page is made into a committed page.

    The committed size specifies how much physical memory is used at present.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  3. #3
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Elysia View Post
    You asked this earlier, remember? The page file contains the pages not present in physical memory at this time (plus some other stuff like memory mapped files).
    It's just that the task manager is confusing by incorrectly specifying the page file size.

    And non-used (but reserved pages) are not inside the page file. What would be the use? Reserved pages are reserved in name only. The physical memory itself is never reserved and is committed upon use when a reserved page is made into a committed page.
    Reserved pages take up space only in the virtual space in the current process [e.g. you ask to allocate 100MB of memory, but only "touch" the first 1MB, you will have 99MB reserved in the virtual space - this memory can not be used for any other purpose, but it's also not taking up any space in the page-file or in physical memory]

    The committed size specifies how much physical memory is used at present.
    I think it is how much virtual memory is in use.
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  4. #4
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks matsp,


    I agree with all of your analysis. Great!

    Just two more comments,

    1.

    Quote Originally Posted by matsp View Post
    I think it is how much virtual memory is in use.
    Suppose the case of copy on write and request for zero, I think even if the memory is marked as comitted during API call to VirtualAlloc, since the memory is not actually used, such type of memory is not counted as a part of committed charge. right?

    2.

    PF is the same as committed charge in task manager. Agree?

    3.

    PF contains totally currently used memory, including the consumed system swap page file and the consumed real physical memory. Right?

    4.

    Quote Originally Posted by Elysia View Post
    You asked this earlier, remember?
    1) You can see from matsp's analysis, there is still something unclear.
    2) With the growth of our experience, we will have different and deeper view of the same question. I am sure you have this experience before.


    regards,
    George

  5. #5
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by George2 View Post
    Thanks matsp,


    I agree with all of your analysis. Great!

    Just two more comments,

    1.



    Suppose the case of copy on write and request for zero, I think even if the memory is marked as comitted during API call to VirtualAlloc, since the memory is not actually used, such type of memory is not counted as a part of committed charge. right?
    Yes, only pages that are ACTUALLY being used are counted, not uncommitted pages, such as those that need to be filled with zero or copied if written.

    2.

    PF is the same as committed charge in task manager. Agree?
    I think, when it comes to Task Manager, this concept holds.

    3.

    PF contains totally currently used memory, including the consumed system swap page file and the consumed real physical memory. Right?
    Most likely, yes.

    4.



    1) You can see from matsp's analysis, there is still something unclear.
    2) With the growth of our experience, we will have different and deeper view of the same question. I am sure you have this experience before.


    regards,
    George
    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  6. #6
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks Mats,


    It is great that you agree with my point 1, 2 and 3. Let us come back to the original question, which I do not agree.

    Let me quote the word from the Windows Internals book,

    --------------------
    Thus, the Process: Page File Bytes performance counter is actually the total process private committed memory, of which none, some or all may be in the paging file (In fact, it's the same as the Process: Private Bytes performance counter).
    --------------------

    You can open perfmon, and monitor Page File Bytes and Private Bytes for _Total, you can obviously see they are different. :-)

    Any comments?

    Quote Originally Posted by matsp View Post
    Yes, only pages that are ACTUALLY being used are counted, not uncommitted pages, such as those that need to be filled with zero or copied if written.

    I think, when it comes to Task Manager, this concept holds.

    Most likely, yes.


    --
    Mats

    regards,
    George

  7. #7
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    They aren't identical, meaning that the measurement is probably not done at exactly the same point [or on exactly the same data], but they are pretty darn close - if you are worried about the last tens of kilobytes in a machine using 100+ MB, then I think you have too much time on your hands.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  8. #8
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks Mats,


    1. so you agree they are not the same thing?

    2. On my machine, they are quiet different, paging file total 400, private bytes total 3600.

    Quote Originally Posted by matsp View Post
    They aren't identical, meaning that the measurement is probably not done at exactly the same point [or on exactly the same data], but they are pretty darn close - if you are worried about the last tens of kilobytes in a machine using 100+ MB, then I think you have too much time on your hands.

    --
    Mats

    regards,
    George

  9. #9
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    What does your application do? Is it, perhaps, using Mapped Views of Files? I expect that to throw the bytes off by a fair bit, since mapped view isn't part of "private data" in the same sense.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  10. #10
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Yes, Mats!


    1.

    I am programming with memory map files. But in this case, I do not run any addtional applications on my Windows Server 2003, and simply watch what system performance counter look like during idle.

    I think your point is private bytes does not include the shared memory map file, so the figures are different, right?

    2.

    But the confusion is, Page File Bytes counter from perfmon and PF from task manager are not the same thing -- too confusing.

    I quote some information here and let me know whether you agree or not?

    from perfmon help

    --------------------
    Page File Bytes is the current amount of virtual memory, in bytes, that this process has reserved for use in the paging file(s). Paging files are used to store pages of memory used by the process that are not contained in other files. Paging files are shared by all processes, and the lack of space in paging files can prevent other processes from allocating memory. If there is no paging file, this counter reflects the current amount of virtual memory that the process has reserved for use in physical memory.
    --------------------

    Windows Internals Version 4,

    (Page 446, Table 7-15)

    --------------------
    Number of bytes of virtual (not reserved) memory that has been committed. This number doesn't necessarily represent page file usage because it includes private committed bytes in physical memory that have never been paged out. Rather, it represents the amount of page file space that would be used if the process was completely made nonresident.
    --------------------

    (Page 447)

    --------------------
    Note the Mem Usage Bar (called PF Usage in Windows XP and Windows Server 2003) is actually the system commit total. This number represents potential page file usage, not actual page file usage.
    --------------------


    Quote Originally Posted by matsp View Post
    What does your application do? Is it, perhaps, using Mapped Views of Files? I expect that to throw the bytes off by a fair bit, since mapped view isn't part of "private data" in the same sense.

    --
    Mats

    regards,
    George

  11. #11
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Well, on my XP system, they are essentially equal - a few tenths of percent difference, but not a factor of ten out.

    I don't know what is happening in Server 2003 to make it different - the kernel is essentially the same with W2K3 and XP.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  12. #12
    Registered User
    Join Date
    May 2006
    Posts
    1,579
    Thanks Mats,


    How about my comment (2) in post #10, do you think Page File Counter and PF are different?

    Quote Originally Posted by matsp View Post
    Well, on my XP system, they are essentially equal - a few tenths of percent difference, but not a factor of ten out.

    I don't know what is happening in Server 2003 to make it different - the kernel is essentially the same with W2K3 and XP.

    --
    Mats

    regards,
    George

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. opening empty file causes access violation
    By trevordunstan in forum C Programming
    Replies: 10
    Last Post: 10-21-2008, 11:19 PM
  2. Formatting a text file...
    By dagorsul in forum C Programming
    Replies: 12
    Last Post: 05-02-2008, 03:53 AM
  3. help with text input
    By Alphawaves in forum C Programming
    Replies: 8
    Last Post: 04-08-2007, 04:54 PM
  4. System
    By drdroid in forum C++ Programming
    Replies: 3
    Last Post: 06-28-2002, 10:12 PM