Retrieving memory usage

[Magos]

I'd like to retrieve the memory usage of a program. I tried using HeapSize() but it seems to only give the size of a memory block inside the heap. What function am I looking for, if it exists? If possible, can you also retrieve the CPU usage?

[Tonto]

I believe relevant API functions include:

VirtualQueryEx(..)
GlobalMemoryStatusEx(..)

[Magos]

They're for retrieving the total memory usage, I'm looking for the memory usage by a single program.

[Magos]

I found some MSDN docs on MSI, but it doesn't give correct data. Like it says it uses 2.2 million % of the processor (!). Though I wish it could, I severly doubt that's the case .

Anyway, if someone has any experience with MSI and could spot the error I'd be glad!

Also, the query gives lots of objects. Are all those processes for the program? The total amount of processes running?

Initialization:

HRESULT Result;

if(FAILED(Result = CoInitializeEx(NULL, COINIT_MULTITHREADED)))
{
	Error.SetMessage("Unable to initialize COM!", Result);
	return FALSE;
}

COMInitialized = TRUE;

if(FAILED(Result = CoInitializeSecurity(NULL, -1, NULL, NULL, RPC_C_AUTHN_LEVEL_DEFAULT, RPC_C_IMP_LEVEL_IMPERSONATE, NULL, EOAC_NONE, NULL)))
{
	Error.SetMessage("Unable to initialize COM security!", Result);
	return FALSE;
}

if(FAILED(Result = CoCreateInstance(CLSID_WbemLocator, NULL, CLSCTX_INPROC_SERVER, IID_IWbemLocator, reinterpret_cast<LPVOID*>(&Locator))))
{
	Error.SetMessage("Unable to create a COM instance!", Result);
	return FALSE;
}

if(FAILED(Result = Locator->ConnectServer(bstr_t("ROOT\\CIMV2"), NULL, NULL, NULL, 0, NULL, NULL, &Services)))
{
	Error.SetMessage("Unable to connect to COM server!", Result);
	return FALSE;
}

if(FAILED(Result = CoSetProxyBlanket(Services, RPC_C_AUTHN_WINNT, RPC_C_AUTHZ_NONE, NULL, RPC_C_AUTHN_LEVEL_CALL, RPC_C_IMP_LEVEL_IMPERSONATE, NULL, EOAC_NONE)))
{
	Error.SetMessage("Unable to set the COM proxy blanket!", Result);
	return FALSE;
}

Cleaning up:

SAFE_RELEASE(Services);
SAFE_RELEASE(Locator);

if(COMInitialized)
{
	CoUninitialize();
	COMInitialized = FALSE;
}

Asking the query:

ULONG Result;
VARIANT Variant;
IWbemClassObject* Object;
IEnumWbemClassObject* Enumerator;

FreeMemory = 0;
UsedMemory = 0;
CpuUsage = 0;

if(FAILED(Services->ExecQuery(bstr_t("WQL"), bstr_t("SELECT * FROM Win32_PerfFormattedData_PerfProc_Process"),
  WBEM_FLAG_FORWARD_ONLY | WBEM_FLAG_RETURN_IMMEDIATELY, NULL, &Enumerator))) return;

VariantInit(&Variant);

while(Enumerator)
{
	if(FAILED(Enumerator->Next(WBEM_INFINITE, 1, &Object, &Result))) break;
	if(Result == 0) break;

	if(SUCCEEDED(Object->Get(L"VirtualBytes", 0, &Variant, NULL, NULL)))
	{
		UsedMemory = static_cast<INT>(Variant.ullVal);
	}

	if(SUCCEEDED(Object->Get(L"PercentProcessorTime", 0, &Variant, NULL, NULL)))
	{
		CpuUsage = static_cast<INT>(Variant.ullVal);
	}

	SAFE_RELEASE(Object);
}

VariantClear(&Variant);

SAFE_RELEASE(Object);
SAFE_RELEASE(Enumerator);

[anonytmouse]

I believe relevant API functions include:

VirtualQueryEx(..)
GlobalMemoryStatusEx(..)

They're for retrieving the total memory usage, I'm looking for the memory usage by a single program.

VirtualQuery can be used to walk the process address space and find the total committed virtual memory (by adding up all the committed blocks). I did a search on my disk and found this simple sample of walking the address space:

	MEMORY_BASIC_INFORMATION mbi;
	UINT  i       = 0;
	TCHAR szFile[MAX_PATH];
	CHAR  szNumber[20];

	while (i < 0x80000000)
	{
		VirtualQuery((LPCVOID) i, &mbi, sizeof(mbi));

		printf("RegionStart: %p\n", (void*) i);
		printf("RegionSize: %u\n", mbi.RegionSize);

		printf("State: %s\n", 
		       (mbi.State == MEM_FREE    ? "MEM_FREE"    :
		        mbi.State == MEM_COMMIT  ? "MEM_COMMIT"  :
		        mbi.State == MEM_RESERVE ? "MEM_RESERVE" :
		        itoa(mbi.State, szNumber, 16)));

		printf("Type: %s\n",
	               (mbi.Type == MEM_MAPPED  ? "MEM_MAPPED"  :
		        mbi.Type == MEM_PRIVATE ? "MEM_PRIVATE" :
		        mbi.Type == MEM_IMAGE   ? "MEM_IMAGE"   :
			mbi.Type == 0           ? "N/A (0)"     :
		        itoa(mbi.Type, szNumber, 16)));

		if (mbi.Type == MEM_MAPPED && 
	            GetMappedFileName(GetCurrentProcess(), (LPVOID) i, szFile, MAX_PATH))
		{
			printf("File Name: %s\n", szFile);
		}

		printf("\n");

		i += mbi.RegionSize;
	}

[anonytmouse]

I found some MSDN docs on MSI, but it doesn't give correct data. Like it says it uses 2.2 million % of the processor (!). Though I wish it could, I severly doubt that's the case .

Are you sure that the variant contains an ull? Could it be returning the value as a string? Have you checked Variant.vt?

Also, the query gives lots of objects. Are all those processes for the program? The total amount of processes running?

It returns one object for each process running on the system. You can restrict the objects it returns by using a WHERE clause. Here is a sample (which uses DispHelper):

#include "disphelper.h"
#include <stdio.h>

int main(void)
{
	DISPATCH_OBJ(wmiSvc);
	DISPATCH_OBJ(colPerf);
	WCHAR szQuery[500];

	dhInitialize(TRUE);
	dhToggleExceptions(TRUE);

	dhGetObject(L"winmgmts:{impersonationLevel=impersonate}!\\\\.\\root\\cimv2", NULL, &wmiSvc);

	wsprintfW(szQuery, L"SELECT * FROM Win32_PerfFormattedData_PerfProc_Process "
	                   L"WHERE IDProcess='%u'", GetCurrentProcessId());

	dhGetValue(L"%o", &colPerf, wmiSvc, L".ExecQuery(%S)", szQuery);

	FOR_EACH(wmiPerfItem, colPerf, NULL)
	{
		char *pszObject = NULL;
		char *pszBytes  = NULL;

		/* GetObjectText_ is useful to get the value of all properties. */
		dhGetValue(L"%s", &pszObject, wmiPerfItem, L".GetObjectText_()");
		dhGetValue(L"%s", &pszBytes,  wmiPerfItem, L".PrivateBytes");

		printf("This process is using %s bytes of VM.\nObject Text:%s",
		       pszBytes, pszObject);

		dhFreeString(pszObject);
		dhFreeString(pszBytes);

	} NEXT(wmiPerfItem);

	SAFE_RELEASE(colPerf);
	SAFE_RELEASE(wmiSvc);

	dhUninitialize(TRUE);
	getchar();
	return 0;
}

Note that the 'PercentProcessorTime' value appears to be for that instant, so you will probably have to spin up another thread to get a value other than zero (or monitor another process).

P.S MSI? I know MSI as Microsoft Installer, this API is usually referred to as WMI. Acronym overload.

[Magos]

You're right, WMI not MSI /slaps forehead

Using GetProcessId() in the query worked wonders!

It seems that all values are in fact BSTR strings no matter their type. However all the PercentXXXTime functions only give the string "0", but you mentioned that already.

VirtualBytes tells that my program uses ~40Mb while task manager says I'm using ~8Mb. If I allocate an extra 40Mb it does say it uses 80Mb, while the task manager indicates no change at all. How accurate is the task manager really? This does not sound right on its side...

Is "VirtualBytes" accurate enough to tell how much memory the program uses?

[anonytmouse]

It seems that all values are in fact BSTR strings no matter their type.

I think that it may only be 64 bit values that are returned as strings. 64bit variant integers (VT_I8 & VT_UI8) were only introduced in Windows XP but WMI targets earlier platforms.

VirtualBytes tells that my program uses ~40Mb while task manager says I'm using ~8Mb. If I allocate an extra 40Mb it does say it uses 80Mb, while the task manager indicates no change at all. How accurate is the task manager really? This does not sound right on its side...

VirtualBytes gives the number of bytes reserved in the virtual address space. This includes mapped modules and files and reserved space for stacks. Much of this is shared with other processes (eg. kernel32.dll) and so is not a good number for estimating the virtual memory usage of a process. It appears that 'PrivateBytes' (as used in previous example) gives a much better number and from my limited experimenting seems to match with what task manager reports. 'WorkingSet' should give the physical memory that the process is using.

The fact that you didn't see a change in task manager after allocating memory suggests that you are looking at the 'Mem Usage' column. This indicates physical memory in use. You may want to configure TM to also show the 'Virtual Memory Size' column.