Thread: setwindowshookex returns null

A bit more info for you.

After the CreateProcess call but before SetWindowsHookEx I now do GetGUIThreadInfo and pass the threadId as the first parameter and a pointer to a memset GUITHREADINFO structure (with the size set correctly) as the second paramerter. This returns false with the same error 87 "Invalid parameter". So it must be the threadId which is incorrect causing both GetGUIThreadInfo and SetWindowsHookEx to fail. The threadId I get from CreateProcess (see my first example). The process is still running when I call both GetGUIThreadInfo and SetWindowsHookEx. I also sleep 10 seconds after I do CreateProcess to give the process time to initialize before calling GetGUIThreadInfo (this is only temporary).

I see the same behaviour for both myapp and google chrome.

I believe this to be the cause of the problem

Looks like you have to wait until the thread makes its first call into the user32 kernel functions, which is when a certain struct is initialized which SWHE and GGTI require. The easiest way to wait for this is WaitForInputIdle. If you remove it from the code below it'll likely give you 87 as an error, which is the invalid parameter error

Code:

#define UNICODE
#define _UNICODE
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <stdio.h>

int main(int argc, char **argv)
{
    PROCESS_INFORMATION pi = {0};
    STARTUPINFO si = {sizeof(si), 0};
    BOOL bRet = CreateProcess(
        L"C:\\Windows\\System32\\winver.exe",
        NULL,
        NULL,
        NULL,
        FALSE,
        0,
        NULL,
        NULL,
        &si,
        &pi
    );
    if(bRet)
    {
        GUITHREADINFO gti = {sizeof(gti), 0};
        WaitForInputIdle(pi.hProcess, INFINITE);
        if(!GetGUIThreadInfo(pi.dwThreadId, &gti))
        {
            DWORD err = GetLastError();
            printf("GetGUIThreadInfo errored with %lu\n", err);
        }
        else
        {
            printf("Active window is %p\n", gti.hwndActive);
        }
        CloseHandle(pi.hThread);
        CloseHandle(pi.hProcess);
    }
    return 0;
}