setwindowshookex returns null

Hi All,

I want to use setwindowshookex for hooking a dll into one of my own applications. setwindowshookex returns null setting error code 87 (invalid parameter), which I believe is the threadid parameter. This is a cut down version of my code to show what im trying to do.

Code:

---

STARTUPINFO startInfo;
PROCESS_INFORMATION procInfo;

memset(&startInfo, 0, sizeof(startInfo));
memset(&procInfo, 0, sizeof(procInfo));

startInfo.cb = sizeof(startInfo);
                               
CreateProcess( "c:/myapp.exe", NULL, NULL, NULL, false, DETACHED_PROCESS, 0, 0,
&startInfo, &procInfo ) ;

HINSTANCE dll = LoadLibrary(TEXT("c:/mydll.dll")) ;
if (dll == NULL) return false ;

DllHookProc hookProc = (DllHookProc)GetProcAddress(dll,"HookProc");

if (hookProc == NULL) return false ; // will also free dll.

HHOOK hook = SetWindowsHookEx( WH_CALLWNDPROC,(HOOKPROC)hookProc,
                dll, procInfo.dwThreadId );

---

the hook variable is set to null at this point, however LoadLibrary and GetProcAddress all work correctly. Also CreateProcess works ok as I can see the application load and the procInfo.dwThreadId is none 0

Any ideas what i am doing incorrectly?

Thanks for any help

Quote:

---

Originally Posted by cloudy

Code:

---

DllHookProc hookProc = (DllHookProc)GetProcAddress(dll,"HookProc");

---

---

What is a 'DllHookProc'?

I can't find 'DllHookProc' on MSDN or google (or am I just not looking in the right spots?).

Can you cast it to a HOOKPROC successfully?

Hi,

Thanks for the reply.

That is a typedef of the hook function signature that is being exported by my dll via a .def file. I.E.

Code:

---

typedef LRESULT (CALLBACK *DllHookProc)(int nCode, WPARAM wParam, LPARAM lParam);

---

The GetProcAddress function returns happily with the address of the HookProc function.

Thanks again

Is myapp.exe an executable that immediately returns? If so, then the threadID would no longer be valid.

Thanks for the reply,

it doesn't immediately return I can still see running.

Does it work if you set the threadID to 0?
Are you attempting to inject a 32 bit DLL into a 64 bit process (or vice-versa)?

Does it work if you set the threadID to 0?
Are you attempting to inject a 32 bit DLL into a 64 bit process (or vice-versa)?

This may be a dumb question but is the SetWindowsHookEX call in the DLL or the EXE?

Most of what that function does only works when you call it from a DLL... any of the parameters in it's description marked "Global" will only work in DLLs.

Quote:

---

Originally Posted by CommonTater

This may be a dumb question but is the SetWindowsHookEX call in the DLL or the EXE?

Most of what that function does only works when you call it from a DLL... any of the parameters in it's description marked "Global" will only work in DLLs.

---

I don't think the SetWindowHookEx call needs to be in the DLL. Just the hook procedure needs to be in the DLL.

Quote:

---

Originally Posted by bithub

I don't think the SetWindowHookEx call needs to be in the DLL. Just the hook procedure needs to be in the DLL.

---

Yeah, it does... the call to SetWindowsHookEx() and the DLLHookProc() have to be in the same scope otherwise the function pointer is invalid.

Here's an example that returns the window handle in a message to the main window proc in an EXE and exports a call that lets you retrieve the program's name....

The DLL...

Code:

---

/// self
#include "Helper.h"

/////////////////////////////////////////////////////////////////
// Shared data segment

#pragma data_seg("Hook")
HINSTANCE  hdll    = NULL;              // dll handle
HWND        HWind  = NULL;              // handle of Server window
HHOOK      Hook  = NULL;                // message hook handle
LAUNCHINFO  LInfo  = {0};                // info about program launch
#pragma data_seg()
#pragma comment(linker,"/SECTION:Hook,S")

/////////////////////////////////////////////////////////////////////////////////////////
// Export
//

// return launch info to server
MMAPI VOID HelperGetInfo(pLAUNCHINFO Info)
  { memcpy(Info,&LInfo,sizeof(LInfo)); }



/////////////////////////////////////////////////////////////////////////////////////////
// gather information
//

// get information about launched program
VOID HookGetInfo(HWND wParm)
  { TCHAR  cmdl[MAX_PATH] = {0};  // command line
    TCHAR  pgm[MAX_PATH] = {0};    // program name
    PTCHAR  args;                  // arguments
    // purge old data
    memset(&LInfo,0,sizeof(LAUNCHINFO));
    // set program window handle
    LInfo.Handle = wParm;   
    // get program info
    GetModuleFileName(NULL,pgm,MAX_PATH);
    _wsplitpath(pgm,NULL,NULL,LInfo.Name,LInfo.Ext);
    // get launched file info
    wcsncpy(cmdl,GetCommandLine(),MAX_PATH);
    args = PathGetArgs(cmdl);
    if(wcslen(args) < 1)
      return;
    args = wcsrchr(args,L'.');
    if (args)
      { wcscpy(LInfo.Type,args);
        // fix trailing quote
        args = wcsrchr(LInfo.Type,L'\"');
        if (args)
          *args = 0; } }



/////////////////////////////////////////////////////////////////////////////////////////
// Handle Shell Hook
//

// hook tosser
MMAPI LRESULT CALLBACK AppCatcher(INT Code, WPARAM wParm, LPARAM lParm)
  { switch (Code)
      { case HSHELL_WINDOWCREATED  :        // new window
          // gather launch info
          HookGetInfo((HWND) wParm);
          // send message to Server, wparam = Handle             
          PostMessage(HWind,UM_HELPERRUN,wParm,0);
          break;
        case HSHELL_WINDOWDESTROYED :        // window closing
          // send messate to Server
          PostMessage(HWind,UM_HELPEREXIT,wParm,0);
          break; }
    return CallNextHookEx(RMHook,Code,wParm,lParm); }



/////////////////////////////////////////////////////////////////
// Set the shell hook
// Returns hook status 1 = active, 0 = not
//

// apply the hook to launched programs
MMAPI BOOL WINAPI SetHook(HWND hWind)
  { HWind = hWind;  // handle of window to receive message
    Hook = SetWindowsHookEx(WH_SHELL,&AppCatcher,hdll,0);
    return (Hook != NULL); }     



// remove the hook from launched programs
MMAPI BOOL WINAPI FreeHook(void)
  { return UnhookWindowsHookEx(Hook); }



/////////////////////////////////////////////////////////////////
//  DLL entry point
//
BOOL APIENTRY DllMain(HINSTANCE hinst, DWORD reason, LPVOID reserved)
  { if (reason == DLL_PROCESS_ATTACH)  // save instance handle
          hdll = hinst;
    return 1; }

---

And the header file....

Code:

---

#ifndef HELPER_H
#define HELPER_H

// for the compiler
#define UNICODE
#define _UNICODE
#define WIN32_DEFAULT_LIBS
#define WIN32_LEAN_AND_MEAN
#define _WIN32_WINNT 0x0502
#define _X86_

// project global headers
#include <windows.h>
#include <shlwapi.h>
#include <wchar.h>

#include <globaldefs.h>

#pragma lib "Helper.lib"

#define MMAPI __declspec(dllexport)

// messages sent by helper dll
#define UM_HELPERRUN  WM_APP + 444
#define UM_HELPEREXIT WM_APP + 445

// launch information
#pragma pack(1)
typedef struct tLAUNCHINFO
  { TCHAR Name[MAX_PROGRAMNAME]; // short program name
    TCHAR Ext[MAX_TYPENAME];    // program type
    TCHAR Type[MAX_TYPENAME];    // file type
    HWND  Handle; }              // program window handle 
  LAUNCHINFO, *pLAUNCHINFO;
#pragma pack()


// retrieve the command line
MMAPI VOID HelperGetInfo(pLAUNCHINFO Info);

// set the hook
MMAPI BOOL WINAPI SetHook(HWND hWind);

// release the hook
MMAPI BOOL WINAPI FreeHook(void);


#endif // HELPER_H

---

This is extracted from working code, some things were changed to not advertise which application ... so any errors are not my fault.

Quote:

---

Originally Posted by CommonTater

Yeah, it does... the call to SetWindowsHookEx() and the DLLHookProc() have to be in the same scope otherwise the function pointer is invalid.

---

No, it doesn't. If it worked like that you'd never be able to call dll functions in the first place.

Quote:

---

Originally Posted by adeyblue

No, it doesn't. If it worked like that you'd never be able to call dll functions in the first place.

---

Don't be silly... we call DLL functions by exporting them... Look at the header file there are three exported functions listed there.

The function needs the address of the hook proc...

Code:

---

HHOOK WINAPI SetWindowsHookEx(
  __in  int idHook,
  __in  HOOKPROC lpfn,                              <---- it won't work with out this!
  __in  HINSTANCE hMod,                           
  __in  DWORD dwThreadId
);

---

How is it going to get that from a completely different module?

Thanks everyone for their help.

It is a 32bit dll into a 32 bit process.

I am calling setwindowshookex in my exe not dll.

If i change the threadId to 0 (i assume you mean do a global system hook), then setwindowshookex does not return null but only injects the hook into some processes not all of them.

To test this I changed the threadid in setwindowshookex to my running visual studio exe and everything worked fine (meaning setwindowshookex did not return null) . But as soon as I changed it back to myapp.exe threadid setwindowshookex returns null again. I also tried changing my threadid to google chrome and setwindowshookex still returns null. It seems to work with some processes like visual studio but not others like chrome.

I assume the reason why it wont inject into myapp is the same as why it wont inject into chrome. Whatever that may be.

Thanks

Quote:

---

Originally Posted by cloudy

Thanks everyone for their help.

It is a 32bit dll into a 32 bit process.

I am calling setwindowshookex in my exe not dll.

---

And that's why it's not working... It can't find the *address* of the hookproc in the dll from the exe file.

Hooks Overview (Windows)

Thanks for the reply.

Why does it work injecting into visual studio then?

I just tried the following from within the Dll:

Code:

---


LRESULT CALLBACK HookProc (int nCode, WPARAM wParam, LPARAM lParam )
{
        return CallNextHookEx( 0, nCode, wParam, lParam);
}

void Test( unsigned long threadId )
{
        HHOOK hook = SetWindowsHookEx(WH_CALLWNDPROC,(HOOKPROC)HookProc,
                NULL, threadId );

        std::ofstream os ("c:/SetWindowsHookEx.txt") ;
        if (hook == NULL)
        os << "IsNull" ;
        else
        os << "IsNotNull" ;
        os.close() ;

}

---

Test is a dll exported function. ThreadId is the thread Id of myapp.exe (also tried again with chrome.exe and got the same result). c:/SetWindowsHookEx.txt contains the line "IsNull".

Thanks

A bit more info for you.

After the CreateProcess call but before SetWindowsHookEx I now do GetGUIThreadInfo and pass the threadId as the first parameter and a pointer to a memset GUITHREADINFO structure (with the size set correctly) as the second paramerter. This returns false with the same error 87 "Invalid parameter". So it must be the threadId which is incorrect causing both GetGUIThreadInfo and SetWindowsHookEx to fail. The threadId I get from CreateProcess (see my first example). The process is still running when I call both GetGUIThreadInfo and SetWindowsHookEx. I also sleep 10 seconds after I do CreateProcess to give the process time to initialize before calling GetGUIThreadInfo (this is only temporary).

I see the same behaviour for both myapp and google chrome.

I believe this to be the cause of the problem

Ok... well, have fun with that.

I even showed you working code from a software package I wrote and explained why yours doesn't work... but you just went wizzing right past it...

I can't help you if you aren't listening to the advice you are given.

Looks like you have to wait until the thread makes its first call into the user32 kernel functions, which is when a certain struct is initialized which SWHE and GGTI require. The easiest way to wait for this is WaitForInputIdle. If you remove it from the code below it'll likely give you 87 as an error, which is the invalid parameter error

Code:

---

#define UNICODE
#define _UNICODE
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <stdio.h>

int main(int argc, char **argv)
{
    PROCESS_INFORMATION pi = {0};
    STARTUPINFO si = {sizeof(si), 0};
    BOOL bRet = CreateProcess(
        L"C:\\Windows\\System32\\winver.exe",
        NULL,
        NULL,
        NULL,
        FALSE,
        0,
        NULL,
        NULL,
        &si,
        &pi
    );
    if(bRet)
    {
        GUITHREADINFO gti = {sizeof(gti), 0};
        WaitForInputIdle(pi.hProcess, INFINITE);
        if(!GetGUIThreadInfo(pi.dwThreadId, &gti))
        {
            DWORD err = GetLastError();
            printf("GetGUIThreadInfo errored with %lu\n", err);
        }
        else
        {
            printf("Active window is %p\n", gti.hwndActive);
        }
        CloseHandle(pi.hThread);
        CloseHandle(pi.hProcess);
    }
    return 0;
}

---