Next problem is this:
DllMain, is as many know, pretty useless for initialization and cleanup, because many things will simply not work there. Window creations fails, destroying windows fails, etc.
So. I need to perform initialization after DllMain, which I solved by allowing the hooker application explicitly call a function to perform initialization after dll is loaded.
However, unload is a bigger problem. Obviously, by this time, the hooker app has terminated. DllMain can't perform cleanup either.
I tried hooking FreeLibrary to perform cleanup before doing actually unloading the DLL.
But since I hooked it, I must manually call FreeLibrary and since the code is within the virtual space of the DLL, when FreeLibrary tries to return, it will try to execute code from an invalid space, thus crashing.
I don't know of a solution to this. Either I use a very nasty hack, such as forging a return address or do something else about this.
Another thing I could think of is delay the unloading of the module for a bit. But I don't think there's such a function, is there?
BOOL WINAPI MyFreeLibrary(HMODULE hModule)
std::tstring pModule = GetModuleFileNameHelper(hModule);
if (_tcscmp(pModule.c_str(), TEXT("G:\\w00t\\Visual Studio 2005\\Projects\\Hooking\\Application\\Debug\\ThreadSpy.dll")) == 0) // If it's trying to free us.
//const int FREELIBRARY_INDEX = 5;
//return ((BOOL (WINAPI*)(HMODULE))HookedFunctions[FREELIBRARY_INDEX].pPrevFunc)(hModule);
return ::FreeLibrary(hModule); // FreeLibrary will crash the app
Anyone has suggestions to this kind of trouble? I'm thinking a hack is necessary to make this work.
BOOL APIENTRY DllMain( HANDLE hModule,
hCurrentHandle = hModule;
// Now hook the functions
// And replicate in all modules
//TerminateHelper(NULL); // Cannot perform cleanup in DllMain