Thread: Delay unloading DLL

Next problem is this:
DllMain, is as many know, pretty useless for initialization and cleanup, because many things will simply not work there. Window creations fails, destroying windows fails, etc.
So. I need to perform initialization after DllMain, which I solved by allowing the hooker application explicitly call a function to perform initialization after dll is loaded.
However, unload is a bigger problem. Obviously, by this time, the hooker app has terminated. DllMain can't perform cleanup either.
I tried hooking FreeLibrary to perform cleanup before doing actually unloading the DLL.
But since I hooked it, I must manually call FreeLibrary and since the code is within the virtual space of the DLL, when FreeLibrary tries to return, it will try to execute code from an invalid space, thus crashing.

I don't know of a solution to this. Either I use a very nasty hack, such as forging a return address or do something else about this.
Another thing I could think of is delay the unloading of the module for a bit. But I don't think there's such a function, is there?

Code:

BOOL WINAPI MyFreeLibrary(HMODULE hModule)
{
	std::tstring pModule = GetModuleFileNameHelper(hModule);
	if (_tcscmp(pModule.c_str(), TEXT("G:\\w00t\\Visual Studio 2005\\Projects\\Hooking\\Application\\Debug\\ThreadSpy.dll")) == 0) // If it's trying to free us.
	{
		TerminateHelper(NULL);
	}
	//const int FREELIBRARY_INDEX = 5;
	//return ((BOOL (WINAPI*)(HMODULE))HookedFunctions[FREELIBRARY_INDEX].pPrevFunc)(hModule);
	UnhookFunctions(HookedFunctions, HookedFunctionSize);
	UnHookAllModules(HookedFunctions, HookedFunctionSize);
	DeleteCriticalSection(&mhSection);
	return ::FreeLibrary(hModule); // FreeLibrary will crash the app
}

Code:

BOOL APIENTRY DllMain( HANDLE hModule, 
					  DWORD  ul_reason_for_call, 
					  LPVOID lpReserved
					  )
{
	switch (ul_reason_for_call)
	{
		case DLL_PROCESS_ATTACH:
		{
			InitializeCriticalSection(&mhSection);
			hCurrentHandle = hModule;
			// Now hook the functions
			HookFunctions(HookedFunctions, HookedFunctionSize);
			// And replicate in all modules
			HookAllModules(HookedFunctions, HookedFunctionSize);

			MapInitHelper();
			break;
		}
		case DLL_THREAD_ATTACH:
			break;
		case DLL_THREAD_DETACH:
			break;
		case DLL_PROCESS_DETACH:
		//UnhookFunctions(HookedFunctions, HookedFunctionSize);
		//UnHookAllModules(HookedFunctions, HookedFunctionSize);
		//DeleteCriticalSection(&mhSection);
		//TerminateHelper(NULL); // Cannot perform cleanup in DllMain
		break;
	}
	return TRUE;
}

Anyone has suggestions to this kind of trouble? I'm thinking a hack is necessary to make this work.

Made I made it unclear. I hooked the FreeLibrary function, so I manually need to call FreeLibrary there, or it will never be called (not during DLL_PROCESS_DETACH).
And if I don't hook it, then I don't need to call FreeLibrary, but I can't clean up properly during DLL_PPROCESS_DETACH.
The whole discussion is how to make the DLL clean up properly without crashing when the DLL is unloaded via FreeLibrary.

I know that when FreeLibrary returns, it returns to an invalid address. So if I could forge the return address, I could make it work...
But I'm not fluent enough in assembly to do it. And the problem is of course, how would I get the return address? It's stored somewhere on the stack, but that could be anywhere depending on how many pushes that have been made.

But that makes the solution to use a fully assembly function. But I have no idea how to do this on MSVC...

What it will do it call my library's cleanup function, which looks like this:

Code:

		MsgDlg.Close();
		MemoryManager::DeleteStaticVars();
		CoUninitialize();
		if (WaitForSingleObject(hMsgThread, 5000) == WAIT_TIMEOUT)
			TRACE("Warning: Waiting for message dialog to terminate timed out in Stuff.dll.\n");

Now. Unfortunately, it fails to properly destroy the window because DestroyWindow blocks while in DLL_PPROCESS_DETACH.

OK, not so ideal. Perhaps it might work.
But unfortunately, it is not so ideal. If I inject the DLL back again and perform initialization, I'll see that MFC's state is messed up and it actually creates, then destroys the window it created (and thus failing) when calling MsgDlg.Create(...).

So either I made a big fat hack with FreeLibrary or I make a smaller hack to hack the state of the dialog before destroying it.
These are the options I see.
(You can already see a hack with MsgDlg.Close, however. There's no such function in a CDialog class. In fact, it's my own small hack. What it does it signal an internal flag in the class and inside WindowProc, it checks this flag and unsets the continue modal loop flag, thus avoiding a nasty ASSERT if closing the dialog from a thread other than the one that's running the message loop. It it for this that it waits for WaitForSingleObject. MFC is in such a sad state...)

Fancy this. My solution for now.

Code:

__declspec(naked) BOOL WINAPI /*__fastcall */MyFreeLibrary(HMODULE hModule)
{
	FreeLibraryInternal(hModule);
	//return ::FreeLibrary(hModule);
	__asm
	{
		pop edx;		// Store return address
		add esp, 4;		// Clean the stack
		push hModule;	// Push argument 1, hModule
		push edx;		// Push spoofed return address
		mov eax, FreeLibrary;
		jmp eax;		// Call FreeLibrary
	}
}

If there's a better way to do this, I'm all ears, of course.
For now, I'll use this.

This assembly code is right, right?

Code:

__declspec(naked) BOOL WINAPI /*__fastcall */MyFreeLibrary(HMODULE hModule)
{
	FreeLibraryInternal(hModule);
	__asm
	{
		mov eax, [esp+4];
		mov edx, [esp];
		add esp, 8;
		push eax;
		push edx;
		jmp dword ptr FreeLibrary;
		ret;
	}
}

It successfully calls FreeLibrary, unloads the DLL and immediately jumps back to the caller who initially called FreeLibrary, but somehow, I get a crash in MFC's message handling routine afterwards.
I had a look, but the important registers seem intact afterwards

Before:
EAX = 01020000 EBX = 00000000 ECX = 7C80E82B EDX = 7C97C0D8 ESI = 0013F3E0 EDI = 0013F4C4 EIP = 004128EA ESP = 0013F3E0 EBP = 0013F4C4 EFL = 00000246

After:
EAX = 00000001 EBX = 00000000 ECX = 7C917304 EDX = 7C97C0D8 ESI = 0013F3E0 EDI = 0013F4C4 EIP = 004128FD ESP = 0013F3E0 EBP = 0013F4C4 EFL = 00000246

Hmmm. What gives?
The app is using MFC (the hooked app).
The spy dll does not use MFC.
The spy dll uses a helper MFC dll to
interface with another MFC dll.

I feel it should work with or without MFC, unless something is going very wrong. But it's the app that crashes, not any of the DLLs. They unload fine.