Delay unloading DLL

This is a discussion on Delay unloading DLL within the Windows Programming forums, part of the Platform Specific Boards category; Next problem is this: DllMain, is as many know, pretty useless for initialization and cleanup, because many things will simply ...

  1. #1
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    22,918

    Delay unloading DLL

    Next problem is this:
    DllMain, is as many know, pretty useless for initialization and cleanup, because many things will simply not work there. Window creations fails, destroying windows fails, etc.
    So. I need to perform initialization after DllMain, which I solved by allowing the hooker application explicitly call a function to perform initialization after dll is loaded.
    However, unload is a bigger problem. Obviously, by this time, the hooker app has terminated. DllMain can't perform cleanup either.
    I tried hooking FreeLibrary to perform cleanup before doing actually unloading the DLL.
    But since I hooked it, I must manually call FreeLibrary and since the code is within the virtual space of the DLL, when FreeLibrary tries to return, it will try to execute code from an invalid space, thus crashing.

    I don't know of a solution to this. Either I use a very nasty hack, such as forging a return address or do something else about this.
    Another thing I could think of is delay the unloading of the module for a bit. But I don't think there's such a function, is there?

    Code:
    BOOL WINAPI MyFreeLibrary(HMODULE hModule)
    {
    	std::tstring pModule = GetModuleFileNameHelper(hModule);
    	if (_tcscmp(pModule.c_str(), TEXT("G:\\w00t\\Visual Studio 2005\\Projects\\Hooking\\Application\\Debug\\ThreadSpy.dll")) == 0) // If it's trying to free us.
    	{
    		TerminateHelper(NULL);
    	}
    	//const int FREELIBRARY_INDEX = 5;
    	//return ((BOOL (WINAPI*)(HMODULE))HookedFunctions[FREELIBRARY_INDEX].pPrevFunc)(hModule);
    	UnhookFunctions(HookedFunctions, HookedFunctionSize);
    	UnHookAllModules(HookedFunctions, HookedFunctionSize);
    	DeleteCriticalSection(&mhSection);
    	return ::FreeLibrary(hModule); // FreeLibrary will crash the app
    }
    Code:
    BOOL APIENTRY DllMain( HANDLE hModule, 
    					  DWORD  ul_reason_for_call, 
    					  LPVOID lpReserved
    					  )
    {
    	switch (ul_reason_for_call)
    	{
    		case DLL_PROCESS_ATTACH:
    		{
    			InitializeCriticalSection(&mhSection);
    			hCurrentHandle = hModule;
    			// Now hook the functions
    			HookFunctions(HookedFunctions, HookedFunctionSize);
    			// And replicate in all modules
    			HookAllModules(HookedFunctions, HookedFunctionSize);
    
    			MapInitHelper();
    			break;
    		}
    		case DLL_THREAD_ATTACH:
    			break;
    		case DLL_THREAD_DETACH:
    			break;
    		case DLL_PROCESS_DETACH:
    		//UnhookFunctions(HookedFunctions, HookedFunctionSize);
    		//UnHookAllModules(HookedFunctions, HookedFunctionSize);
    		//DeleteCriticalSection(&mhSection);
    		//TerminateHelper(NULL); // Cannot perform cleanup in DllMain
    		break;
    	}
    	return TRUE;
    }
    Anyone has suggestions to this kind of trouble? I'm thinking a hack is necessary to make this work.
    Last edited by Elysia; 03-15-2008 at 11:09 AM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  2. #2
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,691
    >> I must manually call FreeLibrary ...
    During DLL_PROCESS_DETACH? That *shouldn't* be done. During DLL_PROCESS_DETACH, either the DLL or Process is going down. Don't see why you *need* to call FreeLibrary() in these cases.

    gg

  3. #3
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    22,918
    Made I made it unclear. I hooked the FreeLibrary function, so I manually need to call FreeLibrary there, or it will never be called (not during DLL_PROCESS_DETACH).
    And if I don't hook it, then I don't need to call FreeLibrary, but I can't clean up properly during DLL_PPROCESS_DETACH.
    The whole discussion is how to make the DLL clean up properly without crashing when the DLL is unloaded via FreeLibrary.

    I know that when FreeLibrary returns, it returns to an invalid address. So if I could forge the return address, I could make it work...
    But I'm not fluent enough in assembly to do it. And the problem is of course, how would I get the return address? It's stored somewhere on the stack, but that could be anywhere depending on how many pushes that have been made.

    But that makes the solution to use a fully assembly function. But I have no idea how to do this on MSVC...
    Last edited by Elysia; 03-15-2008 at 12:48 PM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  4. #4
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,691
    >> I can't clean up properly during DLL_PPROCESS_DETACH
    Well, let's take a step back and look at this problem and why you chose to hook FreeLibrary() as a solution. What kinds of cleanup need to occur that don't like being done from DllMain? What if you simply *leaked* the resources and allowed the OS to clean things up? It's not ideal, but it's better than crashing

    gg

  5. #5
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    22,918
    What it will do it call my library's cleanup function, which looks like this:
    Code:
    		MsgDlg.Close();
    		MemoryManager::DeleteStaticVars();
    		CoUninitialize();
    		if (WaitForSingleObject(hMsgThread, 5000) == WAIT_TIMEOUT)
    			TRACE("Warning: Waiting for message dialog to terminate timed out in Stuff.dll.\n");
    Now. Unfortunately, it fails to properly destroy the window because DestroyWindow blocks while in DLL_PPROCESS_DETACH.

    OK, not so ideal. Perhaps it might work.
    But unfortunately, it is not so ideal. If I inject the DLL back again and perform initialization, I'll see that MFC's state is messed up and it actually creates, then destroys the window it created (and thus failing) when calling MsgDlg.Create(...).

    So either I made a big fat hack with FreeLibrary or I make a smaller hack to hack the state of the dialog before destroying it.
    These are the options I see.
    (You can already see a hack with MsgDlg.Close, however. There's no such function in a CDialog class. In fact, it's my own small hack. What it does it signal an internal flag in the class and inside WindowProc, it checks this flag and unsets the continue modal loop flag, thus avoiding a nasty ASSERT if closing the dialog from a thread other than the one that's running the message loop. It it for this that it waits for WaitForSingleObject. MFC is in such a sad state...)

    Fancy this. My solution for now.
    Code:
    __declspec(naked) BOOL WINAPI /*__fastcall */MyFreeLibrary(HMODULE hModule)
    {
    	FreeLibraryInternal(hModule);
    	//return ::FreeLibrary(hModule);
    	__asm
    	{
    		pop edx;		// Store return address
    		add esp, 4;		// Clean the stack
    		push hModule;	// Push argument 1, hModule
    		push edx;		// Push spoofed return address
    		mov eax, FreeLibrary;
    		jmp eax;		// Call FreeLibrary
    	}
    }
    If there's a better way to do this, I'm all ears, of course.
    For now, I'll use this.
    Last edited by Elysia; 03-15-2008 at 01:56 PM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  6. #6
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,691
    You can't wait on a thread in DllMain either - so hack away

    gg

  7. #7
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    22,918
    This assembly code is right, right?
    Code:
    __declspec(naked) BOOL WINAPI /*__fastcall */MyFreeLibrary(HMODULE hModule)
    {
    	FreeLibraryInternal(hModule);
    	__asm
    	{
    		mov eax, [esp+4];
    		mov edx, [esp];
    		add esp, 8;
    		push eax;
    		push edx;
    		jmp dword ptr FreeLibrary;
    		ret;
    	}
    }
    It successfully calls FreeLibrary, unloads the DLL and immediately jumps back to the caller who initially called FreeLibrary, but somehow, I get a crash in MFC's message handling routine afterwards.
    I had a look, but the important registers seem intact afterwards

    Before:
    EAX = 01020000 EBX = 00000000 ECX = 7C80E82B EDX = 7C97C0D8 ESI = 0013F3E0 EDI = 0013F4C4 EIP = 004128EA ESP = 0013F3E0 EBP = 0013F4C4 EFL = 00000246

    After:
    EAX = 00000001 EBX = 00000000 ECX = 7C917304 EDX = 7C97C0D8 ESI = 0013F3E0 EDI = 0013F4C4 EIP = 004128FD ESP = 0013F3E0 EBP = 0013F4C4 EFL = 00000246
    Last edited by Elysia; 03-15-2008 at 03:28 PM.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  8. #8
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,691
    Since your are injecting a DLL that use MFC, I would try to link with MFC statically. Or don't use MFC in your DLL.

    gg

  9. #9
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    22,918
    Hmmm. What gives?
    The app is using MFC (the hooked app).
    The spy dll does not use MFC.
    The spy dll uses a helper MFC dll to
    interface with another MFC dll.

    I feel it should work with or without MFC, unless something is going very wrong. But it's the app that crashes, not any of the DLLs. They unload fine.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  10. #10
    Banned
    Join Date
    Nov 2007
    Posts
    678
    Sorry to make a not to the topic post!
    But, literally, my mind was boggling, and head was banging hard, while I read this thread!!!

    May be there are always the most complex ways to do the most simple jobs

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. non-MFC DLL with MFC app question.
    By Kempelen in forum Windows Programming
    Replies: 10
    Last Post: 08-20-2008, 08:11 AM
  2. temperature sensors
    By danko in forum C Programming
    Replies: 22
    Last Post: 07-10-2007, 08:26 PM
  3. dll communicating between each other
    By cloudy in forum C++ Programming
    Replies: 5
    Last Post: 06-17-2005, 03:20 AM
  4. Unloading a dll
    By zMan in forum C# Programming
    Replies: 1
    Last Post: 11-10-2004, 12:55 AM
  5. DLL and std::string woes!
    By Magos in forum C++ Programming
    Replies: 7
    Last Post: 09-08-2004, 01:34 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21