Shared class members over Dll Boundary

This is a discussion on Shared class members over Dll Boundary within the C++ Programming forums, part of the General Programming Boards category; Now here's a good question: is it possible to share the SAME instance of a static class member variable in ...

  1. #1
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031

    Shared class members over Dll Boundary

    Now here's a good question: is it possible to share the SAME instance of a static class member variable in a template class over dll boundaries? Currently, both the DLL and the EXE has a local copy of it, which is bad, kinda defeats the purpose of being static.
    How and why all comse back to the memory manager class:

    Code:
    	static std::map<const void*, PointerInfo*> m_InfoMap;
    	static CCriticalSection cThisMapSync;
    I need, or want, these to be shared also by any DLL that uses it. I don't know a good solution to it, nor do I think I have a good solution, or can think of one.
    The workaround so far, is simply not to use the m_InfoMap static member var at all, hence no locking and no local copies used when using an object passed to a exported Dll function.

    Neverless to say, this isn't safe. Not the least bit. Locking variables are supposed to be the same everywhere and only one instance in the class.

    Also, I can throw two more questions, which I'm not sure of myself:
    I don't suppose there's any locking class in the standard library? I already switched out CMap for std::map.
    And...
    Member functions such as these:
    Code:
    	template<typename NewT> operator CMemoryManager<NewT>& () const
    	template<typename T2> static CMemoryManager<T> MemoryManagerNew(T2* pNew) throw(...)
    How do you actually define them in the code? I can only define them where they're declared (inline functions). Everytime I try to move them, the compiler will only say it can't find the member function declaration.
    Here is what I tried:
    Code:
    template<typename T, typename T2> CMemoryManager<T>::operator CMemoryManager<T2>& () const { }
    This does, of course, generate the error: can't find member function.
    Any ideas on that one?

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Since each .DLL and .EXE is compiled as it's own free-standing module, there is no way that a static member of a class that is implemented in a DLL can also be the same static in the .EXE - it just doesn't work like that.

    What you could do is that you instantiate the class in the DLL and pass a pointer to that instance into the .EXE - I think that would work. But if you have multiple instances, they would have their own static data.

    Hang on, have you tried puttign __declspec(dllexport/dllimport) in your header file, for the static data?

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    Quote Originally Posted by matsp View Post
    Since each .DLL and .EXE is compiled as it's own free-standing module, there is no way that a static member of a class that is implemented in a DLL can also be the same static in the .EXE - it just doesn't work like that.
    I've already figured that part out

    Quote Originally Posted by matsp View Post
    What you could do is that you instantiate the class in the DLL and pass a pointer to that instance into the .EXE - I think that would work. But if you have multiple instances, they would have their own static data.
    That's not a very elegant solution since it defeats the purpose of a memory manager. I require multiple instances, lots and lots of them.

    Quote Originally Posted by matsp View Post
    Hang on, have you tried puttign __declspec(dllexport/dllimport) in your header file, for the static data?
    Nope. Maybe I can try.
    Ah, no... it won't work:
    Error 2 error C2491: 'CMemoryManager<T>::m_InfoMap' : definition of dllimport static data member not allowed
    It says dllimport on static data member is not allowed. Hmmm.

    But the problem is that since it's a template class, it can't be exported through a dll.
    Last edited by Elysia; 11-12-2007 at 11:29 AM.

  4. #4
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    Quote Originally Posted by Elysia View Post
    That's not a very elegant solution since it defeats the purpose of a memory manager. I require multiple instances, lots and lots of them.
    But surely the static member is a single one, right? So just "wrap an object" around that - you may even be able to write function that returns the address of your static member inside the class.

    That is if my declspec() trick doesn't work.

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  5. #5
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    Hmmm. So pretty much, I can make function exported through the DLL that returns the address to the a struct that contains these members? That's what I think you might be implying. The memory manager can call the exported function to get the address to a single structure.
    The only problem I can see with that is that it will require me to include the DLL with every project that uses the memory managers and all other dlls/files must also be linked agains the DLL. But nothing comes free, right? It's my common code library, so it should be fine since I will pretty much include it in all projects anyway.
    Last edited by Elysia; 11-12-2007 at 11:35 AM.

  6. #6
    Registered User
    Join Date
    Nov 2006
    Posts
    519
    If this is an option you could switch to linux/shared objects. sharing of static data is supported there. under windows the only workaround I was able to find is passing of references/addresses

  7. #7
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    Unfortunately, Linux is not an option. I'm trying the above method to see if it works. It sounds fun and interesting!

  8. #8
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    So basically I added this:
    Code:
    namespace MemoryManager
    {
    	MemoryManagerStatics sStaticVars;
    	MemoryManagerStatics* GetStaticMembers()
    	{
    		return &sStaticVars;
    	}
    }
    ...And this...
    Code:
    namespace MemoryManager
    {
    	struct PointerInfo
    	{
    		DWORD dwRefCount;
    		CCriticalSection cSync;
    		bool bExistInMap;
    	};
    
    	struct MemoryManagerStatics
    	{
    		std::map<const void*, PointerInfo*> m_InfoMap;
    		CCriticalSection cThisMapSync;
    	};
    
    	AFX_EXT_CLASS MemoryManagerStatics* GetStaticMembers();
    }
    ...And changed from...
    Code:
    			CMemoryManager::cThisMapSync.Lock();
    			CMemoryManager::m_InfoMap[p] = m_pInfo;
    			CMemoryManager::cThisMapSync.Unlock();
    ...to...
    Code:
    			MemoryManager::GetStaticMembers()->cThisMapSync.Lock();
    			MemoryManager::GetStaticMembers()->m_InfoMap[p] = m_pInfo;
    			MemoryManager::GetStaticMembers()->cThisMapSync.Unlock();
    And now all instances of the class should be getting the same instance of the "static" variables.

  9. #9
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,702
    You can export/import static class members - the systax is just a little tricky - and it depends on how you're exporting your other class members. But what you've done works as well.

    gg

  10. #10
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,702
    I would also add that you're "playing with fire" by exporting a std::xxx class object. You're making an implicit assumption that every client that imports that member is compiled with the EXACT same std::xxx implementation.

    If "MemoryManagerStatics" must be exposed to clients, then you can consider using the PIMPL idiom to hide its details within the DLL.

    gg

  11. #11
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    There's no easy way to hide the usage of std::xxx objects since the class is a template class, I can't export it through a DLL (which was the preferred way).
    Each instance must use the MemoryManager code separately.
    If you have any ideas or suggestions on how to go around this, I'm all ears.

  12. #12
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    All this sharing stuff would work for normal classes, but not for templates. Templates are instantiated many times even for a single module, and then the linker kicks out the redundant copies. With a DLL, that just doesn't work.

    Due to the fixed nature of the DLL interface and the variable amount of templates, it's equally impossible to export any function from the DLL that exposes a shared copy of the map, because there's one for every instantiation. Nor is it possible to give the DLL an init function that is passed a pointer to the executable's map, or anything like that.

    The problem is simply that you have an unknown number of maps.

    As far as I can tell, there are only two solutions to the dilemma.
    1) Get rid of the lookup map. (You knew I'd be saying this.)
    2) Have a single map shared between all instantiations. Due to the unique nature of allocated pointers, this is not a type safety problem (to be a problem, you'd have to cast the raw pointer in the first place). However, it does mean that you need to rip the PointerInfo struct out of the class, and this in turn means a lot of ugly casting. This also means an additional performance penalty, because the map now contains every single pointer, and thus quickly grows way out of proportion. Also, every lookup has to look up amidst masses of pointers that are ineligible in the first place.

    By the way, have you considered the overhead of the map thing? A map node is typically 3 pointers and a boolean large (due to alignment, typically 4 pointers, unless ugly tricks are used). Add to that the value type - another pointer plus sizeof(PointerInfo). On a 32 bit system, you get a total of what, 32 bytes? That's a LOT. And it doesn't even count the allocation overhead of the map nodes. Let's say a total of 36 bytes. This may well be about the average of your object size. In other words, unless you use performance critical pointers for most cases, this little gag doubles your memory requirements.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  13. #13
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,702
    You are correct in that non-instantiated templates can not be exported from a DLL - but instantiated templates can be exported - with a few limitations from MS...

    This should cover it: http://support.microsoft.com/kb/q168958/

    >> There's no easy way to hide the usage of std::xxx objects...
    It's fairly easy to hide any implementation detail. Here's a snippet:

    Dll header file:
    Code:
    // this should be exported since it contains MFC object
    struct AFX_EXT_CLASS PointerInfo
    {
    	DWORD dwRefCount;
    	CCriticalSection cSync;
    	bool bExistInMap;
    };
    
    // don't forget to export struct's that are returned by exported functions
    struct AFX_EXT_CLASS MemoryManagerStatics
    {
        struct pimpl;
        pimpl *m_pimpl;
    
        MemoryManagerStatics();
        ~MemoryManagerStatics();
    
        void Insert(const void *p, PointerInfo *pinfo);
        // add methods as needed
    };
    Dll source file:
    Code:
    struct MemoryManagerStatics::pimpl
    {
        std::map<const void*, PointerInfo*> m_InfoMap;
        CCriticalSection cThisMapSync;
    };
    
    MemoryManagerStatics::MemoryManagerStatics()
    {
        m_pimpl = new MemoryManagerStatics::pimpl;
    }
    
    MemoryManagerStatics::~MemoryManagerStatics()
    {
        delete m_pimpl;
    }
    
    void MemoryManagerStatics::Insert(const void *p, PointerInfo *pinfo)
    {
        m_pimpl->cThisMapSync.Lock();
        m_pimpl->m_InfoMap[p] = pinfo;
        m_pimpl->cThisMapSync.UnLock();
        // or you could add Insert() method to pimpl and call m_pimpl->Insert()
    }
    gg

  14. #14
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Posts
    23,031
    Quote Originally Posted by CornedBee View Post
    By the way, have you considered the overhead of the map thing? A map node is typically 3 pointers and a boolean large (due to alignment, typically 4 pointers, unless ugly tricks are used). Add to that the value type - another pointer plus sizeof(PointerInfo). On a 32 bit system, you get a total of what, 32 bytes? That's a LOT. And it doesn't even count the allocation overhead of the map nodes. Let's say a total of 36 bytes. This may well be about the average of your object size. In other words, unless you use performance critical pointers for most cases, this little gag doubles your memory requirements.
    Memory is not an issue.
    But since you wanted to know the overhead of the class, here's some statistics for you:
    CMemoryManagerStatics, all statics variables:

    Code:
    	class AFX_EXT_CLASS CMemoryManagerStatics
    	{
    	public:
    		std::map<const void*, PointerInfo*> m_InfoMap; // Size: 28 bytes
    		CCriticalSection cThisMapSync; // Size: 36 bytes
    		HANDLE hThreadLock; // Size: 4 bytes
    		CMemoryManagerStatics();
    		~CMemoryManagerStatics();
    	};
    Total size: 68 byte
    Padding: 0 bytes

    PointerInfo struct:
    Code:
    	struct PointerInfo
    	{
    		DWORD dwRefCount; // Size: 4 bytes
    		CCriticalSection cSync; // Size: 36 bytes
    		bool bExistInMap; // Size: 1 byte
    		bool bThreadSafetyOn; // Size: 1 byte
    	};
    Total size: 42 bytes (44 bytes with padding).
    Padding: 2 bytes

    CMemoryManager class:
    Code:
    template<typename T> class CMemoryManager
    {
    	// ...
    	T* p; // Size: 4 bytes
    	MemoryManager::PointerInfo* m_pInfo; // Size: 4 bytes
    #ifdef _DEBUG
    	DWORD m_dwThreadId; // Size: 4 bytes
    #endif
    	// ...
    }
    Total size: 12 bytes (16 bytes with padding)
    Padding: 4 bytes

    Also remember that statics is a one-time overhead in the DLL ONLY. It's shared by all classes.
    The PointerInfo structure is shared by all classes pointing to the same memory, so the general overhead is 16 bytes, and 44 bytes with every new pointer. The biggest suckers are the CCriticalSection object, which is necessary for thread safety.
    Here's another one for you: With 100.000 instances of the class, all with unique pointers, the total allocation is 8 800 004 bytes (8.8 MB). On today's systems, that's nothing.
    How did I get this info? Simple:

    Code:
    	CMemoryState a,b,c;
    	a.Checkpoint();
    	
    	ppnew<int>* p = new ppnew<int>[100000];
    	pp<int> p7;
    
    	b.Checkpoint();
    	c.Difference(a, b);
    	c.DumpStatistics();
    I'm wading through all this information to see what can be done, right now. I also have another idea for operations on the pointer itself. I'll have to see if it works out alright.

    Quote Originally Posted by Codeplug View Post
    You are correct in that non-instantiated templates can not be exported from a DLL - but instantiated templates can be exported - with a few limitations from MS...

    This should cover it: http://support.microsoft.com/kb/q168958/
    Codeplug, that's for the information! I thought it might work if I actually specified a type of the class to be exported, but I never knew how to do it.
    Still, I don't think it's a very good idea since it would practically hamper on what objects could be used in the class (and maybe bloat the DLL?).
    Each time I wanted to use a new type, I'd get linking errors and would have to modify and recompile the DLL. Sad to say, it's a little unpretty and totally unusable to anyone but myself.
    Thanks for the info, though.

    Quote Originally Posted by Codeplug View Post
    >> There's no easy way to hide the usage of std::xxx objects...
    It's fairly easy to hide any implementation detail. Here's a snippet:
    I see. So you're basically hiding the STL classes in custom structs/classes and declaring those structs/classes in the .cpp file, right?
    Last edited by Elysia; 11-13-2007 at 12:28 PM.

  15. #15
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    You are correct in that non-instantiated templates can not be exported from a DLL - but instantiated templates can be exported
    For a smart pointer, that's quite useless.

    Also remember that statics is a one-time overhead in the DLL ONLY. It's shared by all classes.
    I'm talking about the per-node allocation the map does.
    Also, static members of a template are per template instantiation. For every type you instantiate your smart pointer with, you get another map. Still, that overhead is really rather irrelevant - even if you instantiate the template for 200 types, it won't exceed a few k.

    I'm still talking about the per-allocation overhead. Suppose:
    Code:
    typedef CMemoryManager<int> int_ptr;
    typedef std::vector<int_ptr> ptr_vec;
    // Measure memory usage here.
    int *demo = new int;
    // Measure memory usage here.
    delete demo;
    ptr_vec manyPointers(100000);
    // Measure memory usage here.
    for(ptr_vec::iterator it = manyPointers.begin(); it != manyPointers.end(); ++it) {
      *it = new int;
    }
    // Measure memory usage here.
    The initial two measurements ought to tell you how much overhead new itself has. This is a guideline to measure the total overhead from your constructs. Every int is 4 bytes large, plus new's own overhead. Make that, for demonstration purposes, 12 bytes in total. Then the 100000 ints we allocate take 1200000 bytes of memory.
    The difference between the last two measurements minus the 1200000 bytes of the ints themselves tell you how much overhead you introduce.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. A question about class members and constructors
    By Megidolaon in forum C++ Programming
    Replies: 5
    Last Post: 01-30-2009, 03:01 PM
  2. Replies: 3
    Last Post: 10-31-2005, 12:05 PM
  3. Private Static class members
    By earth_angel in forum C++ Programming
    Replies: 13
    Last Post: 08-29-2005, 07:37 AM
  4. Protected Inheritance
    By golfinguy4 in forum C++ Programming
    Replies: 8
    Last Post: 12-27-2002, 10:56 AM
  5. Exporting Object Hierarchies from a DLL
    By andy668 in forum C++ Programming
    Replies: 0
    Last Post: 10-20-2001, 02:26 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21