Resource manager tree

This is a discussion on Resource manager tree within the Game Programming forums, part of the General Programming Boards category; The concept seems sound, but you are re-implementing a lot of tree and list functionality that you don't need to ...

  1. #16
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,598
    The concept seems sound, but you are re-implementing a lot of tree and list functionality that you don't need to re-implement.
    How so? I'm not doing anything with the STL map except generating unique IDs. The linked list portion amounts to a couple of pointer swaps. I don't see how this is reinventing the wheel. The STL did not provide the data structure I needed and so I merged two and created my own.

    I re-coded the system to use DWORD IDs. I just simply reset the ID to the new ID if the resource must be re-created.

    I do appreciate the work you put into the explanation and code though. Resource management is fairly complex but I feel my solution has some usefulness. Either way I think either of our solutions will work far better than trying to cram everything into memory at load time.

  2. #17
    Cat without Hat CornedBee's Avatar
    Join Date
    Apr 2003
    Posts
    8,893
    All I'm saying is that Boost already provides such a "merged" data structure.
    All the buzzt!
    CornedBee

    "There is not now, nor has there ever been, nor will there ever be, any programming language in which it is the least bit difficult to write bad code."
    - Flon's Law

  3. #18
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,598
    Ok but gimme some credit here for evaluating the issue and designing my own.

  4. #19
    Registered User
    Join Date
    Apr 2006
    Posts
    43
    I really suck at asking questions in english =P

    I try again...

    What is the return valu of sizeof(DWORD) and sizeof(CResource *) on your plattform? It can't be any space saving to chose one or the other...

    It seems like you are wasting a lot of time searching for unique identifiers when you already have one at hand, i.e. the CResource pointer, which is an unique identifier...therefore you should't really need any random-code to make an unique identifier (observe that even if you use a perfect random number generator you don't get uniqness, only randomness, think of a 6 sided dice for example).

    Instead, do some bit-swizzling to remove apperent ordering of the index, that is cheap and should work ok.

    The suggested boost implementation looked nice too, but that adds external dependencies instead of course.

    And I hope you post the finished code.

    /f

  5. #20
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,598
    The CResource pointer is not a unique indentifier.

    I'm not using sizeof(DWORD) or sizeof(CResource) anymore. Each CResource is now a file-based resource and has a size data member. When the object is added to the cache I add this to the current cache size:


    sizeof(STLListNode)+it->second->GetSize();

    Where it is:

    std::map<DWORD, STLListNode *>::iterator it;

    Every object in the cache is a CResource derived object. The only way to uniquely identify these resources is either to use a hash, string, or value-based ID.
    Last edited by VirtualAce; 09-01-2007 at 10:06 PM.

  6. #21
    Registered User
    Join Date
    Apr 2006
    Posts
    43
    The CResource pointer is not a unique indentifier.
    Even if it currently isn't in your implementation, it definitely could be. Allocate a thin proxy object and the pointer to the object is unique.

    But my main point was the random generator stuff, you don't need it. You just need an increasing index (or the above pointer) that you move around the bits of to get performance out of your tree, that way you have all the uniqueness you need for your index, guaranteed no duplicates, without all that code that still don't get you unique identifiers.

    I'm obviously not able to get my attempt of subtle hinting of no difference of memory usage regarding DWORD and CResource* across so I'll just drop it, it's not like it's important anyway nowdays. I'm still used to thinking about saving a byte here and there in structs... =)

    Did you have performance problems with your resources or did you add the resource manager just in case? On modern computers I have always let the driver take care of texture loading myself, it's usually no problems at all.
    As I understand it you don't do any pre-use loading to avoid stuttering when it's needed in the scene, only chosing which texture is LRU, something that to my knowledge the graphics drivers already takes care of (OpenGL on linux at least).

    Keep coding, it's always fun to see what you're up to with your game, not so many here do as much progress as you =)

    /f

  7. #22
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,598
    How would I go about implementing this data structure using both a std::map and a std::list?

    I would need pointers into the list correct?

    Code:
    struct STLListNode
    {
      CResource *pResource;
      
      //For the linked list
      STLListNode *pNext;
      STLListNode *pPrev;
    };
    
    std::map<DWORD, STLListNode *> Resources;

  8. #23
    Registered User
    Join Date
    Sep 2001
    Posts
    752
    Quote Originally Posted by Bubba View Post
    How would I go about implementing this data structure using both a std::map and a std::list?

    I would need pointers into the list correct?
    You should use iterators to connect the list and map. Don't worry about them becoming invalidated, they won't.
    Lists and Maps have the important property that inserting elements do not invalidate any iterators. Erasing an element also does not invalidate any iterators except, of course, for the iterator pointing at the erased element.
    Code:
    typedef std::list <CResource *> CacheList;
    
    typedef std::map <DWORD, CacheList::iterator> CacheMap;
    Deceptively simple.
    Callou collei we'll code the way
    Of prime numbers and pings!

  9. #24
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,598
    Hehe. Very nice and simple. Dunno why I didn't think of it.

    Thanks. I think I'll go with your example if you don't mind.

    EDIT:

    Implemented the new system and it works great. Had some difficulties at the outset but it is working great now.

    Added a max size and desired size member to the cache system. The minimum size for any cache is 1MB and the max size is whatever the user specifies up to the total amount of memory in the system. Update() now checks for two conditions to detect cache thrash and to optimize the cache to prevent continual disk caching.

    Code:
    void CResMgr::Update()
    {
      bool bCashThrash=false;
    
      //Remove the resources at the bottom of the list until the cache is 
      //back down to the desired size or less than
      //the desired size
      while (m_dwCurSize>=m_dwDesiredSize) 
      {
    
        //Remove resource at the bottom or back of the list
        //This function subtracts CResource::m_uFileSize from the current cache size
        Remove(CacheList.back()->GetID());
        
        //Bail if only 1 item in list - this means that the cache is not big enough
        if (CacheList.size()==1) 
        {
          bCashThrash=true;
          break;
        }
      }
      
      //Detect total cash thrash and/or optimize cache size
      //If the miss ratio is 60&#37; or greater it is safe to say the cache is not large enough
      //May let the user specify this ratio later on
      //Increase the cache size by a half meg and see what happens
      if (bCashThrash || GetMissRatio()>=.60f) IncreaseSize(500000);
        
    }
    The resource manager also works as planned during lost device events. Now ALT-TAB does not kill my app and it recovers just as it should.
    Last edited by VirtualAce; 09-08-2007 at 08:14 AM.

Page 2 of 2 FirstFirst 12
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Sorting out a resource manager
    By psychopath in forum Game Programming
    Replies: 1
    Last Post: 11-10-2008, 07:12 PM
  2. Interpreter.c
    By moussa in forum C Programming
    Replies: 4
    Last Post: 05-28-2008, 06:59 PM
  3. Binary Search Trees Part III
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 10-02-2004, 04:00 PM
  4. Request for comments
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 01-02-2004, 10:33 AM
  5. BST/Red and Black Tree
    By ghettoman in forum C++ Programming
    Replies: 0
    Last Post: 10-24-2001, 11:45 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21