Storing resources into a single file

This is a discussion on Storing resources into a single file within the Game Programming forums, part of the General Programming Boards category; This is not a new idea or anything, but while I'm google'ing for it I'm thinking someone might have some ...

  1. #1
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856

    Storing resources into a single file

    This is not a new idea or anything, but while I'm google'ing for it I'm thinking someone might have some knowledge on the subject. Instead of leaving the dozens of resources (bmp's, wav's, mid's) laying around my game's directory and whatnot, I'm going to opt in to dumping all of them into a single file (like pak and pk3 files) and load them during initialization. My question:
    Should I load the resource in memory or extract them all to a temporary directory, load, and delete? I would prefer the former (unless there are good reasons not to) because I don't like the idea of creating/deleting such temp files every run.

    What do you think? Please point me at any good resources that might assist in my quest. Thanks.

    [edit]
    If you know of any resources detailing how to go about loading resources (BMP, WAV, MID) from memory, let me know.
    [/edit]
    Last edited by LuckY; 08-10-2004 at 08:45 PM.

  2. #2
    vae victus! skorman00's Avatar
    Join Date
    Nov 2003
    Posts
    594
    I would suggest loading them into memory instead of temporary files, simply because they're gonna have to get in there eventually anyway. However, you would only load up the ones you need, and the way you go about that depends on how you organize your resource files (if it's in a giant one, you would only extract that which you need, or you can make a resource file for each level or area and load everything in that file).

  3. #3
    ---
    Join Date
    May 2004
    Posts
    1,379
    isnt this like a virtual file system?
    try and google for that.

  4. #4
    Registered User Frobozz's Avatar
    Join Date
    Dec 2002
    Posts
    546
    You should just load them into memory. With most computers shipping with 256 to 512mb of memory available it isn't too likely you'll have problems.

  5. #5
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856
    I have a class I wrote a while back that provides an interface for PAK file creation/manipulation, so that will serve as my VFS. The bigger issue is that I would like to (as we all agree) load my files into memory then into the application, but I have no idea how that could be done. I know that with a little research loading a bitmap from memory won't be too difficult, but WAVs and MIDIs are a whole other story...
    Without any idea on how to do such a thing I will probably end up just extracting from the PAK to the machine's temp directory, loading them, then deleting them.

  6. #6
    Crazy Fool Perspective's Avatar
    Join Date
    Jan 2003
    Location
    Canada
    Posts
    2,640
    instead of loading the entire blob file from disk and then worrying about creating these types from memory, why dont you build the types as they are loaded from disk. I dont know if this is possible with the pak format, but you can just open the file with a stream or whatever, then pass the stream around to the functions which load the bmp's, wavs, and midi's in the standard ways. This way each component is built as the data is loaded, rather than loading all the data into memory and trying to bend other functions to build from memory.

    just a thought, this would be my approach.

  7. #7
    vae victus! skorman00's Avatar
    Join Date
    Nov 2003
    Posts
    594
    [quote = Perspective]I dont know if this is possible with the pak format[/quote]
    As long as you extract and decompress as necessary, it's the same as loading from a normal file. loading .wavs and midis is different depending on what you're working with. DirectSound is ALMOST as simple as saying "play this file," and DirectPlay is just as easy(you have to work with some com objects and such). I haven't worked with OpenAL, but I hear it's not much different.

  8. #8
    Registered User
    Join Date
    Apr 2002
    Posts
    1,571
    If you're using Win32 API , I suggest you look up mapping files to memory functions.

    This will get you started.

    1) CreateFile
    2) CreateFileMapping
    3) MapViewOfFile

    4) Perform operations

    5) UnmapViewOfFile
    6) CloseHandle [mapping]
    7) CloseHandle [file]

    This will let you treat your file as a pointer to a large chunk of memory. You can pass around the pointer wherever you want. This is probably a good route to take. If you have questions with anything specific let me know, I use this in my projects at work all the time.
    "...the results are undefined, and we all know what "undefined" means: it means it works during development, it works during testing, and it blows up in your most important customers' faces." --Scott Meyers

  9. #9
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856
    That is an enormous help. It looks like that will allow me to do exactly what I've been thinking about.

    DirectMusic8 provides the capability to load sound files in memory and I just successfully ran a little test that actually worked. I'm on the right track and lovin it.

    Thanks a lot Mr Wizard.

  10. #10
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856
    I've got one question for you, MrWizard. In MapViewOfFile(), I am attempting to set the low-order DWORD to the offset of the logical file within the physical file, but it fails. GetLastError() says "The parameter is incorrect." What I did was insert 10 bytes at the start of the file then try to map the file changing the last two args:
    Code:
    m_pMappedFile = MapViewOfFile(
          m_hFileMapping,       // handle to the file mapping object
          FILE_MAP_READ,        // desired access to the map
          0,                    // high order DWORD offset where mapping begins
          10,                    // low order DWORD offset where mapping begins
          m_dwFileSize-10                     // number of bytes of the file to map
        );
    I am clearly misunderstanding something about how file mapping works. Do you have any ideas?

    The other alternative that does work, but may be less desireable (but then again may not be), is to just map the entire file into view then use the pointer to read the header info and such and then just move the pointer to the start of each file accordingly.

    [edit]
    After thinking about it, I don't really see a need for what I'm asking here. All of my resources will be in a single file, so mapping the entire thing makes complete sense.
    [/edit]
    Last edited by LuckY; 08-13-2004 at 11:12 AM.

  11. #11
    Yes, my avatar is stolen anonytmouse's Avatar
    Join Date
    Dec 2002
    Posts
    2,544
    Your use of MapViewOfFile is failing because the offset parameter must be a multiple of the system allocation size. See this blog post for a MapViewOfFile sample.

    There is a function to map a complete file in this post.

    You could also consider using windows .RC resource files, with FindResource, LoadResource, etc.

  12. #12
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856
    Thanks for the resources.

    Now, one final question (with any luck)... I am having nothing but success loading audio with the mapped file (using DirectX, btw) but nothing but trouble trying to load bitmaps. I have traversed google for several hours and tried fiddling around with things myself with no luck. I am perhaps mistakenly attempting to load the bitmap with CreateDIBSection(), with no success and cannot find with my own efforts an alternative method to perform the task. Any ideas, tips, or hints?

  13. #13
    Registered User
    Join Date
    Apr 2002
    Posts
    1,571
    It shouldn't be a problem so long as your parameters to CreateDIBSection are correct. I'd really have to see some code to make sure. Perhaps also I'd even have to see the values in memory from your file. There shouldn't be a problem though.
    "...the results are undefined, and we all know what "undefined" means: it means it works during development, it works during testing, and it blows up in your most important customers' faces." --Scott Meyers

  14. #14
    Magically delicious LuckY's Avatar
    Join Date
    Oct 2001
    Posts
    856
    Ok, so I am right to use CreateDIBSection(). What I'm doing, in brief, is (for just a plain bitmap file for testing) use the pointer from MapViewOfFile() and copy sizeof(BITMAPINFO) bytes into a BITMAPINFO struct.
    Code:
        HDC dc = CreateCompatibleDC(NULL);
        m_hBmp = CreateDIBSection(dc, &bmi, DIB_PAL_COLORS, NULL, m_hFileMapping, sizeof(BITMAPINFO));
    It returns NULL but GetLastError() returns success (zero). Does this seem like I'm using it properly?

  15. #15
    Registered User
    Join Date
    Apr 2002
    Posts
    1,571
    You want DIB_PAL_COLORS ? I'll assume yes.

    Well I'm not sure exactly what all your values are. However, I'll quote MSDN here with regards to the 'offset' parameter (last param) of CreateDIBSection:

    "If hSection is not NULL, the CreateDIBSection function locates the bitmap's bit values at offset dwOffset in the file-mapping object referred to by hSection. An application can later retrieve the hSection handle by calling the GetObject function with the HBITMAP returned by CreateDIBSection. "

    You specify sizeof(BITMAPINFO) which may not yield the correct results from the beginning of the mapping.

    Does that make sense?
    "...the results are undefined, and we all know what "undefined" means: it means it works during development, it works during testing, and it blows up in your most important customers' faces." --Scott Meyers

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Problems passing a file pointer to functions
    By smitchell in forum C Programming
    Replies: 4
    Last Post: 09-30-2008, 02:29 PM
  2. sequential file program
    By needhelpbad in forum C Programming
    Replies: 80
    Last Post: 06-08-2008, 01:04 PM
  3. gcc link external library
    By spank in forum C Programming
    Replies: 6
    Last Post: 08-08-2007, 03:44 PM
  4. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 06:39 AM
  5. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 04:10 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21