Streamed-Level Loading

This is a discussion on Streamed-Level Loading within the Game Programming forums, part of the General Programming Boards category; Hello, this is more of an optimized way to do something question. What I am trying to look into is ...

  1. #1
    Registered User
    Join Date
    Sep 2004
    Posts
    27

    Streamed-Level Loading

    Hello, this is more of an optimized way to do something question.

    What I am trying to look into is streamed loading. The project in question is a side scroller, so the implementation shouldn't have to be too complex. I am working on two ideas, and leaning strongly towards one of them, but have enough doubt to get some outside opinions.

    The levels will be tile-based, and for optimization we only want part of the level in memory. I was thinking of establishing blocks of each level, where the blocks loaded will be the current block the player is in, and the possible connecting blocks that the player could possibly move into next.

    My question to the community as a whole is this:

    Would it be a better idea to have one file hold each block, or have one file per level, with subdivisions detailing each block?

    I see advantages and disadvantages of both ways. Any opinions are greatly appreciated.

  2. #2
    Slave MadCow257's Avatar
    Join Date
    Jan 2005
    Posts
    735
    I don't think this optimization is even neccasary (it might infact make it less optimized). You could load every single level into memory without too much concern, although I would do it level by level.

  3. #3
    Registered User
    Join Date
    Sep 2004
    Posts
    27
    We are aiming our project towards a few goals...

    Low memory usage (Useful when designing games for actual consoles).

    Continuous gameplay, one level starts where the last ended.

    It is mainly experimental, and concepts used may be of use in future projects.

  4. #4
    Slave MadCow257's Avatar
    Join Date
    Jan 2005
    Posts
    735
    I still think you could get away with loading all the levels in at the start and still meet the low memory usage goal. If you efficiently made the level storage scheme all the levels would probably be less then a megabyte which is very very low memory. A single bitmap might be larger then that. Also, it would take a little bit less of the processor to do it this way.

  5. #5
    Registered User
    Join Date
    Aug 2004
    Location
    San Diego, CA
    Posts
    313
    Why not have something like a 4-way linked list? I'll show you:

    ----N----
    W--X--E
    ----S----

    The player is in X. You have your nodes hold pointers to each other node (pNorth, pSouth, pEast, pWest), and each node holds the data that the next block of the level needs.

    This could work.. but I'm not 100% sure of what data you're dealing with.

  6. #6
    Registered User
    Join Date
    Sep 2004
    Posts
    27
    The project is still in the conceptual stages, and we aren't certain on whether we shall use it or not, but it is something we are looking into more for the sheer purpose of doing it moreso than necessity. We want to learn some new techniques, and have had plenty of experience of doing only what we need to. But doing things that way really hurts the ability to learn. This isn't really meant to be a commercial project.

    Please take the scale of the project away from the question. Regardless of actually using it, it is still something I have been interested in checking into for my own knowledge. If anyone has links or information as well, I would like to do better research. I searched gamedev and google, and can't find much on implementation theories.

    As for the linked list, I was planning rather similar to that once I have the actual data. My primary question was rather I should keep all the data in one file, and jump around to find what I need, or to keep the data in multiple files for each section.

    Keeping track of read points in a single file would be challenging, but it would save disk space.

    Multiple files would take more disk space, but it would be easier to keep track of what I am reading into the program.

    As forementioned, we may not even use it in the project, but I am seeking this advice for my own knowledge. I have tried looking into directories of the games I have that I know use similar loading styles, but the files are packed in formats I cannot extract to see the file structure. As I said before, I have had trouble locating any information on the topic, but if any of you have links or even recommended books that cover the topic, I would be very appreciative.
    Last edited by Lifedragn; 03-01-2005 at 07:53 PM.

  7. #7
    Registered User
    Join Date
    Aug 2004
    Location
    San Diego, CA
    Posts
    313
    Oh. Why not keep seperate .dat files for each section?

    CastleA-RoomA.dat
    CastleA-RoomB.dat
    CastleA...

    Would that work?

  8. #8
    Registered User
    Join Date
    Sep 2004
    Posts
    197
    seperate files loaded and unloaded could work, and then you could package them up (similar to tar or zip)
    If any part of my post is incorrect, please correct me.

    This post is not guarantied to be correct, and is not to be taken as a matter of fact, but of opinion or a guess, unless otherwise noted.

  9. #9
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,596
    Since most tile maps break down into something like this:


    10,20,12,23,45,67,1,2,8,10,15,15,15,15,15,0,0,0,0, 0,..................

    Streamed level loading is an absolute waste of time. Unless you are writing a major terrain engine or some type of flight simulator that cannot possibly keep all the data in memory at once, then a streamed loader is what you need. But most games do NOT load on the fly.
    Face it, the hard disk even as fast as it is today is still a slow crawl compared to a memory access. There are two things you don't want to do in any game.

    1. You don't want the program swapping to the hard drive. Major lag here and major frame rate issues.
    2. You don't want to load on the fly. Go out and purchase some flight sims. Even the pro's cannot load in new terrain and object data w/o some type of frame rate hit or major lag in multiplayer. It just cannot be done.

    Here is a tile map class. This demonstrates how loading in a tile map actually lends itself more towards array based file I/O than character based streamed I/O. I wouldn't use streams at all - the C standard library will take care of this quite nicely for you.

    Code:
    #ifndef CTILEMAP
    #define CTILEMAP
    
    
    #include <string>
    #include <sys/stat.h>
    #include <sys/types.h>
    #include <fcntl.h>
    #include <io.h>
    
    struct CTilemapHeader
    {
      char  Signature[2];
      char  Author[40];
      char  Date[10];
      char  Time[5];
      WORD  Width;
      WORD  Height;
      DWORD Size;
    };
    
    class CTilemap
    {
      CTilemapHeader      m_pMapInfo;
      WORD                *m_pMapData;
    
      public:
        CTilemap(void):m_pMapData(NULL) {}
        virtual ~CTilemap(void) {if (m_pMapData) delete [] m_pMapData;};
    
        bool Load(std::string File)
        {
          //Open file
          int handle=_open(File.c_str(),_O_RDONLY | _O_BINARY,_S_IREAD);
    
          //Check for valid handle
          if (handle!=-1)
          {
            read (handle,&m_pMapInfo,sizeof(CTilemapHeader));
            
            //Check for signature in file, bail on failure
            if (m_pMapInfo.ZeldaSig[0]!='Z' && m_pMapInfo.ZeldaSig[1]!='A')
            {
              close(handle);
              return false;   //Not a zelda map
            } 
    
            //Allocate memory
            m_pMapData=new WORD[m_pMapInfo.Size];
    
            //Make sure pointer is valid
            if (m_pMapData)
            {
              
              read(handle,(WORD *)m_pMapData,m_pMapInfo.Size);
          
            } else return false;        //pointer is not valid
          
          } else return false;          //File handle is not valid
          
          return true;   //Map loaded successfully
        }
    
        WORD GetData(WORD column,WORD row)
        {
          DWORD offset=(row*m_pMapInfo.Width)+column;
          if (offset<m_pMapInfo.Size)
          {
    
            return m_pMapData[offset];
          } else return (0xFFFF);
        }
    
        WORD GetData(DWORD offset)
        {
          if (offset<m_pMapInfo.Size)
          {
            return m_pMapData[offset];
          } else return (0xFFFF);
        }
    
        WORD *GetMap(void) 
        {
          if (m_pMapData) 
          {
            return m_pMapData;
          } else return (NULL);
        }
    
    
    };
    
    
    #endif
    That's really all you need for any scroller.

    When you begin you are at offset 0 into the tilemap. So the side scroller render function will draw the tile at offset 0, and then increment the offset and the x position...draw the next, etc.

    When you reach the right side of the screen, you need to increment by the total width of the tilemap or the famous offset=(row*width)+column. If you increment your offset by (width) then you will maintain the correct positioning in the array.

    This is all done in linear fashion and is a simple matter of incrementing values. No need to re-compute offsets or do expensive multiplies mid-function.

    Tile system:

    0. Start at top-left of screen.
    1. Get current tile map (array)
    2. Iterate through tile map.
    3. Pull value from tile map, access texture based on this number
    4. Render Texture
    5. Increment offset, x, and if necessary y.
    6. Goto 2 until we have reached right side bottom of screen.

    For scrolling, this is a simple matter of changing the on-screen grid. Increment the starting offset when (x % tilewidth) or (y % tilewidth) is 0.

  10. #10
    Confused Magos's Avatar
    Join Date
    Sep 2001
    Location
    Sweden
    Posts
    3,145
    I've had seamless transitions in mind for a while, but never had time to actually implement it. My basic idea (inspired from other games, including Metroid Prime for GC) is that when you're approaching a new (unloaded) area you start a new thread that takes care of the level loading while still allowing you to play on, removing (or at least reducing) lag. Of course you have to prevent the player form entering the new area until the level has loaded, but as in MP this is done by keeping the doors locked until loading is complete.
    MagosX.com

    Give a man a fish and you feed him for a day.
    Teach a man to fish and you feed him for a lifetime.

  11. #11
    Registered User
    Join Date
    Sep 2004
    Posts
    27
    As forementioned, the question is a topic removed from the scope of the project. I didn't ask if it was worthwhile, I asked for an opinion on file storage. Please discontinue telling me it is a waste of time, as I know it isn't needed on a scroller, but it may be of great use on larger projects and am using a small-scale project to test and debug methods of accomplishing the task.

    Keeping separate files is the way I was leaning. Thanks everyone, I will try that way first and see how it goes.

  12. #12
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,596
    Please discontinue telling me it is a waste of time...
    Ok, then why spend time coding something that is?

    Have it your way.

  13. #13
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Loading speed, with something that loads the entire level at once anyway, is a moot point. Disk access on a few megs is nothing. It's not worth the effort to focus optimizing. Optimizing as a whole, is done after the project is done anyway. You don't waste time optimizing stuff that isn't even finished. You get the project done, and then you worry about optimization.

    Who really cares if it takes a few more seconds to load a level anyway? The user expects that. Either that, or you simply use the same set of sprites for the whole thing, load them all into memory, and simply load a level map, which is nearly instantaneous anyway, so as stated, it is a waste of time.

    You don't optimize until the end. Then you run a profiler, and optimize. Optimizing the way things load isn't worth the effort unless you have some massive amount of data. You don't, so as stated, it's not worth the time.

    When you ask a question, expect to get at least some truthful answers. You don't have to like the answer. Sometimes truth hurts. These are all things you need to learn. Apparently you prefer to learn the hard way. Well, it's your time. Waste it however you like.

    Quzah.
    Hope is the first step on the road to disappointment.

  14. #14
    Confused Magos's Avatar
    Join Date
    Sep 2001
    Location
    Sweden
    Posts
    3,145
    You don't waste time optimizing stuff that isn't even finished
    I disagree. Why spend time implementing it one way, then spend even more time redesigning it afterwards? It's better to do it the right way from the start.

    And I don't think loading times is something to neglect. It's quite annoying in games, Halflife 2 as an example, that suddenly the game halts and starts loading the next session. Seamless loadings is possible, Metroid Prime 2 as a grand example.
    MagosX.com

    Give a man a fish and you feed him for a day.
    Teach a man to fish and you feed him for a lifetime.

  15. #15
    S Sang-drax's Avatar
    Join Date
    May 2002
    Location
    Göteborg, Sweden
    Posts
    2,072
    Quote Originally Posted by Magos
    And I don't think loading times is something to neglect. It's quite annoying in games, Halflife 2 as an example, that suddenly the game halts and starts loading the next session. Seamless loadings is possible, Metroid Prime 2 as a grand example.
    I agree with you. It's nice to play a game with no loading times and I think that if you want to eliminate the loading times you have to have that in mind early.

    Another example is Baldur's Gate II vs. Diablo II. Baldurs Gate has extremly long loading times very often, even if you do a full install, whereas Diablo II has like 5 seconds in total throughout the whole game.
    Last edited by Sang-drax : Tomorrow at 02:21 AM. Reason: Time travelling

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Cargo loading system
    By kyle rull in forum C Programming
    Replies: 1
    Last Post: 04-20-2009, 12:16 PM
  2. added start menu crashes game
    By avgprogamerjoe in forum Game Programming
    Replies: 6
    Last Post: 08-29-2007, 01:30 PM
  3. Level creation, loading, management etc
    By Ariste in forum C++ Programming
    Replies: 1
    Last Post: 08-02-2005, 11:09 PM
  4. Loading a new level
    By harry_p in forum Game Programming
    Replies: 1
    Last Post: 07-15-2002, 12:37 PM
  5. Loading Screen
    By pinkcheese in forum Windows Programming
    Replies: 2
    Last Post: 04-06-2002, 10:48 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21