Thread: map saving question

  1. #1
    Registered User
    Join Date
    Aug 2003
    Posts
    288

    map saving question

    im working on the loading/saving of maps for my 2d tile game and so far ive managed to get it working but the filesize for the map is around 70+ mb!

    its a 5000x5000 tile map (all towns/levels/dungeons/etc are stored on 1 layer), i have 3 bytes per tile (texture1, texture2, flags (blocked?, water?, and texture blend flags)).

    in the loading routine i calculate the coordinates for the tiles based on the position in the file, so thats another 3 floats (4 bytes per?) and then theres texture coordinates.. etc

    this all adds up to around 600 mb for me, and thats just for the loading of the map.

    does anybody know any better methods? i was thinking of adding compression but that would be the same thing, decompress and load the (large) file.

  2. #2
    Registered User
    Join Date
    Jan 2005
    Posts
    847
    Quote Originally Posted by X PaYnE X
    its a 5000x5000 tile map (all towns/levels/dungeons/etc are stored on 1 layer), i have 3 bytes per tile (texture1, texture2, flags (blocked?, water?, and texture blend flags)).
    This size map is always going to use a fair amount of space. Even if it's one byte per tile then 5000*5000 tiles is 25 MB!

    If you are using a water texture then it should be safe to assume that that tile should have the water flag set. Maybe you can just store a one byte index value for each tile which is used to point to an array of tile information, assuming most of your tiles are not totally different.
    You could also use run length encoding. If you have a huge stretch of water then instead of storing each tile you could store the number of tiles of that type.

  3. #3
    Registered User
    Join Date
    Aug 2003
    Posts
    288
    assuming i use RLE, i still have to load the map into memory. that would still result in 5000x5000 of data being used, unless im supposed to read from file everytime i render.

    Maybe you can just store a one byte index value for each tile which is used to point to an array of tile information, assuming most of your tiles are not totally different.
    that could work except most of the tiles are never the same. e.g. theres shallow and deep water, and shallow and deep lava, etc.. all 4 have water flag set (you 'sink' in) and the deep variations have blocked flag set.

    i was thinking of loading something like 100x100 around the current position of the user, but that would lead to loading as the user travels the map. also, it is fast paced so the user could get from 0x0 to 5000x5000 instantly (portals and stuff like that). loading times would be annoying.

    im curious as to how other 2d/3d tile based games deal with this? do they load the entire map in 1 go to memory?

  4. #4
    Registered User VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,607
    A tile map is just tile ID's. Those ID's could be indexes into a vector of tile structures that are created at load time. The maps will still be large but it sets your resource cache up in a much better state. I wouldn't create a 5000x5000 map in any editor. Break it down. You will never see 5000x5000 tiles on the screen at one time so it's ludicrous to use that as one level. If that is your world then break it up somehow into smaller sections.

    You could do this at save-time by using a quad-tree algo on your map data. This would span your map data over several files, but it would alleviate the large file issue.

  5. #5
    Registered User
    Join Date
    Aug 2003
    Posts
    288
    the reason i put them all in 1 file is for easier loading/access, think of it this way:

    most maps are set up as levels, lets say the world/dungeon floor 1/dungeon floor 2.
    now instead of having 3 different maps, ill put the dungeon floors in the world map, lets say in the middle of the deep ocean or something (somewhere you cant get to normally). that way if you were to enter the dungeon it would merely change your current location to wherever the dungeon is, instead of loading a new map and all that.

    and 5000x5000 is just an example, atm its around 500x500 but im nowhere near done, im predicting it will be somewhere between 4000-5000 once the map is fully complete.

    splitting the map into several files would just create load time and it wouldnt really help with the large file issue because its still large.

  6. #6
    Slave MadCow257's Avatar
    Join Date
    Jan 2005
    Posts
    735
    My suggestion is to compress the file, then at load time decompress a window around the start position + some of the higher risk areas such as around portals. Then you would make a thread that loads from the file as needed by the main game thread. That way you'll save HD space, ram, and never see skips in the game play. Might be kind of hard to implement though..

  7. #7
    Registered User VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,607
    Since this is a tile map a simple RLE compression would work wonders.

    Your tile map is really just a large bitmap in memory. It's just that the numbers in memory now refer to a tile ID number rather than a pixel color. So compressing it using RLE would eliminate redundant tile ID's which you will have a lot of to create floors, pillars, etc.

  8. #8

    Join Date
    May 2005
    Posts
    1,042
    This concept of using ID's is crucial to being able to handle data gracefully (meaning without a horrendous memory footprint).

    You see, ultimately an ID is just an integer...an index into an array. The basic idea of using IDs (and really, any compression mecanism whether it be jpeg or .zip) is that you only store one version of each piece of data.

    I.e, in my texture manager, it just asks to load a filename, and it searches through the already loaded textures and if it hasn't already been loaded, only then does it allocate memory for it, otherwise if it still has that texture, it just returns an ID (an index into the array of textures) that points to the copy that already exists.

    Instead of having all of the data for a dungeon in every tile, you could assume that there are a few different types of dungeons, and each tile instead has an id that denotes what dungeon it points to...that way each tile only needs to have an integer, instead of all of the possible data required for a dungeon (which I'm guessing is just like a bitmap picture of a dungeon or something?)

    Bubba knows what I'm talking about so he will continue to elaborate more eloquently than I.
    I'm not immature, I'm refined in the opposite direction.

  9. #9
    Registered User
    Join Date
    Aug 2003
    Posts
    288
    so basically, i shud have 1 file ('master') that stores all the possible types of tile combinations? (lets say its a max of 255), then use 1 byte in the map file in reference to the 'master' file.

    then in the map file i should use some kind of encoding to decrease the size.. easy enuff, ill give that a shot and see how it goes.

    thanks for the help.

  10. #10
    Absent Minded Programmer
    Join Date
    May 2005
    Posts
    968
    I believe Ultima Online solved this problem by cutting their map up into the "server lines," everytime one would cross a server line (wasn't too often) about a half a second load time happened, which you really didn't notice too much.

    The Ultima Online mapset was easily 5000x5000 tiles if you threw all of it together, but I believe it was sent to the user in probably 100x100 chunks, dungeons, a bit larger than that, were probably sent in like 500x500 chunks... Seems like a good way to go to me..

    About cutting your filesize down - if half of your 5000x5000 tiles are grass tiles, (at least a significant amount), can't you just save one of those? And then have a file (a simple text file even) that just has a couple of numbers to work with, a 1, for grass, 2 for water, 3 for deep water, etc etc, And have a huuuuuge txt file with 5000x5000 numbers on it.. the text file would be much smaller than a file of the actual 5000x5000 tile definitions... Of course maybe I'm a newb at this issues (considering I've only been doing C++ for a year now), but thats my idea :\

    Hey cool it seems I had relatively the same idea as the pro's up there without reading their post, cool!
    Sometimes I forget what I am doing when I enter a room, actually, quite often.

  11. #11

    Join Date
    May 2005
    Posts
    1,042
    >>
    Hey cool it seems I had relatively the same idea as the pro's up there without reading their post, cool!

    If you're going to refer to use with such stature I prefer to be called 'Ninja'

    Bubba can be a pirate.
    I'm not immature, I'm refined in the opposite direction.

  12. #12
    Absent Minded Programmer
    Join Date
    May 2005
    Posts
    968
    alrighty, bob mcgee the jinja, bubba the pirate
    Sometimes I forget what I am doing when I enter a room, actually, quite often.

  13. #13
    Registered User VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,607
    Don't call me pro till I actually have a playable 'finished' game.

    Which might be a loooooooooooonnnnnnnnnnng time.

    Here is my map system. I'm not saying this is the only way to do it, but it works for me.

    Here is how you use it:

    Create a CMapManager.
    Call CMapManager::Init() and pass the height and width of the map.

    Call CMapManager::Add()

    This will add a map to the vector and it is cleared to 0xFFFFFFFF which denotes an empty tile. I figured no one would ever have (2^31) tiles in memory. Note that all subsequent maps have the same width and height. This is because in my application each CMap represents one layer of a tile map. We use layering to achieve effects that w/o this system would require thousands more tiles. But notice that the capability is in CMap and CMapManager to all be different sizes. So with some alteration CMapManager could contain many maps of different sizes.

    To save a map to disk just open a file and pass the handle of the file to SaveToHandle(ID,filenum) where ID is the index of the map in the vector you wish to save. If you want to save all maps then open a file and pass the handle to SaveAllToHandle(handle) and this will save the entire vector to disk. This is done internally by CMapManager. It calls CMap::SaveToHandle(handle) for every CMap in the vector. So you can see that even the saving portion is ready for maps of varying sizes. Each map is preceded in the file by a small header called MapHeader. If you wish to change the file format, feel free to do so.

    Loading maps is done through a similar process. Open a file and either call CMapManager::LoadFromHandle(ID,handle) or CMapManager::LoadAllFromHandle(handle). The maps are then auto-created by CMapManager and the data is read into them in the order they appear in the file.

    Calling or using any CMap function or member variable is not the correct way to use this code. I've made all of CMap private and CMapManager is a friend of the class. CMapManager's main functions or interfaces are public and so you should use them to do what you want. If functionality is not in CMapManager that you need to implement, you should implement it in CMapManager and provide supporting functions and/or directly manipulate CMap members and/or call it's functions.

    Cleanup as far as I know is bug-free. CMapManager will iterate through the vector and delete the items. This will call the CMap destructor which will clean up it's own memory footprint.

    There is a method to the madness but I won't go into detail here.
    Last edited by VirtualAce; 12-09-2005 at 01:45 AM.

  14. #14
    Absent Minded Programmer
    Join Date
    May 2005
    Posts
    968
    I'd make another post, but that seems to be overdoing it...

    What exactly is a DWORD? I see you use them to hold the map info

    Code:
      DWORD m_dwWidth;
      DWORD m_dwHeight;
      DWORD m_dwSize;
    Sometimes I forget what I am doing when I enter a room, actually, quite often.

  15. #15
    Registered User
    Join Date
    Jan 2005
    Posts
    847
    Quote Originally Posted by Shamino
    I'd make another post, but that seems to be overdoing it...

    What exactly is a DWORD? I see you use them to hold the map info

    Code:
      DWORD m_dwWidth;
      DWORD m_dwHeight;
      DWORD m_dwSize;
    a double word, 4 bytes, 32 bits...

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. another map question
    By m37h0d in forum C++ Programming
    Replies: 5
    Last Post: 01-26-2009, 10:25 AM
  2. Creating a map of objects with variable width and height
    By MrSparky in forum C++ Programming
    Replies: 6
    Last Post: 07-30-2007, 03:06 PM
  3. how can I re-sort a map
    By indigo0086 in forum C++ Programming
    Replies: 8
    Last Post: 06-01-2006, 06:21 AM
  4. Saving Map Array To Disk
    By Tommaso in forum Game Programming
    Replies: 4
    Last Post: 07-18-2003, 12:03 AM
  5. Keys of a map
    By nickname_changed in forum C++ Programming
    Replies: 4
    Last Post: 07-10-2003, 04:46 AM