Thread: Save File Organization Question

I've recently started to play with the idea of coding for more than just a passing hobby and one of the first problems that I ran into was what to do with save files. I understand how to write information to disk, but where I get lost is on how that information is typically organized and identified for retrieval later. Is every line given some sort of a tag so it can be searched and assigned to a variable when it's read after the program starts up? Does the program just read everything on line 1 and assign that to some variable?

I guess what I'm looking for here, is if there are any standardized ways to handle reading and writing from disk in an organized and efficient way. I did search the forums but most of what I was able to find was dealing with specific problems. I am looking for more general knowledge. If someone could point me to a link or give me an idea what to search I would be grateful.

Also, in case it helps, the file I am playing with would have to store a very large amount of different variables from many different objects. (Many thousands of objects)

Thanks for any help you can give.

A lot of games, and not a few businesses, use record (struct) based filing systems...
The basic idea is that you organize blocks of data as structs and then just write the struct itself to disk... memory to disk... disk to memory. No translation or intermediate steps needed. In larger storage systems files are made up of lots and lots of structs (millions in some cases) and you can access them very rapidly by using two common techinques "Binary Search" and "Random Access" where you know the size of the struct so if you want the 10,000th one you can go straight to it and read it in fasterthanthis... blink of an eye, really. When you don't know which struct you need Binary Search lets you find 1 of a thousand in only 10 tries, one of 2000 in only 11... even if it's the last record in the file... not quite blink of an eye but surpriszingly fast.

Do some googling on these techniques, it's a fastinating read.

Thanks for the info, this gives me a good place to start. I had an inkling that adding tags to every line was a terribly inefficient way to go about saving and loading stuff.

Originally Posted by Zafaron

Thanks for the info, this gives me a good place to start. I had an inkling that adding tags to every line was a terribly inefficient way to go about saving and loading stuff.

There are situations where it's the right way to do it...
For example... the Random Access method works perfectly when you have the exact same information in large numbers of structs (i.e. they're all the same) ... For example an inventory program tracking 100,000 item inventory.

However this system ain't worth crap if you can't standardize the records. When blocks are differing sizes or you have huge amounts of non-repeating data to store, Random Access ain't gonna do it for ya... That's when you break out the "Formatted Text File" method (look at any windows .ini file) and start reading sequentially.

It really depends on what you're storing, how standardized it is and how many repetitions you have... You may end up with a sequential file of 30,000 variables that are otherwise unrelated or a Random Access file of 10 identical structs that are clearly related...

It's about analysing your needs, doing the research and deciding the best course...
No professional would settle for less.

Originally Posted by phantomotap

even when a simple flat binary is used in the real world, a specific endianness, packing, alignment, size, and format (such as IEEE floating points or some other form), is expected. No two compilers, or even one compiler with different options used during compile time, will agree on all of those details.

I think I'm confused. Do companies often change compilers between printing copy number 50000 and 50001 of a production run?

Originally Posted by phantomotap

Boy do I get tired of hearing that garbage repeated.

And boy do I ever get tired of you repeatedly harping along about how it's somehow a bad thing.

It's done all the time!

Give it a rest, ok?

Fix it in memory, write it in one shot.

Oh, so all of this time I've been wrong telling people of those real world issues and how a given programmer would have to manually serialize data if it is to be in any specific form and how you two are flat out wrong to suggest that `fread' and `fwrite' with whatever chunk of data the compiler may produce will work even in the face of all those considerations, and now you all of a sudden you agree saying "Soma is wrong, but as it turns out a programmer will have to do some manual work after all."?

Priceless. You are really desperate to be right all of sudden.

What happened, did you try that "BMP" on an big-endian platform example I posted and find out that, sure enough, exactly as I said, and have been saying all along, the programmer would have manually serialize the data?

Eh, don't worry; I'm sure you'll find something else to be wrong about later.

Soma

You apparently missed the part from earlier where it was said that those real world problems weren't issues at all. Actually, it has been said several times that even noting that those issues exist is idiotic. So, no, that's not what people was trying to tell me.

In any event, you don't even need to change compilers to need to deal with those issues. Simply changing a compiler option is enough for some compilers.

Even then, if you ever work with an outside stream or an outside file format, you will have to deal with those issues.

Telling newbies to simply dump whatever binary chunk the compiler without at least noting those issues is flat out lying. Further, telling newbies that they aren't issues after having the issues explained to them is incredibly stupid.

Soma