Packed Files (Puting multiple files in one file)

Hello,

I'm attempting to create a virtual file system to be used for a data storage protocol for the proposes of making a compressed sot of Data Volt, from what I can understand the only real way of going by this is by packing multiple files into one single file.

i hade a very long hard look on Google but I couldn't I find and reference to how one would even start to great to make something like this, I figured that it may work by writing separators between the files and then write the file structure at the end of the file but I had a long hard think about it and worked out it wouldn't work because if some super brilliant person made a text file with the separator in it basically the whole virtual file system would break.


I was wondering if any of you brilliant fantastically talented people had any ideas how i could achieve this.

Note: I haven’t included any source code because this is only at the stages of being a theory.

You could write the first X bytes to be the size of the current file, for example:


<4 bytes, size of the file><file data><next 4 bytes of the next file><next file>

Quote:

---

Originally Posted by Mortissus

You could write the first X bytes to be the size of the current file, for example:


<4 bytes, size of the file><file data><next 4 bytes of the next file><next file>

---

Thanks for your reply, the problem with that is if a user created a text document (.TXT) and writes " " the program i think will mess up as its retrieved the separator where it shouldn’t.

i could be wrong.

I don't see what you mean about the separator. Maybe you could elaborate?

Edit: Use a different separator? Nothing simple that is would be easy for someone to recreate. You can try encoding the separator so no one knows what it looks like too.

[

Quote:

---

Originally Posted by cerin

I don't see what you mean about the separator. Maybe you could elaborate?

---

Yeah Sorry, by separator I mean I would need something in between the files so that when I go to unpack them as it were, the file which are packed in side that one file the files would have same names and data here is the sort of file structure I was thinking of using:

Code:

---

(Root Directory){
        (File1){
        Filename=”Blah.txt”
        FileSize=”10KB”
        FileWrites=”777”
        FileData=”Hello World”
}
        (File2){
        Filename=”Blah2.txt”
        FileSize=”10KB”
        FileWrites=”777”
        FileData=”Hello World2”
}
}

---

If you look at the two files I would defiantly need a way of separating the data otherwise the some one could hypothetically mess up the file system just by writing “) in a text file.

If one want a laugh I actually had to lookup up the word “elaborate” I couldn’t remember the definition!

I believe I need to reiterate my edit from the last post:

Quote:

---

Originally Posted by cerin

Use a different separator? Nothing simple that is would be easy for someone to recreate. You can try encoding the separator so no one knows what it looks like too.

---

Have it in simple notations and don't do it like that:
Try:

Code:

---

({<555><<33<>Warcraft3.exe>})
file data goes here
({<><<<>})  //this is end tag

---

It doesn't have to be a really systematic notation. But make it something unlikely to be repeated within the file.
You could even encode it and so it appears to be a bunch of random numbers and letters to the user.

ok that would work perfectly about 99.99&#37; of the time but it really isn't full proof if you know what I mean because someone could still do it completely by accident and mess-up the file system. which would be extremely bad I guess I could write a hammer code as well into the program that checks the file structure of the virtual file system. which it properly going to happen anyway but, can you think of a full proof way of doing it?

Again, Thank you for your help!

Note: I didn't see the above post when i was writing this but i still think this post is relevant regardless.

If you have an encoded notation 10 or more characters it becomes highly unlikely that that exact string will ever be repeated and if it is then it has to be the correct tag. Meaning you can't have 2 file start tags in a row before having an end tag or the second will be skipped. The thing is also that I doubt anyone would want to screw up their own files by adding tags. It wouldn't be extremely bad. You can just output an error saying "This file is corrupt."

Quote:

---

Originally Posted by cerin

If you have an encoded notation 10 or more characters it becomes highly unlikely that that exact string will ever be repeated and if it is then it has to be the correct tag. Meaning you can't have 2 file start tags in a row before having an end tag or the second will be skipped. The thing is also that I doubt anyone would want to screw up their own files by adding tags. It wouldn't be extremely bad. You can just output an error saying "This file is corrupt."

---

OK, thanks.

What I'm going to do now is I'm actually going to have a shoot at make the program and see how it goes i still think I'm going to want some major error checking code, but ill post in a few days to see what my results are!

Once you get that done maybe you can expand on it and add compression or encryption methods to it. Oh ya and sorry to bog you with edits. I just had a lot of after thoughts.

Quote:

---

Originally Posted by Mortissus

You could write the first X bytes to be the size of the current file, for example:


<4 bytes, size of the file><file data><next 4 bytes of the next file><next file>

---

What was wrong with this idea. You won't have any separators and nothing the user could put in thier file could screw it up.

If you are merely packing the files together you would concoct a class to represent your archiving files. This file would have a header telling how many files are contained therein, the size in bytes of the files, and the files in order from start to end. You wouldn't need a separator when you know the information of the file.

To pack the files you would get each of those file's sizes in bytes, and add it to the header via a loop, then copy the file whole into your filetype. to unpack you simply get the file information, loop to add the file sizes to an array, then for each iteration in the loop copy the 'i'th entry size of each file and send them to separate files.

You would also have to know how the files come out so you may need to add the type of file it is to the header, so that when you open the files for writing you know how to send them.

Hello,

I had a little play about and i haven't manged to get anything done, completely nothing!

i was wondering if anyone would be able to point me into the writing such a program, the only way i see of doing it is writing very large amounts of code to scan the files for the files and then extract them.

does anyone have an idea of an easier way?


MrKnights

Try having a look at this:

http://www.gamedev.net/reference/art...rticle1991.asp

Wow, Thanks man thanks for the link. It really gives me a better understanding on how to do this. i shall try and find some free time to check out these methods ill post back when i have news.

Two words:

ZIP file.

Quote:

---

Two words:

ZIP file.

---

Dang, you beat me to it. :) I second the aforementioned ZIP file. ZIPs can keep files names as well as folders, so you can archive in a hierarchical way. Plus, as icing, you get compression! (If you choose to use it.) There are already tons of libraries out there to work with zip, such as libzip and zziplib (although I'm not sure if zziplib does writing...).

Otherwise, if you want to implement it yourself for learning purposes, and thus need/want something simpler, there's TAR, and there ought to be some libraries out there for that too. Might not be simple, but you could boil off most of the details are create something similar.

Quote:

---

Originally Posted by Cactus_Hugger

Dang, you beat me to it. :) I second the aforementioned ZIP file. ZIPs can keep files names as well as folders, so you can archive in a hierarchical way. Plus, as icing, you get compression! (If you choose to use it.) There are already tons of libraries out there to work with zip, such as libzip and zziplib (although I'm not sure if zziplib does writing...).

Otherwise, if you want to implement it yourself for learning purposes, and thus need/want something simpler, there's TAR, and there ought to be some libraries out there for that too. Might not be simple, but you could boil off most of the details are create something similar.

---

i guess for now i'm going to try and build my own one as it's much more fun and a bigger head ack and the more you learn the better person you become or something like that ;)

anyways i'm gonna start with the basic without enciption and compression

i