Database Creation

Hi,

I am trying to get some info on creating a database system using C, C++. Not something that has been implemented but actually create the database.

Anybody have any info on it ?

I suspect a lot of us have had this goal before. Unfortunately, it's not as easy as I at first thought, for a lot of reasons. From what I can tell, it boils down to how much experience do you have, what tools do you have to use, and what type of data do you want to store in the database.

Arrays of primitive data types like int, double, etc. are probably the simplest form of data base. Arrays of structs/classes are probably the next most complicated. Arrays of pointers can be quite interesting. Vectors of structs/classes or vectors of pointers can create quite flexible and create a reasonably complex database. Hash tables and bin storage may well be useful somewhere. File reads and writes for large databases will probably be a requirement. Graphical user interfaces to allow easy interaction with the user is always helpful. In the "How to..." books I looked at they indicated not to bother reinventing the wheel if you want a complex database. Rather, they suggested hooking into a commercial database, get/store the data you want, and formatting a report from there if needed. If you still want to create your own, some of the GUI IDEs (I'm thinking of BCB in particular since that is what I was using when I looked into this type of project) might have built in database table classes you can use. If you aren't familiar with all the concepts listed here, that's where I would start.

Bottom line, this is probably a long term goal, unless you are already quite proficient in C++, in which case I'd be interested in learning what you find out.

Hi,

What exactly do you mean by bin storage, the rest I understand, but I am still trying to find a way to do File read/write better in C++. I find it a bit slow while searches.

One more question, do you know how commercial database vendors actually store data on disk, I mean how do they scramble the data. I cannot imagine them storing it as a text file.

all files are stored as binary on the hard drive. When you view the file it is usually presented to you in a format which is easier to read, maybe hex or text or whatever but the hard drive doesn't store text as such, just binary.

My understanding of bins is that they are containers within a container. Say you have a list of words stored in a database. You want to find the second word starting with the letter s. You could search all words in the database or you could separate the database into bins, where each bin is a container that holds all words in the database starting with a given letter, so one bin holds all words starting with a, one for words starting with b, etc. Then you just search the bins for the s bin and only search the words in that bin. A bin could be a vector in a vector of vectors or a list in a list of lists or a list in a vector of lists or a vector in a list of vectors or whatever.

As to how commercial vendors organize data storage on the harddrive, I imagine it depends/varies from vendor to vendor, as does everything else in C++/computers, although I really don't know. The way I envision it would be to have a master file that keeps track of subfiles (bins) with additional subfiles(bins) as necessary to keep file searching/search times to a minimum, since no matter what you do, searching hard drive files is always going to be slower than searching RAM. They probably speed up the process by keeping recently used data in RAM somewhere, suggesting you use as much RAM as you can, etc. They may have some "think ahead" functions to try to retrieve data before the problem actually needs it, etc. All interesting to speculate about, but I don't know how you would go about actually figuring it out.

Maybe Oracle or Sybase or whoever has made their technology open to the public, but I wouldn't count on it.