Thread: Disk I/O

  1. #1
    Registered User
    Join Date
    Jan 2007
    Posts
    16

    Disk I/O

    I would like to create tree structures to store the data on disk. I wonder if I can use files to store them. Is there any way that I can access the disk block/sectors? so that, I can maintain my own file system for my database.

    Finally, what is the best data structure to store the data? (in terms of performance,reliability and implementation ease).

    Any help in this respect would be a lot useful.

    Thanks in advance.

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    1. Unless you have the disk to yourself - no. Writing to a disk at the sector level on a disk which already has a filesystem is a sure way to screw things up.

    2. Unless you have admin rights - no. Many operating systems forbid such low level access to the hardware. It is restricted to drivers and properly installed file systems.

    > Finally, what is the best data structure to store the data?
    I thought you said you had a tree.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Jan 2007
    Posts
    16
    Thanks a lot for your info, Salem!

    Is there a way to write our own file system to manage the data?

    Thanks in advance.

  4. #4
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    Sure, just grab as much driver and file system development as you can for your OS.
    Then settle down to 6 months of reading, then 6 months of failed attempts and system recovery.

    Writing a reliable file system is hard work, so you need a pretty good reason to want to invent your own.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  5. #5
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,318
    Most importantly, what is the data that you want to store?

    You don't want to modify the filesystem, you probably want to create you own one within a file, much like the way zip files contain the information about the files within them. Databases are another example of highly structured files.

  6. #6
    Registered User
    Join Date
    Jan 2007
    Posts
    16
    Thanks for your info Salem & iMalc !

    I would like to store the trees containing the data (customer records) on the disk, so that I can retrieve them when ever I need with better performance than files. Could you please suggest me a solution without using any of the Databases like oracle, mysql.

    Thanks in advance.

  7. #7
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    A whole bunch of f-ing functions. fopen, maybe fread, paired with fwrite, toss in a few fseeks, finish up with an fclose. Now there are a whole bunch of other f-ing functions I didn't mention, but I'll let you read up on them.


    Quzah.
    Hope is the first step on the road to disappointment.

  8. #8
    Registered User
    Join Date
    Jan 2007
    Posts
    16
    Quote Originally Posted by quzah
    A whole bunch of f-ing functions. fopen, maybe fread, paired with fwrite, toss in a few fseeks, finish up with an fclose. Now there are a whole bunch of other f-ing functions I didn't mention, but I'll let you read up on them.


    Quzah.


    Could you please explain me how do these f-ing functions help access the disk,
    especially the sectors or data blocks. In other words, how can I store the tree elements in the file which will allow me to read/write/update/search the tree elements in the similar fashion that we access the tree elements in RAM? I should be able to perform all the tree operations on the trees stored on the disk. (All the trees are binary search trees)

    Thanks in advance

  9. #9
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    I could. I'm not going to though. I haven't seen even a hint of effort from you in any of your posts.


    Quzah.
    Hope is the first step on the road to disappointment.

  10. #10
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    > how can I store the tree elements in the file which will allow me to read/write/update/search
    > the tree elements in the similar fashion that we access the tree elements in RAM?
    By turning the pointers you use for jumping round in memory into offsets you can use with fseek().

    Well it's one way anyway....
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  11. #11
    {Jaxom,Imriel,Liam}'s Dad Kennedy's Avatar
    Join Date
    Aug 2006
    Location
    Alabama
    Posts
    1,065
    Quote Originally Posted by jayfriend
    Could you please explain me how do these f-ing functions help access the disk,
    especially the sectors or data blocks. In other words, how can I store the tree elements in the file which will allow me to read/write/update/search the tree elements in the similar fashion that we access the tree elements in RAM? I should be able to perform all the tree operations on the trees stored on the disk. (All the trees are binary search trees)

    Thanks in advance
    So, you think that you know how to access the disk faster than a hardened filesystem does, yet you don't even know the basics of C? Very interesting. If you are unhappy with your current filesystem, why not try something else (Reiser, Second Extended, Third Extended, etc).

    More than likely, your problem isn't the speed of the disk, but the speed of the logic.

  12. #12
    Registered User
    Join Date
    Jan 2007
    Posts
    16
    Quote Originally Posted by Kennedy
    So, you think that you know how to access the disk faster than a hardened filesystem does, yet you don't even know the basics of C? Very interesting. If you are unhappy with your current filesystem, why not try something else (Reiser, Second Extended, Third Extended, etc).

    More than likely, your problem isn't the speed of the disk, but the speed of the logic.
    Probably my English is bad or I could not communicate well.
    Yes! I agree, I don't even know the basics of C, sorry for troubling you guys.

    I have a little knowledge about file operations and linked list.
    I heard that file operations are slower than tree operations (binary search trees are faster).
    I really don't know how to store trees in the files and retrieve them with minimum performance impact. As Salem suggested, fseek can be used instead of pointers to access the data on disk,
    But I don't know how to handle the situation if the file size grows. In other words, Is there any way to load the part of a file in to RAM instead of entire file?

    Thanks in advance.

    If you feel that I have not put in my efforts or I am not competent to ask this question, you may please ignore this message.
    Last edited by jayfriend; 01-18-2007 at 01:21 AM.

  13. #13
    {Jaxom,Imriel,Liam}'s Dad Kennedy's Avatar
    Join Date
    Aug 2006
    Location
    Alabama
    Posts
    1,065
    Keep your file sorted. That will at least reduce your search time. As long as you use structured data and that structure doesn't change size, you won't be too bad off. File IO isn't that bad. Most database rely heavily on IO with only a minimal portion of the data kept in memory. You won't run into much trouble with disk IO until you get upwards of 0.5 GB. Then you'll need to think of a way to split your data, but keeping the ties.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. why page based I/O can improve performance?
    By George2 in forum C Programming
    Replies: 1
    Last Post: 06-12-2006, 07:42 AM
  2. 2 questions surrounding an I/O file
    By Guti14 in forum C Programming
    Replies: 2
    Last Post: 08-30-2004, 11:21 PM
  3. Formatting Output
    By Aakash Datt in forum C++ Programming
    Replies: 2
    Last Post: 05-16-2003, 08:20 PM
  4. Towers of Hanoi, special output.
    By spoon_ in forum C Programming
    Replies: 3
    Last Post: 03-15-2003, 06:08 PM
  5. Overlapped I/O and Completion Port :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 0
    Last Post: 10-30-2002, 05:14 PM