Thread: Large Data Files in FlatFile Database...

  1. 05-03-2010 #1
    feeder74
    feeder74 is offline
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10

    Smile Large Data Files in FlatFile Database...

    I got another question open on these forums, but I'm already looking ahead at knowledge I am lacking...

    I've looked at <iostream> and fseek and fread/fwrite are great for what I want to do, but I've got a problem with this, I'm expecting my data files to extend way beyond the 2/4 Gig mark, maybe up to hundreds of Gig.

    The default variable used by fseek/fread/fwrite is a long, so how do I seek a position that is outside of it's scope?

    This is a windows 64 bit OS using Visual C++ Professional 2008.

    Thankyou.
  2. 05-04-2010 #2
    Dino
    Dino is offline
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Chappell Hill, Texas
    Posts
    2,332
    You will be limited by your operating system and hardware architecture for the size of the files. If your OP SYS supports hundreds of gigs for a file size, and your hardware does too - what's the problem? Otherwise, you'll have to come up with a solution that works around your OP SYS and hardware restrictions... perhaps multiple files?
    Mainframe assembler programmer by trade. C coder when I can.
  3. 05-04-2010 #3
    feeder74
    feeder74 is offline
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10
    Quote Originally Posted by feeder74 View Post
    The default variable used by fseek/fread/fwrite is a long
    The largest value that long can be is just shy of 2 gig, if I have a 15 gig file, what will fseek return? will the long overflow? if so where does this overflow data end up?
    Quote Originally Posted by Dino View Post
    You will be limited by your operating system and hardware architecture for the size of the files.
    I am using Windows 7 64-Bit, and it is the target for this application, my HDD is formated in NTFS, I had assumed I could pretty much go to town with file size, the HardDrive in Question is a 1 TB hardrive, roughly 931 Gig, please correct me if I am wrong with this...
    Last edited by feeder74; 05-04-2010 at 09:19 AM.
  4. 05-04-2010 #4
    tabstop
    tabstop is offline
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,336
    You should then use the Windows really-big-file functions, like SetFilePointerEx.
  5. 05-04-2010 #5
    feeder74
    feeder74 is offline
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10
    Ok, and thx, I guess thats the answer I was after...
« Previous Thread | Next Thread »
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. parent in a binary search tree
    By roaan in forum C Programming
    Replies: 4
    Last Post: 08-26-2009, 07:08 PM
  2. Best interprocess communication method for transferring large amounts of data?
    By mntgoat in forum C Programming
    Replies: 1
    Last Post: 05-05-2009, 04:53 PM
  3. Warnings, warnings, warnings?
    By spentdome in forum C Programming
    Replies: 25
    Last Post: 05-27-2002, 06:49 PM
  4. gcc problem
    By bjdea1 in forum Linux Programming
    Replies: 13
    Last Post: 04-29-2002, 06:51 PM
  5. Write and use data stored in data files...
    By alex6852 in forum C++ Programming
    Replies: 2
    Last Post: 11-01-2001, 01:45 PM