Large Data Files in FlatFile Database...

This is a discussion on Large Data Files in FlatFile Database... within the C++ Programming forums, part of the General Programming Boards category; I got another question open on these forums, but I'm already looking ahead at knowledge I am lacking... I've looked ...

  1. #1
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10

    Smile Large Data Files in FlatFile Database...

    I got another question open on these forums, but I'm already looking ahead at knowledge I am lacking...

    I've looked at <iostream> and fseek and fread/fwrite are great for what I want to do, but I've got a problem with this, I'm expecting my data files to extend way beyond the 2/4 Gig mark, maybe up to hundreds of Gig.

    The default variable used by fseek/fread/fwrite is a long, so how do I seek a position that is outside of it's scope?

    This is a windows 64 bit OS using Visual C++ Professional 2008.

    Thankyou.

  2. #2
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    You will be limited by your operating system and hardware architecture for the size of the files. If your OP SYS supports hundreds of gigs for a file size, and your hardware does too - what's the problem? Otherwise, you'll have to come up with a solution that works around your OP SYS and hardware restrictions... perhaps multiple files?
    Mac and Windows cross platform programmer. Ruby lover.

    Quote of the Day
    12/20: Mario F.:I never was, am not, and never will be, one to shut up in the face of something I think is fundamentally wrong.

    Amen brother!

  3. #3
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10
    Quote Originally Posted by feeder74 View Post
    The default variable used by fseek/fread/fwrite is a long
    The largest value that long can be is just shy of 2 gig, if I have a 15 gig file, what will fseek return? will the long overflow? if so where does this overflow data end up?
    Quote Originally Posted by Dino View Post
    You will be limited by your operating system and hardware architecture for the size of the files.
    I am using Windows 7 64-Bit, and it is the target for this application, my HDD is formated in NTFS, I had assumed I could pretty much go to town with file size, the HardDrive in Question is a 1 TB hardrive, roughly 931 Gig, please correct me if I am wrong with this...
    Last edited by feeder74; 05-04-2010 at 10:19 AM.

  4. #4
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    You should then use the Windows really-big-file functions, like SetFilePointerEx.

  5. #5
    Registered User
    Join Date
    May 2010
    Location
    Australia
    Posts
    10
    Ok, and thx, I guess thats the answer I was after...

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. parent in a binary search tree
    By roaan in forum C Programming
    Replies: 4
    Last Post: 08-26-2009, 08:08 PM
  2. Replies: 1
    Last Post: 05-05-2009, 05:53 PM
  3. Warnings, warnings, warnings?
    By spentdome in forum C Programming
    Replies: 25
    Last Post: 05-27-2002, 07:49 PM
  4. gcc problem
    By bjdea1 in forum Linux Programming
    Replies: 13
    Last Post: 04-29-2002, 07:51 PM
  5. Write and use data stored in data files...
    By alex6852 in forum C++ Programming
    Replies: 2
    Last Post: 11-01-2001, 01:45 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21