Thread: OPEN VERY VERY big text file inC?

  1. #1
    Registered User
    Join Date
    Feb 2008

    Question OPEN VERY VERY big text file inC?

    Hi all, first of all, sorry for my english!

    i'm newbie for C in Unix enviroment.

    i have a big structed-text file (about 500 MB in file size). I wanna to open this file but don't know about it's struct.

    Help me:

    1. What's best source code (IN C for Unix) to open this file?
    2. many thanks for your sharing

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Farncombe, Surrey, England
    I'd use standard fopen() - assuming you have no specific requirements about dealing with the file that REQUIRES some other method.

    500MB isn't THAT large a file, so it should be no genuine problems dealing with it.

    Obviously, on a machine with limited memory, you can't necessarily load the entire file into memory [and of course, if the file contains different sections, you may not need to do that either].

    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    {Jaxom,Imriel,Liam}'s Dad Kennedy's Avatar
    Join Date
    Aug 2006
    If you get to the 2.0GB range, then you'll need to start ensuring that you are using the correct pointer types. There are >2.0GB support functions, but they REQUIRE you to use the 64-bit indexes.

  4. #4
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    The reason for this is that C's standard FILE* functions use, as a minimum, signed 32-bit integers (longs) to store the position in the file. The maximum value you can represent in this scheme, of course, is 2^31-1 = 2,147,483,647, or exactly 2 gigabytes.

    Functions that use 64-bit positions can handle absolutely huge files. You probably won't be running into the limit for them for a few decades yet. But they aren't as standard, of course.

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell

    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ:

    My website:
    Projects: codeform, xuni, atlantis, nort, etc.

  5. #5
    Registered User
    Join Date
    Feb 2008
    many thanks!!!!!!!

    everything's OK for me!!!!!!!

  6. #6
    Officially An Architect brewbuck's Avatar
    Join Date
    Mar 2007
    Portland, OR
    500 MB? That's not even 2 gigabytes. You're nowhere close to "very very big..."

  7. #7
    Crazy Fool Perspective's Avatar
    Join Date
    Jan 2003
    awww, and i thought this thread was going to be about processing data that doesn't fit into memory. You got my hopes up.

    For reference, here's the internationally accepted standard of english pronouns for file sizes.
    100's of MB = Large
    GB's = Quite Large
    10's of GB = Big
    100's of GB = Very Big
    TB's = very very big
    PB's = Huge

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Formatting a text file...
    By dagorsul in forum C Programming
    Replies: 12
    Last Post: 05-02-2008, 03:53 AM
  2. How to use FTP?
    By maxorator in forum C++ Programming
    Replies: 8
    Last Post: 11-04-2005, 03:17 PM
  3. Encryption program
    By zeiffelz in forum C Programming
    Replies: 1
    Last Post: 06-15-2005, 03:39 AM
  4. Function is called and I am trying to open a file
    By tommy69 in forum C Programming
    Replies: 88
    Last Post: 05-06-2004, 08:33 AM
  5. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 04:10 PM