OPEN VERY VERY big text file inC?

This is a discussion on OPEN VERY VERY big text file inC? within the Linux Programming forums, part of the Platform Specific Boards category; Hi all, first of all, sorry for my english! i'm newbie for C in Unix enviroment. i have a big ...

  1. #1
    Registered User
    Join Date
    Feb 2008
    Posts
    2

    Question OPEN VERY VERY big text file inC?

    Hi all, first of all, sorry for my english!

    i'm newbie for C in Unix enviroment.

    i have a big structed-text file (about 500 MB in file size). I wanna to open this file but don't know about it's struct.

    Help me:

    1. What's best source code (IN C for Unix) to open this file?
    2. many thanks for your sharing

  2. #2
    Kernel hacker
    Join Date
    Jul 2007
    Location
    Farncombe, Surrey, England
    Posts
    15,677
    I'd use standard fopen() - assuming you have no specific requirements about dealing with the file that REQUIRES some other method.

    500MB isn't THAT large a file, so it should be no genuine problems dealing with it.

    Obviously, on a machine with limited memory, you can't necessarily load the entire file into memory [and of course, if the file contains different sections, you may not need to do that either].

    --
    Mats
    Compilers can produce warnings - make the compiler programmers happy: Use them!
    Please don't PM me for help - and no, I don't do help over instant messengers.

  3. #3
    {Jaxom,Imriel,Liam}'s Dad Kennedy's Avatar
    Join Date
    Aug 2006
    Location
    Alabama
    Posts
    1,065
    If you get to the 2.0GB range, then you'll need to start ensuring that you are using the correct pointer types. There are >2.0GB support functions, but they REQUIRE you to use the 64-bit indexes.

  4. #4
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,048
    The reason for this is that C's standard FILE* functions use, as a minimum, signed 32-bit integers (longs) to store the position in the file. The maximum value you can represent in this scheme, of course, is 2^31-1 = 2,147,483,647, or exactly 2 gigabytes.

    Functions that use 64-bit positions can handle absolutely huge files. You probably won't be running into the limit for them for a few decades yet. But they aren't as standard, of course.
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  5. #5
    Registered User
    Join Date
    Feb 2008
    Posts
    2
    many thanks!!!!!!!

    everything's OK for me!!!!!!!

  6. #6
    Captain Crash brewbuck's Avatar
    Join Date
    Mar 2007
    Location
    Portland, OR
    Posts
    7,274
    500 MB? That's not even 2 gigabytes. You're nowhere close to "very very big..."

  7. #7
    Crazy Fool Perspective's Avatar
    Join Date
    Jan 2003
    Location
    Canada
    Posts
    2,640
    awww, and i thought this thread was going to be about processing data that doesn't fit into memory. You got my hopes up.

    For reference, here's the internationally accepted standard of english pronouns for file sizes.
    100's of MB = Large
    GB's = Quite Large
    10's of GB = Big
    100's of GB = Very Big
    TB's = very very big
    PB's = Huge

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Formatting a text file...
    By dagorsul in forum C Programming
    Replies: 12
    Last Post: 05-02-2008, 04:53 AM
  2. How to use FTP?
    By maxorator in forum C++ Programming
    Replies: 8
    Last Post: 11-04-2005, 03:17 PM
  3. Encryption program
    By zeiffelz in forum C Programming
    Replies: 1
    Last Post: 06-15-2005, 04:39 AM
  4. Function is called and I am trying to open a file
    By tommy69 in forum C Programming
    Replies: 88
    Last Post: 05-06-2004, 09:33 AM
  5. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 05:10 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21