spliting large files

This is a discussion on spliting large files within the C Programming forums, part of the General Programming Boards category; hi i am writing a programm that splits large text files (2gb+). what i am doing for now is that ...

  1. #1
    Registered User stormbringer's Avatar
    Join Date
    Jul 2002
    Posts
    90

    spliting large files

    hi

    i am writing a programm that splits large text files (2gb+). what i am doing for now is that i reads and copy the file to a new one till i found a special marker in the text and then close the outputfile and open a new one and so on. something like:
    [CODE]
    while(~EOF) {
    fopen(myfile.....);
    while(marker not found) {
    fputc(fgetc(source),newfile);
    }
    }

    Problem, while performing on the file, i need twice the size of the file. isn't there a way to get the position where to split and the simply dividing a large file in two without copying?

    thanks

  2. #2
    Confused Magos's Avatar
    Join Date
    Sep 2001
    Location
    Sweden
    Posts
    3,145
    Only if you read the whole file into memory, then remove the file, then save it in parts. This is kinda risky though. Your program could crash in the middle of the process, completely removing the file. Plus, having several gigs of RAM is pretty rare.
    MagosX.com

    Give a man a fish and you feed him for a day.
    Teach a man to fish and you feed him for a lifetime.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. added start menu crashes game
    By avgprogamerjoe in forum Game Programming
    Replies: 6
    Last Post: 08-29-2007, 01:30 PM
  2. fopen vs. _open (for BIG image files)
    By reversaflex in forum C Programming
    Replies: 3
    Last Post: 04-01-2007, 12:52 AM
  3. Platform-independent large files?
    By Cat in forum C++ Programming
    Replies: 2
    Last Post: 08-21-2006, 12:04 AM
  4. Saving large files faster than light!
    By ZapoTex in forum C Programming
    Replies: 19
    Last Post: 01-09-2005, 04:47 PM
  5. Replies: 0
    Last Post: 07-12-2002, 01:40 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21