Thread: spliting large files

  1. #1
    Registered User stormbringer's Avatar
    Join Date
    Jul 2002

    spliting large files


    i am writing a programm that splits large text files (2gb+). what i am doing for now is that i reads and copy the file to a new one till i found a special marker in the text and then close the outputfile and open a new one and so on. something like:
    while(~EOF) {
    while(marker not found) {

    Problem, while performing on the file, i need twice the size of the file. isn't there a way to get the position where to split and the simply dividing a large file in two without copying?


  2. #2
    Confused Magos's Avatar
    Join Date
    Sep 2001
    Only if you read the whole file into memory, then remove the file, then save it in parts. This is kinda risky though. Your program could crash in the middle of the process, completely removing the file. Plus, having several gigs of RAM is pretty rare.

    Give a man a fish and you feed him for a day.
    Teach a man to fish and you feed him for a lifetime.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. added start menu crashes game
    By avgprogamerjoe in forum Game Programming
    Replies: 6
    Last Post: 08-29-2007, 01:30 PM
  2. fopen vs. _open (for BIG image files)
    By reversaflex in forum C Programming
    Replies: 3
    Last Post: 04-01-2007, 12:52 AM
  3. Platform-independent large files?
    By Cat in forum C++ Programming
    Replies: 2
    Last Post: 08-21-2006, 12:04 AM
  4. Saving large files faster than light!
    By ZapoTex in forum C Programming
    Replies: 19
    Last Post: 01-09-2005, 05:47 PM
  5. Replies: 0
    Last Post: 07-12-2002, 01:40 PM