Thread: behead/truncate large file

  1. #1
    Registered User
    Join Date
    Mar 2008
    Posts
    82

    behead/truncate large file

    Hi,

    I have a pretty big wav file and want cut off a small bit of the front, or a small bit of the back. To get and idea of size says its 2MB front or back bits from a 1GB file.

    So there are tools available, such as sox for this. In fact there's a number of ways .. but most involved creating a new large file (less the bits taken off): transfering to disk and overwriting the original file with the new truncated file. Which is all very slow.

    Seeing as the majority of the data is not changing at all, I was wondering if it wasn't possible to leave the majority of the file as is, and recreate the boundaries (front or back) of the file in some way, to make it all much quicker.

    I was lookign for suggestions. Right now I'm looking at the possibillities of mmap, and then I'll look at any apis that the ext2 filesystem may offer. It's not so important about it being a wav, even doing it to a text file is fairly interesting.

    Many thanks in advance !

  2. #2
    Registered User
    Join Date
    May 2009
    Posts
    4,183
    Is it doable, I would guess yes.
    Is it wise to do, I would guess no.

    If you make, a mistake you could destroy all the data on the hard drive.

    Because, the only way to not have to re-write the file will be editing the file structure info on the hard drive.
    I would start by learning as much about the ext2 file system; maybe it has a simple and safe way to do it.

    In NTFS, I believe it would be too dangerous for the slight gain in speed.
    But, I only know how the MSDOS file structure was done. Never looked at the logical disk geometry for NTFS or ext2.

    Tim S.
    "...a computer is a stupid machine with the ability to do incredibly smart things, while computer programmers are smart people with the ability to do incredibly stupid things. They are,in short, a perfect match.." Bill Bryson

  3. #3
    Registered User
    Join Date
    May 2003
    Posts
    1,619
    Even file system manipulation would only help for the trim the beginning case where the new beginning happened at an allocation unit boundary. Files cannot begin at arbitrary positions on the disk (assuming, of course, you don't want to write your own file system to handle this...)
    You ever try a pink golf ball, Wally? Why, the wind shear on a pink ball alone can take the head clean off a 90 pound midget at 300 yards.

  4. #4
    Registered User
    Join Date
    Mar 2008
    Posts
    82
    Many thanks stahta01 and Cat for your suggestions. I agree with both of you this "file boundary modification" seems a mad idea now.

    Equally, I also think that - in general terms - is there is a growing trend towards bigger and bigger files. I got to thinking that perhaps the way is not to modify the file at all, but merely to access it differently, using offsets and such, fseek() and friends. A good example could be Youtube, some of the files now going up are huge, so to draw people attention to a certain part, you can submit a comment prefixed with the time point of the section and this will render as a link when the comment is published.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Truncate an array?
    By acidblue in forum C Programming
    Replies: 4
    Last Post: 03-18-2011, 05:18 PM
  2. [Large file][Value too large for defined data type]
    By salsan in forum Linux Programming
    Replies: 11
    Last Post: 02-05-2008, 04:18 AM
  3. truncate help
    By munna_dude in forum Linux Programming
    Replies: 3
    Last Post: 10-18-2007, 06:04 AM
  4. how can i truncate this
    By munna_dude in forum C Programming
    Replies: 11
    Last Post: 04-10-2007, 11:35 AM
  5. truncate
    By hoodiemama in forum C++ Programming
    Replies: 4
    Last Post: 11-02-2002, 06:37 PM