editing large file

This is a discussion on editing large file within the Windows Programming forums, part of the Platform Specific Boards category; Hi I'd like to edit a large file (~1GB) such that I append a small block of bytes at the ...

  1. #1
    Registered User
    Join Date
    Sep 2007
    Posts
    2

    editing large file

    Hi

    I'd like to edit a large file (~1GB) such that I append a small block of bytes at the beginning of the file. The only way I know of doing that is to create a new file, write the new bytes and then the large file into the stream.

    Under Windows, is there a better solution?

    best regards
    Sebastian

  2. #2
    Technical Lead QuantumPete's Avatar
    Join Date
    Aug 2007
    Location
    London, UK
    Posts
    894
    What are you prepending to a 1GB file? A jump instruction for a virus routine? ;-)
    I'm afraid afaik, you will either need to read in the entire file and write it out again or write your bytes to a new file, then copy byte-by-byte the old file. Then move new to old. Presto.
    Also, what language are you using?

    QuantumPete
    "No-one else has reported this problem, you're either crazy or a liar" - Dogbert Technical Support
    "Have you tried turning it off and on again?" - The IT Crowd

  3. #3
    Registered User Codeplug's Avatar
    Join Date
    Mar 2003
    Posts
    4,671
    The only other way is to do it "in-place" using CreateFileMapping and MapViewOfFile. However, this method is complicated due to the size of the file.

    When doing this the "traditional" way as you describe, you may gain a little performance by using FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_NO_BUFFERING and performing your reads and writes in multiples of the sectors size. See CreateFile API for more info.

    gg

  4. #4
    Registered User
    Join Date
    Sep 2007
    Posts
    2
    Hi

    I'm editing a medical image and want to edit header information. Unfortunately, the header can vary in length but the actual picture data remains the same.

    I'll give it a try with the CreateFile and Caching modes for sequential access.

    Thanks again

    Sebastian

  5. #5
    Unregistered User Yarin's Avatar
    Join Date
    Jul 2007
    Posts
    1,628
    >> What are you prepending to a 1GB file? A jump instruction for a virus routine? ;-)
    Don't be giving people any ideas.
    A class that doesn't overload all operators just isn't finished yet. -- SmugCeePlusPlusWeenie
    A year spent in artificial intelligence is enough to make one believe in God. -- Alan J. Perlis

  6. #6
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,558
    > I'm editing a medical image and want to edit header information. Unfortunately,
    > the header can vary in length but the actual picture data remains the same.
    So like you're adding some kind of comment information (like the patient name) to the image?

    Do you have any control over how the file gets created in the first place?
    If you do, you might want to 'pre-pad' the header information when the file is first created with a useful maximum size. Then all you have to do (most of the time) is just overwrite the header with your new data and the rest of the file remains unchanged.

    Only in rare cases (when you need to add more than your max guess) would you need to recopy the entire file.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  7. #7
    Super Moderator VirtualAce's Avatar
    Join Date
    Aug 2001
    Posts
    9,596
    I'm editing a medical image and want to edit header information. Unfortunately, the header can vary in length but the actual picture data remains the same.
    If the header varies in length then it should have a data member that indicates its size. If not how would any other program know where the image data started since the header length could vary from file to file? It will either have a member that indicates the header size or an offset into the file where the image data begins.

  8. #8
    Registered Abuser
    Join Date
    Jun 2006
    Location
    Toronto
    Posts
    591
    A third possibility is that the header could be relying on the presence of some byte pattern that marks the end of the header/start of the image data. In which case you'd (or any program) would have to scan through the file first, searching for this pattern. Probably not the best of header designs, but it is a possibility.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. File transfer- the file sometimes not full transferred
    By shu_fei86 in forum C# Programming
    Replies: 13
    Last Post: 03-13-2009, 12:44 PM
  2. Game Pointer Trouble?
    By Drahcir in forum C Programming
    Replies: 8
    Last Post: 02-04-2006, 01:53 AM
  3. Unknown Memory Leak in Init() Function
    By CodeHacker in forum Windows Programming
    Replies: 3
    Last Post: 07-09-2004, 09:54 AM
  4. Hmm....help me take a look at this: File Encryptor
    By heljy in forum C Programming
    Replies: 3
    Last Post: 03-23-2002, 09:57 AM
  5. Need a suggestion on a school project..
    By Screwz Luse in forum C Programming
    Replies: 5
    Last Post: 11-27-2001, 01:58 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21