Large Binary File Copy

This is a discussion on Large Binary File Copy within the C++ Programming forums, part of the General Programming Boards category; I am setting up a dll that automatically creates a date/time stamped copy of my Database Backup before I overwrite ...

  1. #1
    Registered User
    Join Date
    Jan 2003
    Posts
    4

    Question Large Binary File Copy

    I am setting up a dll that automatically creates a date/time stamped copy of my Database Backup before I overwrite it with a new backup. As the DB backup has got bigger, the copy process has started to take over my processor for the 10-minutes it takes to do the copy (backup is now in the 2Gb size).

    Is there a way I can change the code below to run as a 'background' process, or a way to restrict the amount of PC resource it can grab?

    I am currently using a very simple:
    Code:
    do
    	{
    	dwRead = sourceFile.Read(buffer, blocksize);
    
    	destFile.Write(buffer, dwRead);
    }
    while (dwRead > 0);

  2. #2
    Code Monkey Davros's Avatar
    Join Date
    Jun 2002
    Posts
    812
    You could try copying the file in contiguous, but seperate, blocks with a small sleep between each block copy. I would recommend a block size of around 32KB, but you may need to experiment with this.

    Your whole copy process will then take considerably longer than 10 mins, but it shouldn't hammer your CPU as much.

  3. #3
    Registered User
    Join Date
    Jan 2003
    Posts
    4

    Thanks

    Thanks Davos. I originally had a sleep but thought that was 'lazy' coding - will relook at it now.

    Is there any good reason for using a particular block size? You suggest 32k. Is there a reason for that size (or is it just experience)?

  4. #4
    Code Monkey Davros's Avatar
    Join Date
    Jun 2002
    Posts
    812
    If you are reading the contents of a file into memory in order to write it to another file, using a block copy is advantageous because you do not load the whole contents into memory at once. If this is happening with your 2GB file copy, your OS will start using virtual memory (and dumping things to swap file itself).

    Your OS will be implementing its own read-ahead block operations. So it could be beneficial to chose your block size to match that used by the OS. Normally this is around 64KB on most operating systems. But I would experiment a bit.
    OS: Windows XP
    Compilers: MinGW (Code::Blocks), BCB 5

    BigAngryDog.com

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Dikumud
    By maxorator in forum C++ Programming
    Replies: 1
    Last Post: 10-01-2005, 07:39 AM
  2. Binary Search Trees Part III
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 10-02-2004, 04:00 PM
  3. Request for comments
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 01-02-2004, 10:33 AM
  4. copying binary file
    By samc2004 in forum C Programming
    Replies: 5
    Last Post: 12-09-2003, 01:34 PM
  5. what does this mean to you?
    By pkananen in forum C++ Programming
    Replies: 8
    Last Post: 02-04-2002, 03:58 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21