fclose slow

Hello,

I've made a file synchronization application that copies files from one machine to another across the network. For this I use the standard fopen, fread, fwrite and fclose calls (see code below). The copying itself works a treat but when I call fclose (specially for big files) it takes a long time to close the file (even longer than it took to copy!). I've disabled buffering with a setbuf call but this does not seem to have any effect. I even throw in a fflush call during the copy process. It's really annoying - the user sees a progress bar when copying the file which runs fine up to 100% and then hangs for a long time while it's closing the file. Is there anything I can do to make the fclose return quicker?
I'm copying from a Linux PC to a Windows PC across my home network.
Also, other sync programs that I have do not seem to have this problem.

I'm using GNU C on a Linux PC (Xubuntu)
IDE = Anjuta + GTK+ and Libglade

Ruud

==========================================

Code:

---

gboolean copy ( gchar *src_name,
                gchar *dst_name,
                int size )
{
  gchar buf [ BUFSIZ ];
  FILE *src;
  FILE *dst;
  int r,w;
  int done = 0;
  struct stat src_sb;
  struct utimbuf file_times;
  gboolean rtn = TRUE;

  set_progress ( done, size );
  if ( stat ( src_name, &src_sb ) < 0 )
  {
    g_warning ( "Could not stat %s\n", src_name );
    return FALSE;
  }
  if ( ( src = fopen ( src_name, "rb" ) ) == NULL )
  {
    g_warning ( "Error opening %s", src_name );
    return FALSE;
  }
  setbuf ( src, NULL );
  if ( ( dst = fopen ( dst_name, "wb" ) ) == NULL ) // consider using tmpfile
  {
    g_warning ( "Error opening %s", dst_name );
    rtn = FALSE;
  }
  else
  {
    setbuf ( dst, NULL );
    rtn = in_progress ();
    while ( rtn && ( r = fread ( &buf, sizeof ( gchar ), BUFSIZ, src ) ) > 0 )
    {
      rtn = in_progress ();
      if ( rtn )
      {
        w = fwrite ( &buf, sizeof ( gchar ), r, dst );
        fflush ( dst );
        done = done + w;
        set_progress ( done, size );
        if ( w != r )
        {
          g_warning ( "Error writing to %s", dst_name );
          rtn = FALSE;
        }
      }
      else
        g_warning ( "Aborted...." );
    }
    fclose ( dst );
  }
  fclose ( src );
  if ( rtn )
  {
    file_times.actime = src_sb.st_atime;
    file_times.modtime = src_sb.st_mtime;
    if ( utime ( dst_name, &file_times ) != 0 )
    {
      g_warning ( "Error preserving times for %s", dst_name );
      rtn = FALSE;
    }
  }
  return rtn;
}

---

That is likely because the fclose causes the data to be ACTUALLY written to disk - up until the fclose, the OS just keeps it all in RAM, but once you close the file, it says "Ok, that's it, lets store this into the file to get rid of it".

Using the low-level function open() with O_DIRECT will make each write happen as it is issued, rather than being delayed. I'm far from sure this will ACTUALLY HELP, but it may do. You then have to either use open, read, write and close instead for the fopen, etc, or use fdopen() to make a FILE pointer from a file-descriptor from open().

--
Mats

You must think me so very rude.
I never got a notification for your reply (or have but forgot about it).
Anyway my apologies and a (very) belated thank you.
Your suggestion worked perfectly :-)

You have to set your "Default Thread Subscription Mode" under User CP->edit options to receive email notification.

Kind of interesting setbuf() and fflush() did not work. Altho setbuf() does not actually turn buffering off, for that you need setvbuf(). setbuf() sets the buffer to a default value (eg, on my normative linux system it is 8kb).

Thanks for the Default Thread Subscription Mode tip.
It's set to e-mail notifications now (I would have thought that would be the default).

Thanks also for the tip on setvbuf. however. I've reverted back to simple low-level functions open, close, read and write. This gives me more control and is more transparent.
I open the source file with src = open ( src_name, O_RDONLY | O_SYNC )
and the destination file with dst = open ( dst_name, O_WRONLY | O_CREAT | O_SYNC )
O_DIRECT did not work for me.