Thread: Very slow file writing of 'fwrite' function in C

  1. #1
    Registered User
    Join Date
    Jul 2006
    Posts
    2

    Angry Very slow file writing of 'fwrite' function in C

    First of all, I am not sure if this problem comes from the OS or Hardware or even my program.
    Anyway, when I excute a simple C-code which is doing some processing and writes the results
    into the hard disc, the program takes to much time to finish up the writing. The file size is about 450MB. I am using 'fwrite' function( Pretty sure, in a correct way) which is normally
    working well in otherwhere but in our new computer ( workstation which has 4 CPUs and each
    is dure core), it shows really slow file writing. Sometimes, it takes a couple of minitues
    to finish up the writing. Sometimes, when I slightly change my code (Definiely, other parts
    which is not related with this writing module), it is fast ( done within a few secs).

    Any one has same problem ? or Any idea will be reall appreciated.

  2. #2
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,897
    What's your memory setup like? Can you describe the writing algorithm?
    My best code is written with the delete key.

  3. #3
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,218
    You're probably just running into OS file buffer weirdness. There's no way the harddisk is fast enough to write 450MB in a few seconds.
    If you understand what you're doing, you're not learning anything.

  4. #4
    Registered User
    Join Date
    Jul 2006
    Posts
    2
    It's very simple. I'm just reserving a space using 'malloc' function as follows,
    Sorry, I learned how to use code tag. So I changed my previous and also
    fixed a bug which was a typo.

    Code:
    float *proj;
    
    if((proj = (float *)malloc(proj_size*sizeof(float)))==NULL)
    {
          printf("Memory allocation failed!\n");
          exit(1);
    }
    
    fp = fopen("test.dat","wb");
    fwrite(proj,sizeof(float),proj_size,fp);
    fclose(fp);
    That's pretty much it for my code....
    And , yes it will not be finished withing a few secs but it doesn't take
    more than 10 secs and even it should not take a couple of minutes.

    Anyway thanks for replies.
    Last edited by scho; 08-03-2006 at 02:48 PM.

  5. #5
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,218
    There's a bug in your code anyway...
    Code:
    if((proj = (float *)malloc(proj_size*sizeof(proj)))==NULL)
    Look at what type proj is. A pointer to a float is not necessarily the same size as a float. Also, you don't need to cast the return value of malloc() in C.

    And next time please use code tags.
    If you understand what you're doing, you're not learning anything.

  6. #6
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,897
    >It's very simple.
    Not quite. You say that the call to fwrite is slow, but you also call malloc, fopen, and fclose. malloc is notoriously slow even under ideal conditions, which don't include memory fragmentation and other aspects of the memory manager that would cause a lot of extra work to be done. What value does proj_size contain? How much RAM does this machine have?

    I think it's not your code, but the fact that you're hitting degenerate behavior in several areas that creates the intermittent slowness.
    My best code is written with the delete key.

  7. #7
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    There seems to be an endemic talent for posting without code tags at the moment.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Beginner Needs help in Dev-C++
    By Korrupt Lawz in forum C++ Programming
    Replies: 20
    Last Post: 09-28-2010, 01:17 AM
  2. Data Structure Eror
    By prominababy in forum C Programming
    Replies: 3
    Last Post: 01-06-2009, 09:35 AM
  3. Trouble writing to file using fwrite()
    By yougene in forum C Programming
    Replies: 4
    Last Post: 12-30-2008, 05:13 PM
  4. Replies: 28
    Last Post: 07-16-2006, 11:35 PM
  5. Interface Question
    By smog890 in forum C Programming
    Replies: 11
    Last Post: 06-03-2002, 05:06 PM