Subtle(?) File I/O Problem

Hi All,

I've got another sanity-questioning problem to pose for you. I am trying to read the contents of a file, but I'm only able to read a small fraction of the file (15232 / 2912998 Bytes). gfc->mfbuf_p is a file pointer that was just recently writing to this same file. To be safe, I close this file and reopen it for reading only with a new file pointer. The code is given below.

Code:

---

void func(...) {

        FILE *fp;
        int sz;
        struct flags *gfc;
        sample_t mfbuf[2][MFSIZE];  // MFSIZE is 3984

        ...

        // Open the temporary file for storing sampled data...
        gfc->mfbuf_p = fopen(TEMP_FILE, "w+");
        if(gfc->mfbuf_p == NULL) {
                printf("Could not open temporary file!\n");
                exit(1);
        }
       
       
        ...
       
        // Fill up a large mfbuf
        sz = fwrite((void*) mfbuf[0], sizeof(sample_t), mf_needed, gfc->mfbuf_p);
        sz += fwrite((void*) mfbuf[1], sizeof(sample_t), mf_needed, gfc->mfbuf_p);

        if (sz != 2*mf_needed) {
                printf("FWRITE failed!  Only %d bytes were written, not %d bytes", sz, 2*mf_needed);
                exit(1);
        }
       
        ...
       
        // Read saved samples from file and encode them...
        //        Had some problems reading.. let's close and reopen
        if(fflush(gfc->mfbuf_p) != 0) {
                perror("FFLUSH failed!");
                exit(1);
        }

        if (fclose(gfc->mfbuf_p) != 0) {
                perror("Closing failed!");
                exit(1);
        }

        fp = fopen(TEMP_FILE, "r");

        if (fp == NULL) {
                perror("Unable to open tmp file for reading!\n");
                exit(1);
        }

        while (!feof(fp)) {
               
                int mf_needed, sz;
                long off;


                off = ftell(fp);
                printf("\n\nOFfset: %ld\n", off);

                mf_needed = 1904;

                sz = fread((void*) &mfbuf[0][0], sizeof(sample_t), mf_needed, fp);

                off = ftell(fp);
                printf("\n\nOFfset: %ld\n", off);

                if(sz != mf_needed  && !feof(fp)) {
                        perror("\n\nFAILED to read from temp file ONE!\n\n");
                        exit(1);
                }
               
                sz += fread((void*) &mfbuf[1][0], sizeof(sample_t), mf_needed, fp);

                if(sz != 2*mf_needed  && !feof(fp)) {
                        perror("\n\nFAILED to read from temp file!\n\n");
                        exit(1);
                }
        }
       
}

---

This seems to work the first iteration, but fails the second iteration. It says it reads 1377 Bytes (out of the 1904 it should have read), and errno indicates an invalid argument. As I step through, I can see that the file pointer itself still points to a valid FILE structure, though after the failing read, I suspect some of its internal pointers may be corrupt (but I have no way of knowing).

Any ideas?

You closed gfc->mfbuf_p and then open a file with fp, but you're still testing gfc->mfbuf_p. Either you've been working on this code for too long and need a break, or you have serious issues with the language. I suspect it's the former. Take a break. ;)

And btw, don't use feof() to control a loop.

Thanks for the quick reply! Unfortunately, that is just a typo. Please substitute gfc->mfbuf_p for fp at that point. The problem still remains.

On a side note, I am compiling with Visual Studio's compiler (not by choice). Also, simply reading the file in the same manner in a sanity program is able to read the entire file.

[edit] And yes, I have been looking at this code much too long [/edit]

Silly question: How do mf_needed and MFSIZE compare?

You're resetting sz each iteration..... are you sure you mean to do that?

There are too many unknowns since you didn't post all the code, therefore, too many assumptions have to be made on my (our) part. Can you condense in into a complete, albeit small, failing example?

http://faq.cprogramming.com/cgi-bin/...&id=1043284351
Re MacGyver's reply earlier.

Dino: I haven't been able to reproduce it in a smaller example yet. I updated my original post with all the code relating to file I/O. In a sample program, I wrote about 1 MB to disk and then read it back in without problems.

tabstop: MFSIZE is 3984, and mf_needed is always 1904. I hardcoded this just for fun and still get the same results.

MacGyver: Yes, I mean to reset sz each iteration. All I wanted to check with sz was that I read the number of bytes I was expecting.

Salem/MacGyver: That little article about the "folly" of feof() was interesting, but it seems like there's absolutely no problem as long as you keep checking return values. For instance, fread will return the number of bytes read: if it's not what you expect, check to see if you are at EOF (fread will have tried to read past the end of file, so it should be set, if not then next time you will read 0 bytes and notice EOF). Did I miss something?

In general, it seems that something is causing these file I/O functions to think it has reached EOF, when all physical indications seem to suggest otherwise.

Thanks again for your input!

True, however yours was a rather exceptional post which DOES actually check the return results of say fread() and do the right thing.

> sz = fwrite((void*) mfbuf[0], sizeof(sample_t), mf_needed, gfc->mfbuf_p);
Am I missing something here, or is mf_needed uninitialised?

If you use "rb" and "wb", then does the size of the file equal sizeof(mfbuf)?

Also, why do you write mfbuf[0] then mfbuf[1] separately, when you could just write the whole array in a single call?

[edit]I thought I had a test case exposing the bug, but I was mistaken... [/edit]

Salem: mf_needed is hardcoded to 1904 at the moment. There's no particular reason to do mfbuf[0] and [1] separately, other than I need that 2-D structure, and I'm not sure about the continuity in memory of 2-D arrays. Sorry, but what do you mean by "rb" and "wb". The example I posted is rather trivial, and does not store the entire file that is read. Eventually mfbuf will be consumed in some manner and repopulated with new data.