Subtle(?) File I/O Problem

This is a discussion on Subtle(?) File I/O Problem within the C Programming forums, part of the General Programming Boards category; Hi All, I've got another sanity-questioning problem to pose for you. I am trying to read the contents of a ...

  1. #1
    Registered User
    Join Date
    Jul 2008
    Posts
    8

    Subtle(?) File I/O Problem

    Hi All,

    I've got another sanity-questioning problem to pose for you. I am trying to read the contents of a file, but I'm only able to read a small fraction of the file (15232 / 2912998 Bytes). gfc->mfbuf_p is a file pointer that was just recently writing to this same file. To be safe, I close this file and reopen it for reading only with a new file pointer. The code is given below.

    Code:
    void func(...) {
    
    	FILE *fp;
    	int sz;
    	struct flags *gfc;
    	sample_t mfbuf[2][MFSIZE];  // MFSIZE is 3984
    
    	...
    
    	// Open the temporary file for storing sampled data...
    	gfc->mfbuf_p = fopen(TEMP_FILE, "w+");
    	if(gfc->mfbuf_p == NULL) {
    		printf("Could not open temporary file!\n");
    		exit(1);
    	}
    	
    	
    	...
    	
    	// Fill up a large mfbuf
    	sz = fwrite((void*) mfbuf[0], sizeof(sample_t), mf_needed, gfc->mfbuf_p);
    	sz += fwrite((void*) mfbuf[1], sizeof(sample_t), mf_needed, gfc->mfbuf_p);
    
    	if (sz != 2*mf_needed) {
    		printf("FWRITE failed!  Only %d bytes were written, not %d bytes", sz, 2*mf_needed);
    		exit(1);
    	}
    	
    	...
    	
    	// Read saved samples from file and encode them...
    	//	Had some problems reading.. let's close and reopen
    	if(fflush(gfc->mfbuf_p) != 0) {
    		perror("FFLUSH failed!");
    		exit(1);
    	}
    
    	if (fclose(gfc->mfbuf_p) != 0) {
    		perror("Closing failed!");
    		exit(1);
    	}
    
    	fp = fopen(TEMP_FILE, "r");
    
    	if (fp == NULL) {
    		perror("Unable to open tmp file for reading!\n");
    		exit(1);
    	}
    
    	while (!feof(fp)) {
    		
    		int mf_needed, sz;
    		long off;
    
    
    		off = ftell(fp);
    		printf("\n\nOFfset: %ld\n", off);
    
    		mf_needed = 1904;
    
    		sz = fread((void*) &mfbuf[0][0], sizeof(sample_t), mf_needed, fp);
    
    		off = ftell(fp);
    		printf("\n\nOFfset: %ld\n", off);
    
    		if(sz != mf_needed  && !feof(fp)) {
    			perror("\n\nFAILED to read from temp file ONE!\n\n");
    			exit(1);
    		}
    		
    		sz += fread((void*) &mfbuf[1][0], sizeof(sample_t), mf_needed, fp);
    
    		if(sz != 2*mf_needed  && !feof(fp)) {
    			perror("\n\nFAILED to read from temp file!\n\n");
    			exit(1);
    		}
    	}
    	
    }

    This seems to work the first iteration, but fails the second iteration. It says it reads 1377 Bytes (out of the 1904 it should have read), and errno indicates an invalid argument. As I step through, I can see that the file pointer itself still points to a valid FILE structure, though after the failing read, I suspect some of its internal pointers may be corrupt (but I have no way of knowing).

    Any ideas?
    Last edited by cecomp64; 07-16-2008 at 10:55 AM. Reason: typo

  2. #2
    Deathray Engineer MacGyver's Avatar
    Join Date
    Mar 2007
    Posts
    3,211
    You closed gfc->mfbuf_p and then open a file with fp, but you're still testing gfc->mfbuf_p. Either you've been working on this code for too long and need a break, or you have serious issues with the language. I suspect it's the former. Take a break.

    And btw, don't use feof() to control a loop.

  3. #3
    Registered User
    Join Date
    Jul 2008
    Posts
    8
    Thanks for the quick reply! Unfortunately, that is just a typo. Please substitute gfc->mfbuf_p for fp at that point. The problem still remains.

    On a side note, I am compiling with Visual Studio's compiler (not by choice). Also, simply reading the file in the same manner in a sanity program is able to read the entire file.

    [edit] And yes, I have been looking at this code much too long [/edit]
    Last edited by cecomp64; 07-15-2008 at 07:02 PM. Reason: sanity

  4. #4
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Silly question: How do mf_needed and MFSIZE compare?

  5. #5
    Deathray Engineer MacGyver's Avatar
    Join Date
    Mar 2007
    Posts
    3,211
    You're resetting sz each iteration..... are you sure you mean to do that?

  6. #6
    Jack of many languages Dino's Avatar
    Join Date
    Nov 2007
    Location
    Katy, Texas
    Posts
    2,309
    There are too many unknowns since you didn't post all the code, therefore, too many assumptions have to be made on my (our) part. Can you condense in into a complete, albeit small, failing example?
    Mac and Windows cross platform programmer. Ruby lover.

    Quote of the Day
    12/20: Mario F.:I never was, am not, and never will be, one to shut up in the face of something I think is fundamentally wrong.

    Amen brother!

  7. #7
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,338
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  8. #8
    Registered User
    Join Date
    Jul 2008
    Posts
    8
    Dino: I haven't been able to reproduce it in a smaller example yet. I updated my original post with all the code relating to file I/O. In a sample program, I wrote about 1 MB to disk and then read it back in without problems.

    tabstop: MFSIZE is 3984, and mf_needed is always 1904. I hardcoded this just for fun and still get the same results.

    MacGyver: Yes, I mean to reset sz each iteration. All I wanted to check with sz was that I read the number of bytes I was expecting.

    Salem/MacGyver: That little article about the "folly" of feof() was interesting, but it seems like there's absolutely no problem as long as you keep checking return values. For instance, fread will return the number of bytes read: if it's not what you expect, check to see if you are at EOF (fread will have tried to read past the end of file, so it should be set, if not then next time you will read 0 bytes and notice EOF). Did I miss something?

    In general, it seems that something is causing these file I/O functions to think it has reached EOF, when all physical indications seem to suggest otherwise.

    Thanks again for your input!

  9. #9
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,338
    True, however yours was a rather exceptional post which DOES actually check the return results of say fread() and do the right thing.

    > sz = fwrite((void*) mfbuf[0], sizeof(sample_t), mf_needed, gfc->mfbuf_p);
    Am I missing something here, or is mf_needed uninitialised?

    If you use "rb" and "wb", then does the size of the file equal sizeof(mfbuf)?

    Also, why do you write mfbuf[0] then mfbuf[1] separately, when you could just write the whole array in a single call?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  10. #10
    Registered User
    Join Date
    Jul 2008
    Posts
    8
    [edit]I thought I had a test case exposing the bug, but I was mistaken... [/edit]

    Salem: mf_needed is hardcoded to 1904 at the moment. There's no particular reason to do mfbuf[0] and [1] separately, other than I need that 2-D structure, and I'm not sure about the continuity in memory of 2-D arrays. Sorry, but what do you mean by "rb" and "wb". The example I posted is rather trivial, and does not store the entire file that is read. Eventually mfbuf will be consumed in some manner and repopulated with new data.
    Last edited by cecomp64; 07-16-2008 at 01:56 PM.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Need Help Fixing My C Program. Deals with File I/O
    By Matus in forum C Programming
    Replies: 7
    Last Post: 04-29-2008, 07:51 PM
  2. File i/o problem
    By tezcatlipooca in forum C++ Programming
    Replies: 18
    Last Post: 01-01-2007, 08:01 AM
  3. File I/O problem
    By Onions in forum C++ Programming
    Replies: 41
    Last Post: 02-24-2006, 03:32 PM
  4. File I/O problem
    By 81N4RY_DR460N in forum C++ Programming
    Replies: 12
    Last Post: 09-03-2005, 12:14 PM
  5. Possible circular definition with singleton objects
    By techrolla in forum C++ Programming
    Replies: 3
    Last Post: 12-26-2004, 09:46 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21