Problem saving data in an array of structures to a file

This is a discussion on Problem saving data in an array of structures to a file within the C Programming forums, part of the General Programming Boards category; I have a big array of data, basically an array of a structured date type and I am trying to ...

  1. #1
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794

    Problem saving data in an array of structures to a file

    I have a big array of data, basically an array of a structured date type and I am
    trying to write it into a file to save it.


    However it does not save it all, only about 10,459 bytes (sigificant number?)
    I get this size by using a hex editor, its all nulls after that.


    File size in windows is 907kb (928,019 bytes)
    908 KB (929,792 bytes)


    Basically there is data for 105 people in the array but only data for 90 is saved in the data file.

    Here is some of the relevant code

    Code:
    
    // The data.
    #define MAXLIST 8000
    
    typedef struct {
    			char name[40];		
    			float stack;	
    			float dolstack;
    			float tempval;
    			int	gamesplayed;
    			int tookpart;
    			int tookpart3;
    			int tookpart5;
    			int tookpart8;
    			int five;
    			int eight;
    			int three;
    			int unmatched;
    			int pfr;
    			int afr;
    			int atr;
    			int arr;
    			float win;
    			float win5;
    			float tempstack;
    			} PLYR;
    			
    PLYR jim, pname[MAXLIST];


    And here is the code that saves it.






    Code:
    datasave(){
    int i;
    	if (datafile==FALSE){
    	
    		printf("\n openinig %d", errno);
    	
    		if (	(dataptr=open("data.doc", O_WRONLY|O_CREAT )) != NULL) {
    	
    			
    		//	for(i=0; i<listlen; i++){
    				printf("\n writing ");
    				//	fd = open( "data.dat", O_RDWR|O_CREAT|O_APPEND );
    				printf("\n error  aaa %d", errno);
    
    
    
    				/* write the binary structure right to the file */
    				
    
                                                                     write( dataptr, pname, sizeof( pname   ));
    
    
    
    
    
    				printf("\n error  bbb %d", errno);
    			//}
    				close(dataptr);
    			
    		}
    		else {
    			printf("\n error %d", errno);
    			puts("\nCant creat datafile");
    			exit(2);
    
    		}
    		printf("\n error after %d", errno);
    	}
    }

    As you can see I have left in some redundant code (I realised/hoped I could do it in one line).

    Here is some of the output for the error codes (may not be errors at all, just my debug statements). Any comments welcome. It's a microsoft machine.

  2. #2
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    OH here is what the debug stuff produces.

    openinig 0
    writing
    error aaa 4
    error bbb 6
    error after 6


    And their meanings

    #define EACCES 4 (access error?)

    #define EBADF 6 (bad file descriptor?)

    These codes may be misleading

  3. #3
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,171
    What's listlen? I would have imagined it would be a variable that holds how many array elements are currently in use (filled). But your write() call tells a different story; that you're writing the entire array at once. Your listlen loop is confusing. I don't see why you'd want to spit out the whole array more than once.

    Also, open() never returns NULL. It returns -1 on failure.

    What's the point of opening data.dat anyway? You never write anything to it. That whole function is a little messed up.

    You're closing dataptr in the loop and never reopening it again. That would explain why you're getting a bad file descriptor error. Did you actually try to understand the function you showed? Try to follow the logic path?
    Last edited by itsme86; 06-06-2007 at 06:42 PM.
    If you understand what you're doing, you're not learning anything.

  4. #4
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    Listlen is commented out.

  5. #5
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    I have removed some of the commented out stuff to make it clearer. (I thought I needed a loop before, but it appears I can do it in one line!)

    Code:
    datasave(){
    int i;
    	if (datafile==FALSE){
    	
    		printf("\n openinig %d", errno);
    	
    		if (	(dataptr=open("data.doc", O_WRONLY|O_CREAT )) != NULL) {
    	
    				printf("\n writing ");
    				printf("\n error  aaa %d", errno);
    
    
    				/* write the binary structure right to the file */
    				write( dataptr, pname, sizeof( pname   ));
    
    
    				printf("\n error  bbb %d", errno);
    				close(dataptr);
    			
    		}
    		else {
    			printf("\n error %d", errno);
    			puts("\nCant creat datafile");
    			exit(2);
    
    		}
    		printf("\n error after %d", errno);
    	}
    }

  6. #6
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    -1 is NULL anyway, however it does not matter really the files is created?

    I do write to the file, 88&#37; (ish) of the data is there

  7. #7
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    I tried it again with three times the amount of data but it still only write the same 90 od records when there are now over 200.
    I am wondering if there is some sort of compiler limitation or something, is there some significance in about 10kB?

  8. #8
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    Something to do with record size, maybe max record size is 10KB???

  9. #9
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,171
    What does the return value of write() say?

    Try writing one element at a time to the file and see if it does the whole thing. Also, that write() that you're using isn't trying to write 105 elements, it's trying to write 8000 (it's using the size of the array).
    If you understand what you're doing, you're not learning anything.

  10. #10
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    Quote Originally Posted by itsme86 View Post
    What does the return value of write() say?

    Try writing one element at a time to the file and see if it does the whole thing. Also, that write() that you're using isn't trying to write 105 elements, it's trying to write 8000 (it's using the size of the array).
    I will try and find the value of the return from the write() and post it later.

    I know it is trying to write 8000 records, thats OK (as long as it does it).

    It is convienient if it would because then I would not have to mess aroud with printf and a loop,
    as I do already do for data derived from the file for 'reports'. It would mean I could save the whole 'database' in one foul swoop and I would not have to modify the code when i modified the data structure. I could just save the whole 'data space' and then read it back in again.
    (It takes ages (20 minutes) to create it).
    It just occured to me I could just specify a pointer to the data and the length and then write it as one record, so easy. It seems to work upto about 10Kb so I think maybe this is some sort of
    record length limit?
    I suppose it is not too much trouble to put it in a loop and write one record at a time, which is what I initially was going to do untill I had my 'bright idea'. I think I will give that a go in the meantime.

  11. #11
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    I might have more joy with fwrite not write?

  12. #12
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    That seems to have done the trick!!!

    There are all the records thee now 200+, the last one is there, all I need to do now is
    write a function to read the lot back in again!!

    Here is the modified code (have to change the return value to a file pointer too as write uses an integer.) Here is the modified code. (tidied up!!).
    It's so easy, no messing about with a loop and formatting crap etc...just half a dozen lines
    and its done and you dont have to maintian it when you make changes to the data structure!!
    A useful function I think you will agree!!

    Code:
    datasave(){
    	if (	(dataptr=fopen("data.doc", "w+" )) != NULL) {
    		fwrite(  pname, sizeof( pname   ), 1, dataptr);
    		fclose(dataptr);
    	}
    	else {
    		puts("\nCant creat datafile");
    		exit(2);
    	}
    }

  13. #13
    CSharpener vart's Avatar
    Join Date
    Oct 2006
    Location
    Rishon LeZion, Israel
    Posts
    6,484
    fwrite( pname, sizeof( pname ), 1, dataptr);

    if pname is a pointer
    sizeof pname is sizeof void* is 4 on most compilers

    don't you want sizeof *pname?
    The first 90% of a project takes 90% of the time,
    the last 10% takes the other 90% of the time.

  14. #14
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    Quote Originally Posted by vart View Post
    fwrite( pname, sizeof( pname ), 1, dataptr);

    if pname is a pointer
    sizeof pname is sizeof void* is 4 on most compilers

    don't you want sizeof *pname?
    Well you could be right, I did think I had put sizeof(pname[MAXLIST)), indeed "itsme86" seems to have assumed this too and looking at my actually code I do seem to have just pname!!!

    Anyway, it does work in that it writes all the data into a file, so it is definitely using more than a value of 4 it seems.

    It's likely I got confused because I was playing two poker tables (badly) as I was writing it!!

    The size of the file on disk is about 907KB and I have 8000 records in the array so.. that would give a record length of about 113 for the data structure.
    In the structure I have a character string of 40 + 6 floats and 12 ints which seems in the
    'right area', if ints are 4 and floats are 6? thats 40+36+48=124, which is not too far off given the giggery pokery of these things.

    I think maybe the compiler 'corrected' my error, which is nice, because that will be a first!!
    It may have 'known' the size of pname, I think it actually does, I have a vague recollection of reading that when I was looking it up.

    Anyway whatever the case it seems to work which is great, I have had loads of problems modifying printf statement for my reports, it is so easy to make a error, but with this you just give the name of the structure (and maybe size) and you don't have to worry about formatting etc.. it's all done for you, no loop counter either!!

    The program saves player data from poker games incidently, so it is handy to be able to read in the old data rather than recreate it, especially when you have about 1/2 a million games (files) to process. It takea ages when they are all disk, as disks are slow, but once they are in buffered in memory I can can run a similar report in a fraction of the time, probably a 100 times faster.

    Each day I get another several hundred files which don't take long to do but if I have to reprocess a years worth it takes ages (so I currently only do 3 months) but if I can store the datafile it will be done in a few seconds.

    Anyway I don't think the compiler sees pname as a pointer now, it sees it as the structure as it is defined using the length given at the time.

  15. #15
    Fountain of knowledge.
    Join Date
    May 2006
    Posts
    794
    Quote Originally Posted by itsme86 View Post
    Ignore any "advice" esbo tries to give you. It's wrong.
    Well glad to see you still remember me!!

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Data file problem use FILE, loop, and if-else
    By Cyberman86 in forum C Programming
    Replies: 3
    Last Post: 03-25-2009, 11:52 PM
  2. Replies: 48
    Last Post: 09-26-2008, 04:45 AM
  3. Replies: 4
    Last Post: 06-14-2005, 06:45 AM
  4. simulate Grep command in Unix using C
    By laxmi in forum C Programming
    Replies: 6
    Last Post: 05-10-2002, 05:10 PM
  5. File Database & Data Structure :: C++
    By kuphryn in forum C++ Programming
    Replies: 0
    Last Post: 02-24-2002, 11:47 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21