Thread: Problem saving data in an array of structures to a file

I have a big array of data, basically an array of a structured date type and I am
trying to write it into a file to save it.


However it does not save it all, only about 10,459 bytes (sigificant number?)
I get this size by using a hex editor, its all nulls after that.


File size in windows is 907kb (928,019 bytes)
908 KB (929,792 bytes)


Basically there is data for 105 people in the array but only data for 90 is saved in the data file.

Here is some of the relevant code

Code:

// The data.
#define MAXLIST 8000

typedef struct {
			char name[40];		
			float stack;	
			float dolstack;
			float tempval;
			int	gamesplayed;
			int tookpart;
			int tookpart3;
			int tookpart5;
			int tookpart8;
			int five;
			int eight;
			int three;
			int unmatched;
			int pfr;
			int afr;
			int atr;
			int arr;
			float win;
			float win5;
			float tempstack;
			} PLYR;
			
PLYR jim, pname[MAXLIST];

And here is the code that saves it.

Code:

datasave(){
int i;
	if (datafile==FALSE){
	
		printf("\n openinig %d", errno);
	
		if (	(dataptr=open("data.doc", O_WRONLY|O_CREAT )) != NULL) {
	
			
		//	for(i=0; i<listlen; i++){
				printf("\n writing ");
				//	fd = open( "data.dat", O_RDWR|O_CREAT|O_APPEND );
				printf("\n error  aaa %d", errno);



				/* write the binary structure right to the file */
				

                                                                 write( dataptr, pname, sizeof( pname   ));





				printf("\n error  bbb %d", errno);
			//}
				close(dataptr);
			
		}
		else {
			printf("\n error %d", errno);
			puts("\nCant creat datafile");
			exit(2);

		}
		printf("\n error after %d", errno);
	}
}

As you can see I have left in some redundant code (I realised/hoped I could do it in one line).

Here is some of the output for the error codes (may not be errors at all, just my debug statements). Any comments welcome. It's a microsoft machine.

OH here is what the debug stuff produces.

openinig 0
writing
error aaa 4
error bbb 6
error after 6


And their meanings

#define EACCES 4 (access error?)

#define EBADF 6 (bad file descriptor?)

These codes may be misleading

Listlen is commented out.

I have removed some of the commented out stuff to make it clearer. (I thought I needed a loop before, but it appears I can do it in one line!)

Code:

datasave(){
int i;
	if (datafile==FALSE){
	
		printf("\n openinig %d", errno);
	
		if (	(dataptr=open("data.doc", O_WRONLY|O_CREAT )) != NULL) {
	
				printf("\n writing ");
				printf("\n error  aaa %d", errno);


				/* write the binary structure right to the file */
				write( dataptr, pname, sizeof( pname   ));


				printf("\n error  bbb %d", errno);
				close(dataptr);
			
		}
		else {
			printf("\n error %d", errno);
			puts("\nCant creat datafile");
			exit(2);

		}
		printf("\n error after %d", errno);
	}
}

-1 is NULL anyway, however it does not matter really the files is created?

I do write to the file, 88% (ish) of the data is there

I tried it again with three times the amount of data but it still only write the same 90 od records when there are now over 200.
I am wondering if there is some sort of compiler limitation or something, is there some significance in about 10kB?

Something to do with record size, maybe max record size is 10KB???

Originally Posted by itsme86

What does the return value of write() say?

Try writing one element at a time to the file and see if it does the whole thing. Also, that write() that you're using isn't trying to write 105 elements, it's trying to write 8000 (it's using the size of the array).

I will try and find the value of the return from the write() and post it later.

I know it is trying to write 8000 records, thats OK (as long as it does it).

It is convienient if it would because then I would not have to mess aroud with printf and a loop,
as I do already do for data derived from the file for 'reports'. It would mean I could save the whole 'database' in one foul swoop and I would not have to modify the code when i modified the data structure. I could just save the whole 'data space' and then read it back in again.
(It takes ages (20 minutes) to create it).
It just occured to me I could just specify a pointer to the data and the length and then write it as one record, so easy. It seems to work upto about 10Kb so I think maybe this is some sort of
record length limit?
I suppose it is not too much trouble to put it in a loop and write one record at a time, which is what I initially was going to do untill I had my 'bright idea'. I think I will give that a go in the meantime.

I might have more joy with fwrite not write?

That seems to have done the trick!!!

There are all the records thee now 200+, the last one is there, all I need to do now is
write a function to read the lot back in again!!

Here is the modified code (have to change the return value to a file pointer too as write uses an integer.) Here is the modified code. (tidied up!!).
It's so easy, no messing about with a loop and formatting crap etc...just half a dozen lines
and its done and you dont have to maintian it when you make changes to the data structure!!
A useful function I think you will agree!!

Code:

datasave(){
	if (	(dataptr=fopen("data.doc", "w+" )) != NULL) {
		fwrite(  pname, sizeof( pname   ), 1, dataptr);
		fclose(dataptr);
	}
	else {
		puts("\nCant creat datafile");
		exit(2);
	}
}

Originally Posted by vart

fwrite( pname, sizeof( pname ), 1, dataptr);

if pname is a pointer
sizeof pname is sizeof void* is 4 on most compilers

don't you want sizeof *pname?

Well you could be right, I did think I had put sizeof(pname[MAXLIST)), indeed "itsme86" seems to have assumed this too and looking at my actually code I do seem to have just pname!!!

Anyway, it does work in that it writes all the data into a file, so it is definitely using more than a value of 4 it seems.

It's likely I got confused because I was playing two poker tables (badly) as I was writing it!!

The size of the file on disk is about 907KB and I have 8000 records in the array so.. that would give a record length of about 113 for the data structure.
In the structure I have a character string of 40 + 6 floats and 12 ints which seems in the
'right area', if ints are 4 and floats are 6? thats 40+36+48=124, which is not too far off given the giggery pokery of these things.

I think maybe the compiler 'corrected' my error, which is nice, because that will be a first!!
It may have 'known' the size of pname, I think it actually does, I have a vague recollection of reading that when I was looking it up.

Anyway whatever the case it seems to work which is great, I have had loads of problems modifying printf statement for my reports, it is so easy to make a error, but with this you just give the name of the structure (and maybe size) and you don't have to worry about formatting etc.. it's all done for you, no loop counter either!!

The program saves player data from poker games incidently, so it is handy to be able to read in the old data rather than recreate it, especially when you have about 1/2 a million games (files) to process. It takea ages when they are all disk, as disks are slow, but once they are in buffered in memory I can can run a similar report in a fraction of the time, probably a 100 times faster.

Each day I get another several hundred files which don't take long to do but if I have to reprocess a years worth it takes ages (so I currently only do 3 months) but if I can store the datafile it will be done in a few seconds.

Anyway I don't think the compiler sees pname as a pointer now, it sees it as the structure as it is defined using the length given at the time.

Originally Posted by itsme86

Ignore any "advice" esbo tries to give you. It's wrong.

Well glad to see you still remember me!!