Thread: File reading and writing

Hi!

I know how to read from file and write into a file, but if I do this with fscanf and fprintf, I notice that this procedure is very slow.

How to read and write files with some faster procedures?

Look up fread and fwrite. I'm not sure if they are faster, but they are available. Good luck...

--Garfield

It depends on what you intend to do with the file. If you plan on merely copying the file then there are several speedy algorithms that can be used. If you are traversing a text file and processing it at the same time then that is naturally time consuming.

For example, if you are choosing a string at random from a file with a single pass then the easiest way is to save a 'survivor' into a char array. You run through the file and choose between the survivor and the next string at random, then save winner to the array. At the end you have a string chosen completely at random in a single pass, but it's inefficient.

If you're taking a data file and placing each line in the file into a structure then fread and fwrite are the fastest ways to do this. fprintf and fscanf are just a pain for this type of use.

For the most part, using files tends to be slow because of the amount of data that has to be traversed. Try searching the net for several algorithms and then try them all; use your compiler's disassembly function to look at the corresponding assembler code and you'll quickly find out which is the most efficient method.

-Prelude

PRELUDE:

"If you plan on merely copying the file then there are several speedy algorithms that can be used."

If you know them than tell me how to write them.

Here's one, it's not portable but it's the simplest I can think of. Assuming you are using a windows o/s this can be modified several ways to suit your needs and still work, such as taking the two files from argv and concatenating them into the string.

Code:

#include <stdio.h>
#include <stdlib.h>

int main(void){
	char *copy = "copy c:\\windows\\desktop\\in.txt "
                     "c:\\windows\\desktop\\out.txt";

	system(copy);
	return EXIT_SUCCESS;
}

If you want something more portable then consider fread and fwrite, remember that the computer will handle files faster and more efficiently if you transfer data in large blocks. Say perhaps the size of the file

Code:

#include <stdio.h>
#include <stdlib.h>

struct file{
	char *p;
};

int main(void){
	FILE *in, *out;
	struct file f;
	in = fopen("c:\\windows\\desktop\\in.txt", "r");
	out = fopen("c:\\windows\\desktop\\out.txt", "w");

	f.p = (char *)malloc(10 * sizeof(f.p));

	fread(f.p, 10, 1, in);
	fwrite(f.p, 10, 1, out);

	return EXIT_SUCCESS;
}

Please excuse the crude code, I was in a hurry :P

-Prelude

Thanks, but why this "10 * sizeof(f.p)", why not just sizeof(f.p)?

Preludes right on track, there. Also you can do some thing like:

Code:

char * GetFileBuffer(char *filename)
{
FILE *fp;

fp = fopen(filename, "r");

fseek(fp,0L,SEEK_END);

int filesize = ftell(fp);

rewind(fp);

char *buffer = (char *)malloc(sizeof(char) * filesize);

char line[1000]; //...no worries, loses scope on exit...

while(!feof(fp))
{
 fgets(line, 1000, fp);
 strcat(buffer, line);
}

fclose(fp);

return buffer;
}

This was my first text retrieval routine. It works fine but is not as efficient as fread(). See if you can rewrite it with fread. Accessing each word then is not really that hard. My approach was to count the number of words in the buffer, then declared an array of char *pointers of that many and read them into the array of pointers with a function that specifically returns the next succesive word. Parsing is really the art of redirecting snippets of text. Writing routines that will parse out ints, floats, strings, etc will make this sort of task childs play, but the algorithms for parsing are not always obvious though they are usually quite simple. Consider for a moment the act of accessing each individual word in a string. you could use sscanf or you can use the simple loop:
isalpha(buff) ? temp[i++] = buff[i]:strcpy(output, temp);
in other words, as soon as a flag indicates we have reached a non-space character, the copying to a temp string is initiated. As soon as we encounter another space character, the process is stopped, the temp string is copied to the malloc'ed output string, and the function exits. The placeholder for such a function is an int which serves as the index ('i'). The protoype might be:
char *function(char *buff, int *i){//remember to access i with the '*'}

Anyway I hope that helps some

Aha, I understand. That 10 is the file size, right?

Yes, I was using a simple text file and the size was 10 bytes. As I said, I was in a hurry and failed to mention it, my appologies. In a better example, I should have used a variable that held the size and was a bit more descriptive than the integer constant 10.

-Prelude

Make sure you remember to check the return value of fopen() which returns NULL if the file cannot be opened. As far as I have noticed the best way to do it is like the others said, fread() and fwrite(). Also you can check the return values of fwrite() to make sure everything was written.

What about if I only want that 1 MB of 2 MB size is read. How do I do that with fread and fwrite (this I'll use for splitting files)?

If the file is HUGE, you'll still use block swapping. But instead of using a structure that holds the entire file, divide the size of the file by however much you want and have the block hold that much. Then you can use fread and fwrite to pass smaller chunks to the other file. Just place the read and write in a loop and use the number that you divided by as the condition for the loop. So if I divide the file by six, I fread and fwrite until I've done so six times and the file will be copied completely.

-Prelude

Ok, previously example was this "f.p = (char *)malloc(10 * sizeof(f.p));", but what do I write now for allocating memory?

If you knew how to use malloc, you wouldn't be asking. If you read a huge thread on memory allocation, you'd know how to use malloc.

Anyway...

Read the links, and you'll be wise in the ways of malloc.

x = malloc( sizeof(data) * number_of_data );
x = calloc( number_of_data, sizeof(data) );

Additonally, you don't use malloc as you've described. You could, but there is no need to.

Quzah.

You're right Quzah, as usual. I have a project that I use to help out with programs on these forums and it's saved as a .cpp file. Oddly enough, when I use malloc it requires a cast as opposed to saving it as a .c file where the cast is not needed.

-Prelude