File reading and writing

This is a discussion on File reading and writing within the C Programming forums, part of the General Programming Boards category; Hi! I know how to read from file and write into a file, but if I do this with fscanf ...

  1. #1
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669

    Question File reading and writing

    Hi!

    I know how to read from file and write into a file, but if I do this with fscanf and fprintf, I notice that this procedure is very slow.

    How to read and write files with some faster procedures?
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  2. #2
    the Corvetter
    Join Date
    Sep 2001
    Posts
    1,584
    Look up fread and fwrite. I'm not sure if they are faster, but they are available. Good luck...

    --Garfield
    1978 Silver Anniversary Corvette

  3. #3
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,796
    It depends on what you intend to do with the file. If you plan on merely copying the file then there are several speedy algorithms that can be used. If you are traversing a text file and processing it at the same time then that is naturally time consuming.

    For example, if you are choosing a string at random from a file with a single pass then the easiest way is to save a 'survivor' into a char array. You run through the file and choose between the survivor and the next string at random, then save winner to the array. At the end you have a string chosen completely at random in a single pass, but it's inefficient.

    If you're taking a data file and placing each line in the file into a structure then fread and fwrite are the fastest ways to do this. fprintf and fscanf are just a pain for this type of use.

    For the most part, using files tends to be slow because of the amount of data that has to be traversed. Try searching the net for several algorithms and then try them all; use your compiler's disassembly function to look at the corresponding assembler code and you'll quickly find out which is the most efficient method.

    -Prelude
    My best code is written with the delete key.

  4. #4
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669
    PRELUDE:

    "If you plan on merely copying the file then there are several speedy algorithms that can be used."

    If you know them than tell me how to write them.
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  5. #5
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,796
    Here's one, it's not portable but it's the simplest I can think of. Assuming you are using a windows o/s this can be modified several ways to suit your needs and still work, such as taking the two files from argv and concatenating them into the string.
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    int main(void){
    	char *copy = "copy c:\\windows\\desktop\\in.txt "
                         "c:\\windows\\desktop\\out.txt";
    
    	system(copy);
    	return EXIT_SUCCESS;
    }
    If you want something more portable then consider fread and fwrite, remember that the computer will handle files faster and more efficiently if you transfer data in large blocks. Say perhaps the size of the file
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    struct file{
    	char *p;
    };
    
    int main(void){
    	FILE *in, *out;
    	struct file f;
    	in = fopen("c:\\windows\\desktop\\in.txt", "r");
    	out = fopen("c:\\windows\\desktop\\out.txt", "w");
    
    	f.p = (char *)malloc(10 * sizeof(f.p));
    
    	fread(f.p, 10, 1, in);
    	fwrite(f.p, 10, 1, out);
    
    	return EXIT_SUCCESS;
    }
    Please excuse the crude code, I was in a hurry :P

    -Prelude
    My best code is written with the delete key.

  6. #6
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669
    Thanks, but why this "10 * sizeof(f.p)", why not just sizeof(f.p)?
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  7. #7
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,707
    Preludes right on track, there. Also you can do some thing like:


    Code:
    
    char * GetFileBuffer(char *filename)
    {
    FILE *fp;
    
    fp = fopen(filename, "r");
    
    fseek(fp,0L,SEEK_END);
    
    int filesize = ftell(fp);
    
    rewind(fp);
    
    char *buffer = (char *)malloc(sizeof(char) * filesize);
    
    char line[1000]; //...no worries, loses scope on exit...
    
    while(!feof(fp))
    {
     fgets(line, 1000, fp);
     strcat(buffer, line);
    }
    
    fclose(fp);
    
    return buffer;
    }
    This was my first text retrieval routine. It works fine but is not as efficient as fread(). See if you can rewrite it with fread. Accessing each word then is not really that hard. My approach was to count the number of words in the buffer, then declared an array of char *pointers of that many and read them into the array of pointers with a function that specifically returns the next succesive word. Parsing is really the art of redirecting snippets of text. Writing routines that will parse out ints, floats, strings, etc will make this sort of task childs play, but the algorithms for parsing are not always obvious though they are usually quite simple. Consider for a moment the act of accessing each individual word in a string. you could use sscanf or you can use the simple loop:
    isalpha(buff) ? temp[i++] = buff[i]:strcpy(output, temp);
    in other words, as soon as a flag indicates we have reached a non-space character, the copying to a temp string is initiated. As soon as we encounter another space character, the process is stopped, the temp string is copied to the malloc'ed output string, and the function exits. The placeholder for such a function is an int which serves as the index ('i'). The protoype might be:
    char *function(char *buff, int *i){//remember to access i with the '*'}

    Anyway I hope that helps some
    Code:
    #include <cmath>
    #include <complex>
    bool flip(bool value)
    {
           return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) * std::complex<float>(std::atan(1.0)*(1 << (value + 2)))
        ).real() < 0;
    }

  8. #8
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669
    Aha, I understand. That 10 is the file size, right?
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  9. #9
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,796
    Yes, I was using a simple text file and the size was 10 bytes. As I said, I was in a hurry and failed to mention it, my appologies. In a better example, I should have used a variable that held the size and was a bit more descriptive than the integer constant 10.

    -Prelude
    My best code is written with the delete key.

  10. #10
    Registered User
    Join Date
    Dec 2001
    Posts
    26
    Make sure you remember to check the return value of fopen() which returns NULL if the file cannot be opened. As far as I have noticed the best way to do it is like the others said, fread() and fwrite(). Also you can check the return values of fwrite() to make sure everything was written.
    one fish two fish
    red fish blue fish

  11. #11
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669

    Question

    What about if I only want that 1 MB of 2 MB size is read. How do I do that with fread and fwrite (this I'll use for splitting files)?
    Last edited by GaPe; 12-28-2001 at 03:43 PM.
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  12. #12
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,796
    If the file is HUGE, you'll still use block swapping. But instead of using a structure that holds the entire file, divide the size of the file by however much you want and have the block hold that much. Then you can use fread and fwrite to pass smaller chunks to the other file. Just place the read and write in a loop and use the number that you divided by as the condition for the loop. So if I divide the file by six, I fread and fwrite until I've done so six times and the file will be copied completely.

    -Prelude
    My best code is written with the delete key.

  13. #13
    Code Warrior
    Join Date
    Nov 2001
    Posts
    669
    Ok, previously example was this "f.p = (char *)malloc(10 * sizeof(f.p));", but what do I write now for allocating memory?
    Current projects:
    1) User Interface Development Kit (C++)
    2) HTML SDK (C++)
    3) Classes (C++)
    4) INI Editor (Delphi)

  14. #14
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    If you knew how to use malloc, you wouldn't be asking. If you read a huge thread on memory allocation, you'd know how to use malloc.

    Anyway...

    Read the links, and you'll be wise in the ways of malloc.

    x = malloc( sizeof(data) * number_of_data );
    x = calloc( number_of_data, sizeof(data) );

    Additonally, you don't use malloc as you've described. You could, but there is no need to.

    Quzah.
    Hope is the first step on the road to disappointment.

  15. #15
    Code Goddess Prelude's Avatar
    Join Date
    Sep 2001
    Posts
    9,796
    You're right Quzah, as usual. I have a project that I use to help out with programs on these forums and it's saved as a .cpp file. Oddly enough, when I use malloc it requires a cast as opposed to saving it as a .c file where the cast is not needed.

    -Prelude
    My best code is written with the delete key.

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. segmetation fault (reading and writing file in c)
    By tasosa in forum C Programming
    Replies: 5
    Last Post: 04-13-2009, 06:04 AM
  2. Reading out of and writing into the same file
    By Wiretron in forum C Programming
    Replies: 8
    Last Post: 12-30-2006, 01:04 PM
  3. Replies: 3
    Last Post: 03-04-2005, 01:46 PM
  4. file writing and reading
    By Micko in forum C Programming
    Replies: 8
    Last Post: 01-13-2004, 10:18 AM
  5. what does this mean to you?
    By pkananen in forum C++ Programming
    Replies: 8
    Last Post: 02-04-2002, 02:58 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21