Efficiency of Reading a File

This is a discussion on Efficiency of Reading a File within the C Programming forums, part of the General Programming Boards category; I have a question regarding the efficiency of a proposed program: I would like to copy a file's contents into ...

  1. #1
    Registered User
    Join Date
    Jun 2007
    Posts
    19

    Efficiency of Reading a File

    I have a question regarding the efficiency of a proposed program: I would like to copy a file's contents into an array, sort the array, create a frequency array (maybe a histogram as well), and then print the results. The text file will contain on string on each line. My question is: should I create a separate text file to store the sorted data, or should I use an array. Which option is more efficient and/or logical?


    Thanks.

  2. #2
    Registered User ssharish2005's Avatar
    Join Date
    Sep 2005
    Location
    Cambridge, UK
    Posts
    1,682
    It would be more better if you could use the array for sorting and stuff. Since the swaping between the array elements is more easy in arrays rather than in file.

    And its more faster as well. Sorting within the file will require to create one more temp file as well.

    You need to consider l, how big the file is, which your getting stored on to the array. Even that matter.

    ssharish2005

  3. #3
    Registered User
    Join Date
    Jun 2007
    Posts
    19
    The text file will should have approximately 500 lines max. On a typical run, it will have 100 entries.

    The problem that I am having is reading the file contents and storing them into an array.
    How would I go about storing the contents? The below code reads the file and then stores the contents into a new file. Instead of storing the contents into a new file, I would rather store them into an array. I have tried to use the string array "buff", but the program always terminates with errors. Do you have any suggestions on how I should modify my code?

    Thanks.

    Code:
    int main()
    {
       FILE *fp1, *fp2;
       char filename1[]="pass.txt";
       char filename2[]="outpass.txt";
       char buff[MAX_LEN];
       
       if((fp1=fopen(filename1, "r")) == NULL)
       {
          printf("Cannot open %s for reading", filename1);
       }else if((fp2=fopen(filename2, "w")) == NULL)
        {
            printf("Cannot open %s for writing", filename2);
         }
       while(fgets(buff, MAX_LEN, fp1) != NULL)
       {
          fputs(buff, fp2);
          printf("%s", buff);
       }
       system("PAUSE");
       return 0;
        
    }

  4. #4
    Registered User manofsteel972's Avatar
    Join Date
    Mar 2004
    Posts
    317
    If you have very large files, you would need to put the array on the Heap not the stack to avoid stack overflow. There is limited stack space, but if you use the Heap then if you run out of RAM usually the OS virtual memory would kick in and use a swap file.
    "Knowledge is proud that she knows so much; Wisdom is humble that she knows no more."
    -- Cowper

    Operating Systems=Slackware Linux 9.1,Windows 98/Xp
    Compilers=gcc 3.2.3, Visual C++ 6.0, DevC++(Mingw)

    You may teach a person from now until doom's day, but that person will only know what he learns himself.

    Now I know what doesn't work.

    A problem is understood by solving it, not by pondering it.

    For a bit of humor check out xkcd web comic http://xkcd.com/235/

  5. #5
    Registered User
    Join Date
    Jun 2007
    Posts
    19
    Quote Originally Posted by manofsteel972 View Post
    If you have very large files, you would need to put the array on the Heap not the stack to avoid stack overflow. There is limited stack space, but if you use the Heap then if you run out of RAM usually the OS virtual memory would kick in and use a swap file.
    Thanks, the text files that I am working with are between 1KB - 5KB. I have 2GB of RAM. Would this be okay for the stack? Also, do you have any suggestions on storing the file into an array?

    --KiaiViper

  6. #6
    Registered User manofsteel972's Avatar
    Join Date
    Mar 2004
    Posts
    317
    char buff[MAX_LEN]

    Will only hold a single string, You would need an array of strings. If the number of strings in the file are not going to exceed a certain number such as 100 then you could declare the array

    char buff[100][MAX_LEN];

    If the number of strings in the file will vary, then you would need a dynamic array of strings. You could count the number of strings in the file and then dynamically allocate the array. Just be sure to release the dynamic array when you are done with it.
    "Knowledge is proud that she knows so much; Wisdom is humble that she knows no more."
    -- Cowper

    Operating Systems=Slackware Linux 9.1,Windows 98/Xp
    Compilers=gcc 3.2.3, Visual C++ 6.0, DevC++(Mingw)

    You may teach a person from now until doom's day, but that person will only know what he learns himself.

    Now I know what doesn't work.

    A problem is understood by solving it, not by pondering it.

    For a bit of humor check out xkcd web comic http://xkcd.com/235/

  7. #7
    Registered User
    Join Date
    Jun 2007
    Posts
    19
    Okay, I have created an array str[2][100] as a test. The text file contains:

    Jane
    Bob

    How would I copy the file into the array? I have tried using fgets:

    Code:
     while(fgets(ary,100,fp1) != NULL)
       {
          scanf("%s", str[0][0]);
       }
    Should I be using a different function?

    Thanks.



    Hey guys, I kind of figured it out. Here is my code:

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    
    int main()
    {
       FILE *fp1;
       char filename[]="pass.txt";
       char ary[100][100];
       int i;
       
       
       if((fp1=fopen(filename, "r")) == NULL)
       {
          printf("Cannot open %s for reading.", filename);
       }
       
       for(i=0;i<100;i++)
       {
          fscanf(fp1, "%s", ary[i]);
       }
       for(i=0;i<100;i++)
       {
          printf("%s\n", ary[i]);
       }
       
       system("PAUSE");
       return 0;
    }
    I am using fscanf to read in the file into the array. I now need to dynamically allocate memory for the contents of the file. Going to brush up on dynamic memory allocation. Do you have an suggestions regarding which allocation method I should use to find the exact size of the text file?

    Thanks.


    --KiaiViper
    Last edited by kiai_viper; 07-20-2007 at 02:24 PM. Reason: Double posting

  8. #8
    Registered User MacNilly's Avatar
    Join Date
    Oct 2005
    Posts
    311
    Quote Originally Posted by kiai_viper View Post
    I have a question regarding the efficiency of a proposed program: I would like to copy a file's contents into an array, sort the array, create a frequency array (maybe a histogram as well), and then print the results. The text file will contain on string on each line. My question is: should I create a separate text file to store the sorted data, or should I use an array. Which option is more efficient and/or logical?


    Thanks.
    What is a "frequency array?" I can only assume it's an array that stores the frequency of data types entered, char/int/whatever.

    You say you have one string on each line, so that data type must be a "string", no? Unless you plan to store multiple values on each line in the file and use scanf() or similar to get multiple values from each line...

    For the last question, the difference between storing on a file or an array... well the answer is very basic: If you want permanent access to that data, store in a text file. If you want to manipulate values in "memory," there is no need to create a permanent copy. As far as efficency, writing to memory locations is always faster than writing to disk space, AFAIK.

  9. #9
    Registered User
    Join Date
    Jun 2007
    Posts
    19
    Quote Originally Posted by MacNilly View Post
    What is a "frequency array?" I can only assume it's an array that stores the frequency of data types entered, char/int/whatever.
    A frequency array shows the occurrence of a variable in a given array. If there is an array of integers, you can create a frequency array that shows the "frequency" of the number 10 in the first array. Check out the my previous post regarding a frequency array for a more detailed answer:

    Frequency Array

    Thanks for your clarification on the efficiency issue. I did not want to use a secondary text file.


    --KiaiViper

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. opening empty file causes access violation
    By trevordunstan in forum C Programming
    Replies: 10
    Last Post: 10-21-2008, 11:19 PM
  2. sequential file program
    By needhelpbad in forum C Programming
    Replies: 80
    Last Post: 06-08-2008, 01:04 PM
  3. Can we have vector of vector?
    By ketu1 in forum C++ Programming
    Replies: 24
    Last Post: 01-03-2008, 04:02 AM
  4. Game Pointer Trouble?
    By Drahcir in forum C Programming
    Replies: 8
    Last Post: 02-04-2006, 01:53 AM
  5. System
    By drdroid in forum C++ Programming
    Replies: 3
    Last Post: 06-28-2002, 10:12 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21