Efficiency of Reading a File

**kiai_viper** · 07-20-2007

I have a question regarding the efficiency of a proposed program: I would like to copy a file's contents into an array, sort the array, create a frequency array (maybe a histogram as well), and then print the results. The text file will contain on string on each line. My question is: should I create a separate text file to store the sorted data, or should I use an array. Which option is more efficient and/or logical?

Thanks.

**ssharish2005** · 07-20-2007

It would be more better if you could use the array for sorting and stuff. Since the swaping between the array elements is more easy in arrays rather than in file.

And its more faster as well. Sorting within the file will require to create one more temp file as well.

You need to consider l, how big the file is, which your getting stored on to the array. Even that matter.

ssharish2005

**kiai_viper** · 07-20-2007

The text file will should have approximately 500 lines max. On a typical run, it will have 100 entries.

The problem that I am having is reading the file contents and storing them into an array.
How would I go about storing the contents? The below code reads the file and then stores the contents into a new file. Instead of storing the contents into a new file, I would rather store them into an array. I have tried to use the string array "buff", but the program always terminates with errors. Do you have any suggestions on how I should modify my code?

Thanks.

Code:

int main()
{
   FILE *fp1, *fp2;
   char filename1[]="pass.txt";
   char filename2[]="outpass.txt";
   char buff[MAX_LEN];
   
   if((fp1=fopen(filename1, "r")) == NULL)
   {
      printf("Cannot open %s for reading", filename1);
   }else if((fp2=fopen(filename2, "w")) == NULL)
    {
        printf("Cannot open %s for writing", filename2);
     }
   while(fgets(buff, MAX_LEN, fp1) != NULL)
   {
      fputs(buff, fp2);
      printf("%s", buff);
   }
   system("PAUSE");
   return 0;
    
}

**manofsteel972** · 07-20-2007

If you have very large files, you would need to put the array on the Heap not the stack to avoid stack overflow. There is limited stack space, but if you use the Heap then if you run out of RAM usually the OS virtual memory would kick in and use a swap file.

**kiai_viper** · 07-20-2007

Originally Posted by manofsteel972

If you have very large files, you would need to put the array on the Heap not the stack to avoid stack overflow. There is limited stack space, but if you use the Heap then if you run out of RAM usually the OS virtual memory would kick in and use a swap file.

Thanks, the text files that I am working with are between 1KB - 5KB. I have 2GB of RAM. Would this be okay for the stack? Also, do you have any suggestions on storing the file into an array?

--KiaiViper

**manofsteel972** · 07-20-2007

char buff[MAX_LEN]

Will only hold a single string, You would need an array of strings. If the number of strings in the file are not going to exceed a certain number such as 100 then you could declare the array

char buff[100][MAX_LEN];

If the number of strings in the file will vary, then you would need a dynamic array of strings. You could count the number of strings in the file and then dynamically allocate the array. Just be sure to release the dynamic array when you are done with it.

**kiai_viper** · 07-20-2007

Okay, I have created an array str[2][100] as a test. The text file contains:

Jane
Bob

How would I copy the file into the array? I have tried using fgets:

Code:

 while(fgets(ary,100,fp1) != NULL)
   {
      scanf("&#37;s", str[0][0]);
   }

Should I be using a different function?

Thanks.

Hey guys, I kind of figured it out. Here is my code:

Code:

#include <stdio.h>
#include <stdlib.h>

int main()
{
   FILE *fp1;
   char filename[]="pass.txt";
   char ary[100][100];
   int i;
   
   
   if((fp1=fopen(filename, "r")) == NULL)
   {
      printf("Cannot open %s for reading.", filename);
   }
   
   for(i=0;i<100;i++)
   {
      fscanf(fp1, "%s", ary[i]);
   }
   for(i=0;i<100;i++)
   {
      printf("%s\n", ary[i]);
   }
   
   system("PAUSE");
   return 0;
}

I am using fscanf to read in the file into the array. I now need to dynamically allocate memory for the contents of the file. Going to brush up on dynamic memory allocation. Do you have an suggestions regarding which allocation method I should use to find the exact size of the text file?

Thanks.

--KiaiViper

**MacNilly** · 07-21-2007

Originally Posted by kiai_viper

I have a question regarding the efficiency of a proposed program: I would like to copy a file's contents into an array, sort the array, create a frequency array (maybe a histogram as well), and then print the results. The text file will contain on string on each line. My question is: should I create a separate text file to store the sorted data, or should I use an array. Which option is more efficient and/or logical?

Thanks.

What is a "frequency array?" I can only assume it's an array that stores the frequency of data types entered, char/int/whatever.

You say you have one string on each line, so that data type must be a "string", no? Unless you plan to store multiple values on each line in the file and use scanf() or similar to get multiple values from each line...

For the last question, the difference between storing on a file or an array... well the answer is very basic: If you want permanent access to that data, store in a text file. If you want to manipulate values in "memory," there is no need to create a permanent copy. As far as efficency, writing to memory locations is always faster than writing to disk space, AFAIK.

**kiai_viper** · 07-23-2007

Originally Posted by MacNilly

What is a "frequency array?" I can only assume it's an array that stores the frequency of data types entered, char/int/whatever.

A frequency array shows the occurrence of a variable in a given array. If there is an array of integers, you can create a frequency array that shows the "frequency" of the number 10 in the first array. Check out the my previous post regarding a frequency array for a more detailed answer:

http://cboard.cprogramming.com/showthread.php?t=91897

Thanks for your clarification on the efficiency issue. I did not want to use a secondary text file.

--KiaiViper

Thread: Efficiency of Reading a File

Thread Tools

Search Thread

Display

Efficiency of Reading a File

Similar Threads

opening empty file causes access violation

sequential file program

Can we have vector of vector?

Game Pointer Trouble?

System