Thread: entire file

I need to read an entire file. Past the \n all the wy to EOF. I am thinking about using unbuffered file reading(open) Is there another way.

Using open() and read() are your best bet.

How about using fgetc in a loop testing for EOF?

Oh god that would be horribly slow. Having to make a disk access call each time /shudder

Just to play around I wrote a sample program for it

Code:

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

#define MEG 1048576

int main(void)
{
  char buffer[MEG];
  int fd, total=0, got=0;

  fd = open("file.txt", O_RDONLY);
  if ( fd == -1)
  {
    perror("Couldn't open file");
    return 1;
  }

  while (total < sizeof(buffer) )
  {
    got = read(fd, &buffer[total], sizeof(buffer) - total);
    if ( got == -1 )
    {
      perror("Problem with read()");
      break;
    }
    if ( got == 0)
      break;
    total += got;
  }
  printf("Got %d characters\n", total);
  printf("%s\n", buffer);
  return 0;
}

It will read a meg - 1 worth of data. -1 is the error code for read and 0 is the EOF code.

Good to know.

Of course reading one character at a time will still be slower then reading the entire block at once. For example I made a one meg file filled with random characters from ASCII value of 0 to 127. Ran the following two programs on it
entirefile.c

Code:

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

#define MEG 1048576

int main(void)
{
  char buffer[MEG];
  int fd, total=0, got=0;

  fd = open("file.txt", O_RDONLY);
  if ( fd == -1)
  {
    perror("Couldn't open file");
    return 1;
  }

  while (total < sizeof(buffer) )
  {
    got = read(fd, &buffer[total], sizeof(buffer) - total);
    if ( got <= 0)
    {
      if ( got == -1 )
      {
        perror("Problem with read()");
        break;
      }
      if ( got == 0)
        break;
    }
    total += got;
  }
  printf("Got %d characters\n", total);
  return 0;
}

entirefile2.c

Code:

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

#define MEG 1048576

int main(void)
{
  char buffer[MEG];
  int total=0;
  FILE *fp;

  fp = fopen("file.txt", "r");
  if ( fp == NULL)
  {
    perror("Couldn't open file");
    return 1;
  }

  while (total < sizeof(buffer) )
  {
    buffer[total]=fgetc(fp);
    if ( buffer[total] == EOF)
    {
      buffer[total]='\0';
      break;
    }
    total++;
  }
  printf("Got %d characters\n", total);
  return 0;
}

Times for entirefile.c
real 0m0.013s
user 0m0.000s
sys 0m0.010s

Times for entirefile2.c
real 0m0.115s
user 0m0.110s
sys 0m0.010s


Granted they are both under 1 second but stil fgetc was a little under 9 times slower.

Thantos, heh, I immediately went and did something simular.

I used time.h clock_t and clock() to get cpu ticks.

Results reading a 1MB file of random junk into a buffer:

fgetc took approx 731 cpu ticks.
fread took approx 20 cpu ticks.

That's almost 40X faster using fread to read in the entire file vs
reading in each byte separately.

... and you don't see a bug in this code?

Which one? The bug that assumes nbytes is really the size you want, the bug that results in a null pointer access if malloc fails or the bug that results in a null pointer access if fopen fails?

Edward, well done, you found three bugs that don't exist.

applause.

nbyes will be the size of the file

True. But will the size of the file be the size that you expected?

But still, you missed another one

Not in your snippet unless I'm going blind (that I wear glasses suggests that this could very well be the case). However, in the original code there appears to be a & hiding where it shouldn't be that I missed in my first glance.

Edward, well done, you found three bugs that don't exist.

Really. Look again until you see at least two of them. I can tolerate the first because it isn't likely to be a problem.

Ok, so we have to assume you omitted error checking for clarity purposes.

Am I the only one that doesn't wear glasses here? I could swear
the second line of my post says "note that error checking has
been omitted" or is this a case of selective blindness?

Which leaves the one bug which both Edward and I have yet to find.

Cat got your tongue?