Printable View
Hello,
I would like to read a file and store it in an array. But I want to read the whole file, so when I specify the number of bytes to read, I want it to read the whole file. How could I go about reading the whole file instead of just a number of bytes? Thank you!
(Im trying to write something close to a "cat" command in Unix)
Get the size of the file in bytes, with fseek() and ftell() then allocate enough memory to hold the file then read it in with fread().
Or something like http://msdn2.microsoft.com/en-us/library/ms810613.aspx
!= Eof ?
I wrote this but it refuses to work and gives me a segfault all the time.
Code:int fd;
int buffer_size;
char *buffer;
int i;
i = 0;
fd = open("main.c", O_RDONLY);
if (fd == -1)
{
return (-1);
}
buffer_size = lseek(fd, 0, SEEK_END);
buffer = malloc((buffer_size + 1) * sizeof(*buffer));
read(fd, buffer, buffer_size);
}
Check the return value of malloc(), it could be failing. Why not get the source of 'cat' and take a look?
I'd suggest using mmap'd files for this, it seems like a big waste of memory ;)
If you are actually trying to replicate cat, then I would think the following would do:Quote:
Originally Posted by +Azazel+
--Code:int main(int argc, char **argv)
{
int i;
char buffer[1000];
for(i = 1; i < argc; i++) {
FILE *f = fopen(argv[i], "r");
if (!f) {
perror(argv[i]);
return 1;
} else {
while(fgets(buffer, sizeof(buffer), f)) {
fputs(buffer, f);
}
}
}
return 0;
}
Mats
> (Im trying to write something close to a "cat" command in Unix)
Except cat doesn't care about the size of the file (it has no need to). Indeed, cat works with devices and pipes where there is no concept of a file size.
Allocating a massive block of memory just to store the file for a few seconds is just wrong.
Not to mention that I have quite often done something like:Quote:
Originally Posted by Salem
where somefile may well be MUCH larger than the amount of RAM + Swap that my system has [because somefile is some type of logfile that has been running for ages, and built up to many gigabytes]. cat should cope with "cat /dev/hda1", where hda1 is a 250GB hard-disk [obviously it shouldn't be mounted, otherwise you may find it problematic]Code:$> cat somefile|grep sometihing|grep -v soemthingelse
--
Mats
Big thanks.
I modified the code abit to get this:
But in the end of each file I cat, apart from showing the whole file, it gives me these symbols in the end:Code:#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int i;
int fd;
char buffer[1000];
i = 1;
if (argc <= 1)
{
my_putstr("Usage: Please enter 1 or more parameters");
my_putstr("\n");
}
while (i < argc)
{
fd = open(argv[i], O_RDONLY);
if (!fd)
{
my_putstr("Error reading file");
my_putstr("\n");
}
else
{
while (read(fd, buffer, sizeof(buffer)))
my_putstr(buffer);
}
i++;
}
return (0);
}
Ï(ë¿¿(
What could be causing this?
Also if the second file is smaller than the first, it not only shows the second but parts of the first mixed in. I think I need to clear the buffer each time? or?
read() doesn't append a \0.
What you should be using is the return result of read() to tell you how many bytes to write.
I dont really understand. Read returns the number of bytes read. But I need to specify first how many bytes I want to read (which seems a problem to me). Would I choose randomly? Read the first 50 bytes and write those bytes, but then like you said read doesnt append a '\0' so how do I know when to stop?
There are two problems here:
1. You read 1000 bytes at a time, and then print it with "myputstr". Presumably, myputstr prints a string, meaning that the data sent to myputstr would have to be zero-terminated to indicate the size of the string. Currently, there is no code to zero terminate the string, so it continues printing beyond the end of the string until it finds some other zero in the memory.
2. If there is less than 1000 bytes [left] in the file, then you need to NOT print beyond what has been read - that will just be whatever happens to be in your buffer, which is most likely some previously read stuff. Since read gives you the number of bytes ACTUALLY read, it is pretty easy to just stick the zero at the end of the string.
One should of course note that the original cat is not dealing with the content as strings - it is just one long binary stream. It is perfectly valid to do
"cat /boot/vmlinuz|gzip -c - |od -x|grep -n 0x1F".
If you read a binary file, [I presume] your myputstr will stop printing the content of the file when it reaches a zero in the input string - which is not how the original cat works.
--
Mats
I understand, so is there any way to go around this?
Ohhh, nevermind. It works right now :)
Heres the code:
But it refuses to work for more than one parameter.Code:#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int i;
int x;
int b;
int fd;
char buffer[1];
i = 1;
b = 1;
x = 1;
if (argc <= 1)
{
my_putstr("Usage: Please enter 1 or more parameters");
my_putstr("\n");
}
while (i < argc)
{
fd = open(argv[i], O_RDONLY);
if (!fd)
{
my_putstr("Error reading file");
my_putstr("\n");
}
else
{
while (x)
{
x = read(fd, buffer, sizeof(buffer));
lseek(fd, b, SEEK_SET);
my_putstr(buffer);
b++;
}
}
i++;
}
return (0);
}
That is TERRIBLY inefficient [and you shouldn't have a lseek, as that will not work on devices that don't have a concept of filesize (or otherwise can't be "seeked")].
Your original approach is much better - just check how many bytes came back from read(), and set a zero in your string at that point [make your buffer 1001 bytes].
--
Mats
Could you explain why? I thought that would be very memory efficient. I use an array of only size 1 and replace that character each time. I am not a pro so I might be missing something here.
Yes, I don't deny you that it's memory efficient. But since your stack [which is where your local variables are stored] will certainly be at the very least 4KB, I don't think putting 1000 bytes on it would be too much of a "waste".
The ineffecicency I'm referring to is the fact that a single byte read from read() is going to take many hundreds of CPU cycles, whilst a read() of 1000 bytes will probably only add a few hundred to a few thousand cycles [compared to a few hundred thousand cycles to read 1000 bytes as a loop of read() calls]. Each time you make calls into the OS, you spend a large amount of time just transitioning from your application into the OS and back again. If you can amortize that over a reasonable amount of bytes read from a file (or such), then you get a much lower overhead per byte read.
--
Mats
Ah I see now. I redid the program using 1024 bytes and without lseek.
It works right now marvelously. Thank you mats for all your help!Code:#include <unistd.h>
#include <fcntl.h>
int main(int argc, char **argv)
{
int i;
int x;
int fd;
char buffer[1024];
i = 1;
if (argc <= 1)
{
my_putstr("Usage: Please enter 1 or more parameters");
my_putstr("\n");
}
while (i < argc)
{
x = 1;
fd = open(argv[i], O_RDONLY);
if (!fd)
{
my_putstr("Error reading file");
my_putstr("\n");
}
else
{
while (x > 0)
{
x = read(fd, buffer, 1024);
buffer[x] = '\0';
my_putstr(buffer);
}
}
close(fd);
i++;
}
return (0);
}
Actually, you are now setting buffer[1024] = 0 if your file is bigger than 1024 bytes - that is a buffer overflow. Your buffer needs to be (at least) 1025 long.
[Of course, it still cut the file short if there's a zero-byte in there, but I guess most people don't "cat" files other than text - although I've certainly been known to do that - and I mean on purpose, rather than an accidental binary file].
Edit: You also probably should say "Error opening file" rather than "reading".
Edit2: Some further "tidiness" comments:
1) I would use a "for(i = 1; i < argc; i++)" as the loop over arguments.
2) If you use "do ... while(x > 0);" instead of while, you don't need to initialize x to 1.
3) You should check if fd > 0, not if fd != 0 - zero is a valid file-descriptor [technically, fd is an index into a array, and of course, in C, arrays are zero-based, so the first file opened will have fd = 0].
--
Mats