Reading a buffer line by line, while using system read

Hello,
the goal of my programe is to read from a log file "A" (comma delimited text file) which is generated when events occurs while the system is running and to extract some fields from that log file "A" and create a comma delimited text file "B" consists of the extracted fields from the parent log file "A".
I have opted the way of "Reading the file in to a buffer" after some seconds. using system I/O "read (filehandle, buffer, buffersize) "
and then extracting the fields from the buffer by getting one line at a time and process/extract the required fields from the line and putting the extracted fields in to the destination file (comma delimited).

I am facing difficulty while reading the buffer line by line so i can extract the required fields from the each line one by one from the buffer when it got filled in a read operation.

Please help me for this, and also comment on effeciency of the programe implemented in this way.

Exactly what difficulty are you having, and what exactly are you doing (post the code).

To comment on the efficiency, we would really have to see the code - there's no way I can tell if you are doing it more or less efficiently than for example using fgetc().

--
Mats

Yes it is difficult to just say, where you are making a mistake or perhaps to talk about the efficiency. But here is an easy way dealing this.

1. When you say records, do see any inconsistency between them in a value like any extra field compared from one records to an another.
2. Cant you read your records from the file, like in chunks and fill it into structure and then access the data members of that structure?
3. How many fields do you have in a record?
4. You can also use fread and fwrite, indeed these two function trap the kernel to call read and write respectively.

ssharish

Dear mats and ssharish, thanks for the quick reply,
1. When you say records, do see any inconsistency between them in a value like any extra field compared from one records to an another.
the fields are consistant in all records that is all records have the same number and type of fields
2. Cant you read your records from the file, like in chunks and fill it into structure and then access the data members of that structure?
I am trying to do so that get the data in chunk and fill the required fields in the structure
3. How many fields do you have in a record?
Each and every record consists of 23 fields, but i am interested in only first 4 to 6 fields that is why i dont want to traverse through all other fields, some times file size reaches to 80MB and sometimes it remains in 600KB, it actually depends on the events occur in the system, i am getting difficulty in fgets (perhaps memory leak or overflow).

Code:

#define BUFFSIZE 512
char Rd_buff[BUFFSIZE];
int main(int argc, char* argv[])
{
int inhandle, outhandle, seek, C_read=0, T_read = 0;
char linestore[512];
char *ptr;
ptr = Rd_buff;
inhandle =  open("sample.txt", O_RDONLY);
seek = lseek(inhandle, C_read, SEEK_SET);
outhandle =  open("Destination.txt", O_CREAT | O_WRONLY | O_APPEND, S_IWRITE);
while( (C_read = read(inhandle, Rd_buff, BUFFSIZE)) > 0){	
T_read = T_read + C_read;
seek = lseek(inhandle, T_read, SEEK_SET);  

		while ( (fgets(linestore, sizeof(linestore), ptr) ) != NULL){
		printf("Line by Fgets  = %s", linestore);
		}       

write(outhandle, Rd_buff, C_read);
}}

Code:

		while ( (fgets(linestore, sizeof(linestore), ptr) ) != NULL){

This line does not work - ptr is a char pointer that points to your buffer, not a pointer to a FILE struct. There is no way you can make this work in the way you have it right now.

If you want to use stdio.h functions, you need to open the file using fopen, and use the pointer to that file. If you want to use the low-level read functions, you need to write your own line-parsing code (and bear in mind that lines may straddle whatever block-size you choose to use). You will probably find it easiest to write a function that reads a character from the buffer, and if the buffer is empty, read in another buffer-full (also bear in mind that the buffer may not be completely full on the last attempt).

--
Mats

What is the effecient way of doing such work either using low-level functions or use of standard functions like fopen, fread etc?
while using the fopen & fread a stream is established, is it effecient to access the file for each character further i want ot access after some time delay for example after each 30 seconds, that is why i adopted for the buffer so that i hav the chunk of data and i can then process it.

There is a principle in programming called "Don't optimize prematurely".

Get the code (all parts of it) working correctly first, then spend time making it run faster.

The EASIEST solution is to just open the file with fopen() and use fgets() to read the file.

Don't quite understand what you are asking for when you talk about time-delay - can you explain what you are trying to achieve?

--
Mats

thank you dear for the help i will come again after writing the function ( reading character from the buffer) while using system calls for filling the buffer.
by time delay i mean "lets suppose that 10 events are recorded per second in the system then i am of view that i will get data of 100 events in the buffer after 10 seconds so read system call will read the file after 10 seconds each time ".
Mean while any good and simple tutorial for Debugging? i am using Visual studio 6.0 for the development.

> Mean while any good and simple tutorial for Debugging? i am using Visual studio 6.0 for the development.
Start by upgrading the compiler

I hear VS Express 2008 is good (and it's free), and comes with a nice debugger.

Originally Posted by Hammad Saleem

thank you dear for the help i will come again after writing the function ( reading character from the buffer) while using system calls for filling the buffer.
by time delay i mean "lets suppose that 10 events are recorded per second in the system then i am of view that i will get data of 100 events in the buffer after 10 seconds so read system call will read the file after 10 seconds each time ".
Mean while any good and simple tutorial for Debugging? i am using Visual studio 6.0 for the development.

If it's console application, you could do something like this:

Code:

while(some_stop_condition)  // E.g. user input, number of loops or some other "way" to stop the app. 
{
    displayStuff();
    Sleep(10000);   // Wait for 10s. 
}

I don't see why you need to mess about with read/write rather than stdio.h functions when you have such a slow delivery of data tho' - it makes absolutely no sense.

--
Mats