I'm writing a utility to search a very large text file. I only need to search the first few letters of each record to know if it is the one I need.
The file I have to search is a text file by necessity due to other programmes that use it too.
The problem is that the file is so large that I can't read it into a buffer without using up all of the virtual memory in windows. I can't just increase the size of my pagefile because the application must be portable to other workstations in the company without the need for environment changes.
I'm attempting to read in the file in chunks as is neccessary but any way I try this doesn't seem to work. This is what I have so far (excluding any extraneous information of course)
Code:
#define MAX_LGT_EXT 50
#define MAX_NO_EXT 1000
......
int i, j, success=1;
char **buffer, Image[MAX_LGT_EXT], search[15], serial[5], refresh[2];
FILE *fopen(), *file_list;
......
if( ( file_list = fopen("file_images.dat", "r") ) == NULL ){
puts("\nError opening datafile: please ensure file_images.dat is in this directory\n");
system("pwd");
return 1;
}
buffer = malloc( MAX_NO_EXT * sizeof(char *) );
if( buffer == NULL ){
puts("\nBuffer error: out of memory\n");
return 1;
}
for(i=0; i<MAX_NO_EXT; i++){
buffer[i] = malloc( MAX_LGT_EXT * sizeof(char) );
if( buffer[i] == NULL ){
for(j = 0; j < i; ++j) {
free(buffer[j]);
}
puts("\nBuffer error: out of memory\n");
return 1;
}
}
i=0;
while( i<MAX_NO_EXT && (fgets(buffer[i], MAX_LGT_EXT, file_list) ) != NULL ) {
i++;
}
puts("Please wait- searching");
for(j=0; j<i && success!=0; j++){
if( strncmp(buffer[j], "Bill", 4) == 0){
strcpy(Image, buffer[j]);
}
if( strncmp(buffer[j], search, 13) == 0){
printf("File: %s", buffer[j]);
printf("%s", Image);
success=0;
}
if( j==i-1 ){
i=0;
j=0;
while( i<MAX_NO_EXT && (fgets(buffer[i], MAX_LGT_EXT, file_list) ) != NULL ) {
i++;
}
}
}
......
The program compiles fine, but the problem arises when I attempt to run this section of the code: I get three errors-
The first two say-The instruction at "a hex number" referenced memory at "a hex number". The memory could not be 'read'.
The third says- The exception unknown software exception ("a hex number") occurred in the application at location "a hex number"
If you need any more information or code just let me know.
Any assistance anyone could offer would be much appreciated!