copying contents of text file defined at runtime into an array

This is a discussion on copying contents of text file defined at runtime into an array within the C Programming forums, part of the General Programming Boards category; Hi Guys, Having an issue with using command line arguments and data manipulation. The program I am trying to write ...

  1. #1
    Registered User
    Join Date
    Aug 2009
    Posts
    16

    copying contents of text file defined at runtime into an array

    Hi Guys,

    Having an issue with using command line arguments and data manipulation.

    The program I am trying to write allows user to specify input and output file at runtime e.g

    ./suchthefool -input.txt -output.txt

    This is working fine and handling errors, but I need to copy the contents of the input file into a string or a 1d array without knowing the size of the input file.

    I have been searching for a few days and not found anything. If i define the inputfile in the code rather than at runtime i can do it but not when using argc *argv[]
    e.g. char inputstring[] = "input.txt";

    I'm going to leave out my error handling for the command line aguments, will post them if any of you think I might be going wrong in there..

    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
     
    int main(int argc, char *argv[])
    {
            FILE *fptr1, *fptr2;
     
     
            int c;
     
            fptr1 = fopen (argv[1], "r");
            fptr2 = fopen (argv[2], "w");
     
     
            char inputstring[] = "char *argv[1]";
    //////////////////////////////////////////////////////////
    //        strcpy(inputstring, argv[1]); 
    // Tried this and it didnt work either
    /////////////////////////////////////////////////////////
     
            printf("%s\n\nThis is the string\n\n", inputstring);
     
    //////////////////////////////////////////////////////////
    // debug copy from input to output without string
    //      while((c = getc(fptr1)) != EOF)
    //      {
    //              putc(c, fptr2);
    //              putc(c, stdout);
    //      }
     
    //////////////////////////////////////////////////////////
    //      code to print inputfile to screen for debugging
    //      char x;
    //
    //      while ((x = fgetc(fptr1)) != EOF)
    //      {
    //              printf("%c", x);
    //      }
    //////////////////////////////////////////////////////////
     
            fclose(fptr1);
            fclose(fptr2);
     
            return 0;
    }
    I am wanting the contents of input.txt to be copied into

    Code:
    //pseudo
    char inputContents[sizeof/strlen(argv[1])];
    There are so many tutorials on google etc that show me how to do something like

    Code:
    char inputContents[] = "input.txt";
    but this doesn't let me specify the input file when I run the program

    Any help much appreciated!

    Ben

  2. #2
    Registered User
    Join Date
    Aug 2009
    Posts
    16
    I cannot delete this thread from programmingforums.org. I posted it there earlier because I had forgotten I had the account here. Remembered cboard and came here with the problem, not intentionally cross-posting as would have deleted it from there the min I came here If i could.

  3. #3
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    there are two solutatins for your problem

    1)create dynamic array and relocating the array multiple times.
    2)create dynamic array and use unix low level io and use dynamic memory allocation once.

    what is wrong with our code.
    line 16 there is no use.

  4. #4
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    this code continuously reallocates a new space for the string.
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    
    int main(int argc, char *argv[])
    {
            FILE *fptr1;
    
    int i=0;
            char c;
    
            fptr1 = fopen (argv[1], "r");
           // fptr2 = fopen (argv[2], "w"); you ar gong to copy the content to string hence ther is no nead to second file
    
    
    char *string;
    string = (char *)malloc(sizeof(char ));
    
    while((c=getc(fptr1))!=EOF)
    {
        string=(char *)realloc(string,(i+1)*sizeof(char));//reallocates for every input.
        string[i]=c;
        ++i;
    }
            fclose(fptr1);
          //  fclose(fptr2);
    printf("%s",string);
            return 0;
    }

  5. #5
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Character by character? No way... slow, silly and will make swiss-cheese out of memory.

    Take a look at THIS

    At a minimum load a line at a time.
    Realloc in blocks of lines, not character by character.

  6. #6
    Registered User
    Join Date
    Aug 2009
    Posts
    16
    Thank-you guys,

    Let me wake up a bit and take a look at it

    Ben

  7. #7
    Registered User
    Join Date
    Aug 2009
    Posts
    16
    Also, CommonTater

    Is that similar to reading a defined block length? It's not important yet with the code but I think ideally if it could read a user defined block length that would be best.

    Cheers,

  8. #8
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    Reallocate in blocks of lines, not character by character.
    yes you are correct. it is a good idea if the file is too large. but we have to use if condition to check weather the memory is sufficient or not for each time. this again slows down the program.
    all computers today are provided with direct memory access. hence this will not be very slow as you think.

  9. #9
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    hear is the another way with out performing reallocation.
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    
    int main(int argc, char *argv[])
    {
            FILE *fptr1;
    
    int i=0;
            char c;
    
            fptr1 = fopen (argv[1], "r");
           // fptr2 = fopen (argv[2], "w"); you ar gong to copy the content to string hence ther is no nead to second file
    
    // calucate the size of the file.
    fseek(fptr1, 0L, SEEK_END);
    int sz = ftell(fptr1);
    //You can then seek back to the beginning:
    fseek(fptr1, 0L, SEEK_SET);
    
    
    char *string;
    string = (char *)malloc(sz*sizeof(char ));
    
    while((c=getc(fptr1))!=EOF)
    {
       // string=(char *)realloc(string,(i+1)*sizeof(char));//reallocates for every input.
        string[i]=c;
        ++i;
    }
            fclose(fptr1);
          //  fclose(fptr2);
    printf("%s",string);
            return 0;
    }

  10. #10
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by sagar474 View Post
    yes you are correct. it is a good idea if the file is too large. but we have to use if condition to check weather the memory is sufficient or not for each time. this again slows down the program.
    all computers today are provided with direct memory access. hence this will not be very slow as you think.

    Actually, modern computers (i.e. anything since DOS) do not allow direct memory access at the application level. Memory is now virtualized and the "addresses" you see for pointers are merely offsets into the application's Heap. Windows in particular forbids direct memory access outside of an application's own heap.

    All you need to do is check if realloc() fails ...

    Code:
    array = realloc(array, size * sizeof (var));
    if (!array)
      { printf("Memory allocation failed\n\n");
         exit (some_error_number); }
    If you are allocating new string pointers in blocks as my example demonstrates the overhead of this extra check is minimal.

    As I explained in the other thread, I commonly load 100+mb files this way and have never yet had a problem with it.

  11. #11
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by sagar474 View Post
    hear is the another way with out performing reallocation.
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    
    int main(int argc, char *argv[])
    {
            FILE *fptr1;
    
    int i=0;
            char c;
    
            fptr1 = fopen (argv[1], "r");
           // fptr2 = fopen (argv[2], "w"); you ar gong to copy the content to string hence ther is no nead to second file
    
    // calucate the size of the file.
    fseek(fptr1, 0L, SEEK_END);
    int sz = ftell(fptr1);
    //You can then seek back to the beginning:
    fseek(fptr1, 0L, SEEK_SET);
    
    
    char *string;
    string = (char *)malloc(sz*sizeof(char ));
    
    while((c=getc(fptr1))!=EOF)
    {
       // string=(char *)realloc(string,(i+1)*sizeof(char));//reallocates for every input.
        string[i]=c;
        ++i;
    }
            fclose(fptr1);
          //  fclose(fptr2);
    printf("%s",string);
            return 0;
    }
    If you know the file size and are loading the whole file... why don't you just do this...

    Code:
    // calucate the size of the file.
    fseek(fptr1, 0L, SEEK_END);
    int sz = ftell(fptr1);
    //You can then seek back to the beginning:
    fseek(fptr1, 0L, SEEK_SET);
    
    
    data = malloc(sz);
    fread(data ,sz, 1, fptr1);
    Yes... read the whole file in one read operation...

    Seriously, messing with character by character reads and writes is old hat... it's slow, it's cumbersome and it's very CPU intensive.

  12. #12
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by suchthefool View Post
    Also, CommonTater

    Is that similar to reading a defined block length? It's not important yet with the code but I think ideally if it could read a user defined block length that would be best.

    Cheers,
    No... the problem with text files is that nothing is going to line up to your defined block lengths. One line is 20 characters, the next is 80 and so on... so there's no way to predict the size of any block. As I'm showing you your choices are to either load the file line by line (with fgets()) assigning memory for each line as you go (see the link) or to load the file whole-cloth into one allocated block (with fread()).

    This character by character stuff is really a terrible way to read or write a file.

    If you want to see defined blocks (i.e. random access) files in action you can copy-paste and compile the following for a demonstration...

    Code:
    //random access file demo
    #include <stdlib.h>
    #include <stdio.h>
    #include <time.h>
    #include <ctype.h>
    #include <string.h>
    
    #define FNAME "random.dat"
    
    // test data struct
    struct t_Record
      { int number;
        char word[16]; }
      Record;
    
    
    
    ///////////////////////////////////////////////////////
    // Random Access File Handlers
    //
    
    // open or create the file
    FILE *FileOpen(char* Filename)
      { FILE* pFile;
        pFile = fopen(Filename,"rb+");
        if (!pFile)
          pFile = fopen(Filename,"wb+");
        return pFile; }
    
    
    // Write a record to the file
    int WriteRecord(FILE *File, int RecNum)
      { if( fseek(File, RecNum * sizeof(Record), SEEK_SET) == 0 )
          if ( fwrite(&Record,sizeof(Record),1,File) )
            return 1;
        return 0; }
    
    
    // read a record from the file
    int ReadRecord(FILE *File, int RecNum)
      { if( fseek(File, RecNum * sizeof(Record), SEEK_SET) == 0 )
          if ( fread(&Record,sizeof(Record),1,File) )
            return 1;
        return 0; }
    
    
    int AddRecord(FILE *File)
      { fseek(File,0,SEEK_END);
        fwrite(&Record,sizeof(Record),1,File);
        return (ftell(File) / sizeof(Record)) - 1; }
    
    
    
    //////////////////////////////////////////////////////////////
    // View a Record
    //
    
    int ViewRecord (FILE *File, int RecNum)
      { if (! ReadRecord(File,RecNum))
          { printf("Invalid record\n"); 
            return -1; }
        printf("-----\n");
        printf("Record        : %d\n",RecNum);
        printf("Number Value  : %d\n",Record.number);
        printf("Word Value    : %s\n",Record.word);
        printf("-----\n");  
        return RecNum; }
    
    
    
    //////////////////////////////////////////////////////////////
    // Add a new record
    //
    
    int AddNewData(FILE *File)
      { memset(&Record,0,sizeof(Record));
        printf("\nEnter a number : ");
        scanf("%d", &Record.number);
        printf("Enter a word : ");
        scanf(" %s",Record.word);
        return AddRecord(File); }
    
    
    
    //////////////////////////////////////////////////////////////
    // Edit a record
    //
    
    int EditRecord(FILE *File, int RecNum)
      { if (! ReadRecord(File,RecNum))
          { printf("Invalid record\n");  
            return -1; }
        printf("\n-----\n");
        printf("Record        : %d\n",RecNum);
        printf("Number Value  : %d\n",Record.number);
        printf("Word Value    : %s\n",Record.word);
        printf("-----\n");  
        
        do
          { while(getchar() != '\n');
            printf("Change Values: Number, Word or Save (N, W or S) ? ");
            switch (toupper(getchar()))
              { case 'N' :
                  printf("\nEnter new number : ");
                  scanf("%d",&Record.number);
                  break;
                case 'W' : 
                  printf("Enter new word : ");
                  scanf(" %15s", Record.word);
                  break;
                case 'S' :
                  if (WriteRecord(File,RecNum))
                    printf("\nRecord #%d updated\n",RecNum);
                  return RecNum; } }
        while(1);
        return -1; }
    
    
    ////////////////////////////////////////////////////////////////
    // List records
    // 
    
    void ListRecords(FILE *File )
      { int i = 0;
        printf("\nRecord     Number\tWord\n\n");
        while (ReadRecord(File,i))
          { printf("%3d%16d\t%s\n",i,Record.number,Record.word); 
            i++; }
        printf("\n\n"); }
    
    
    
    ////////////////////////////////////////////////////////
    // this is for demonstration purposes only
    // you would not do this in a real program
    void InitFile(FILE* File)
     { int x, y;
       memset(&Record,sizeof(Record),0);
       for (x = 0; x < 10; x++)
          { Record.number = rand();
            for (y = 0; y < ((Record.number % 15) + 1); y++)
              Record.word[y] = (rand() % 26) + 'a';
            Record.word[y] = 0;
            if (! WriteRecord(File,x))
              printf("Oh drat!");  } }
     
    
    
    //////////////////////////////////////////////////////////
    // program mains
    //
    int main (void)
      { int Rec = 0; // record number
        FILE *File;
    
        srand(time(NULL));
    
        File = FileOpen(FNAME); 
        if (!File)
          { printf("Curses foiled again!\n\n");
            exit(-1); }
    
        printf("Random Access File Demonstration\n\n");
     
        do
          { printf("Menu : Dummy, Add, Edit, View, List, Quit (D, A, E, V, L or Q) : ");
            switch(toupper(getchar()))
              { case 'D' :
                  printf("Creating dummy file of 10 entries\n");
                  InitFile(File);
                  break;
                case 'A' :
                  Rec = AddNewData(File);
                  printf("Record #%d Added\n\n", Rec);
                  break;              
                case 'E' :
                  printf("\nRecord number (-1 Cancels): ");
                  scanf("%d",&Rec);
                  if (Rec > -1)
                    EditRecord(File,Rec);
                  break;
                case 'V' :
                  printf("\nRecord number (-1 Cancels): ");
                  scanf("%d",&Rec);
                  if (Rec > -1)
                    ViewRecord(File,Rec);
                  break;
                case 'L' :
                  ListRecords(File);
                  break;
                case 'Q' :
                  fclose(File);
                  return 0; } 
                  
             while(getchar() != '\n'); }
        while (1); 
        return 0; }

  13. #13
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    you can also use this instead of while loop and char by char.

    Code:
    fread(string,sizeof(char),sizeof(char)*sz,fptr1);
    string[sz]='\0';
       fclose(fptr1);

  14. #14
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Quote Originally Posted by sagar474 View Post
    you can also use this instead of while loop and char by char.

    Code:
    fread(string,sizeof(char),sizeof(char)*sz,fptr1);
    string[sz]='\0';
       fclose(fptr1);
    I routinely load entire text files into memory with one read call then go along using strtok() and build an array of pointers to the first character in each line. Since strtok() replaces it's search characters with nulls, you end up with an array of pointers to valid C-Strings.

    When you do this character by character you are working at the disk drives *unbuffered* speed as it needs to load 1 character at a time for you... this is very slow. If you are processing a multi-megabyte file it can take several minutes... even hours.

    Line by line, as I described earlier is better, but still not braggable...

    When you just read the whole file into memory you get the OS buffers optimizing the operation and can often load huge files in under a second. Then you capitalize on the very high speed of memory (thousands of times faster than a disk drive) to do the post-load processing in the blink of an eye.

    Last fall I did a job converting .m3u playlist files in various unicode formats to utf8 (internet standard) unicode. There were approximately 10,000 files in a couple of dozen folders. Each file had an average of about 100 lines for an average size of 5k. Each line had to be validated as a UNC path in file shares on the host network, or discarded and then converted to utf8 unicode. Lots and lots of processing. My little proggy, using more or less what I've outlined ran through and did the job in a hair over 2 hours. If I had used character by character processing, it would have taken a full day, or more.
    Last edited by CommonTater; 11-09-2011 at 09:40 AM. Reason: fix minor math error

  15. #15
    Registered User sagar474's Avatar
    Join Date
    Jun 2011
    Location
    Kakinada
    Posts
    56
    When you do this character by character you are working at the disk drives *unbuffered* speed as it needs to load 1 character at a time for you... this is very slow. If you are processing a multi-megabyte file it can take several minutes... even hours.

    Stream I/O is BUFFERED: That is to say a fixed ``chunk'' is read from or written to a file via some temporary storage area (the buffer). NOTE the file pointer actually points to this buffer.
    it is explained hear Input and Output (I/O):stdio.h

    even when we read char by char we are not working on disk drives.
    Name:  stream.gif
Views: 1255
Size:  2.6 KB

    if you really need to read directly form disk you can use Low Level I/O.
    Last edited by sagar474; 11-09-2011 at 10:30 AM.

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Printing contents of a text file
    By never_lose in forum C Programming
    Replies: 10
    Last Post: 04-28-2011, 09:25 AM
  2. pass contents of text file to 2d array
    By darksifer in forum C Programming
    Replies: 0
    Last Post: 11-22-2010, 11:45 AM
  3. need to read a User defined file during runtime
    By john_newbie in forum C Programming
    Replies: 8
    Last Post: 03-06-2004, 01:08 AM
  4. displaying contents of a text file
    By Unregistered in forum C Programming
    Replies: 3
    Last Post: 02-26-2002, 01:05 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21