Thread: Processing binary or plaintext files

  1. #1
    Registered User
    Join Date
    Aug 2006
    Posts
    8

    Processing binary or plaintext files

    Hello,
    I'm working on a little C program which transfers files across a [Unix] network. At the moment, it can handle things like text files just fine, but trying to read and transmit a binary executable (eg. "app1.exe") results in failure/loss of data.

    I was originally using fread and fwrite, but then switched to read() and write() because read() will not go beyond the actual number of bytes remaining in the file, AND because it returns exactly how many bytes were read. IE., if I tell fread to try reading 100 bytes into my char buffer, but there are only 50 remaining in the file, it ended up polluting the buffer with "false" data past index 50. Anyway..

    Is it possible to handle text AND binary files using the same IO process (that is, without coding something like an FTP binary/ascii mode) ?

    Thanks.

  2. #2
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    No it doesn't, you're not using it right. Try paying attention to what fread returns and simply open in "binary mode". (Not that there's a difference on most *nix variants.) Just write exactly the amount you read, and vice versa, and stop writing the entier buffer. Because that's what your problem is. fread returns that it has read 15 bytes, and your buffer is 30, so you just write the whole buffer, and oops, you've written too much.


    Quzah.
    Hope is the first step on the road to disappointment.

  3. #3
    Registered User
    Join Date
    Aug 2006
    Posts
    8

    Same effect

    Alright, so evidently fread works in that manner too -- BUT, it still screws up binary files. Eg. a test file "prog1.exe" of approx. size 13,500 bytes, is read and output to "prog1_copy.exe" with a resulting size of only 800 bytes. Again, text files are 'copied' perfectly. Snippet of reading:

    Code:
    filepointer = fopen("file", "rb");
    result = fread(char_array, sizeof(char), char_array_size, filepointer)
    char_array[result] = '\0'
    Is the char datatype the problem?

  4. #4
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,218
    What exactly is char_array_size set to?

    It could be the char datatype. You should be using an unsigned char buffer instead.

    And are you working across an NFS volume or something? Your original post mentioned a network, but I'm not seeing anything network-related going on here...

    And why are you appending a '\0' to the array after reading?
    Last edited by itsme86; 08-04-2006 at 09:31 AM.
    If you understand what you're doing, you're not learning anything.

  5. #5
    Registered User
    Join Date
    Aug 2005
    Location
    Austria
    Posts
    1,990
    Show the send function.
    Code:
    char_array[result] = '\0'
    makes me think that you might use strlen() in the send function.
    Kurt

  6. #6
    Registered User
    Join Date
    Aug 2006
    Posts
    8
    char_array_size is determined by a commandline option. the array will be initialized to that size+1. I changed it to unsigned char, same result.

    The network code was not relevant to this issue -- outputting the file locally with fwrite() immediately after fread() produces the mentioned corruption.

    Why append a null-byte? Always good to know where your array ends If I don't do that (as when I was misusing the fread return value, I ended up with excess junk).

  7. #7
    Registered User
    Join Date
    Aug 2005
    Location
    Austria
    Posts
    1,990
    Quote Originally Posted by Jags
    Why append a null-byte? Always good to know where your array ends
    With binary data you will have many array-ends in one block.
    Kurt

  8. #8
    Registered User
    Join Date
    Aug 2006
    Posts
    8
    ZuK -- I AM using strlen in fwrite; and in send. Seemed like a good way to determine how much to write?

  9. #9
    Registered User
    Join Date
    Aug 2005
    Location
    Austria
    Posts
    1,990
    Quote Originally Posted by Jags
    ZuK -- I AM using strlen in fwrite; and in send. Seemed like a good way to determine how much to write?
    No it isn't for binary data. you have to pass the returnvalue of fread() to the send function and then there is no need to append the '\0'.
    Kurt

  10. #10
    Registered User
    Join Date
    Aug 2006
    Posts
    8
    Well, Zuk, that did the trick. Now why didn't I think of that... ah, probably because I havent tried binary io in C before! (and hopefully never again lol)

    Thanks

  11. #11
    Gawking at stupidity
    Join Date
    Jul 2004
    Location
    Oregon, USA
    Posts
    3,218
    Also, instead of making the user specify the buffer size, you can do multiple reads/writes until the file is done. Something like:
    Code:
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <errno.h>
    
    int main(int argc, char **argv)
    {
      FILE *fpin, *fpout;
      unsigned char buf[BUFSIZ];
      unsigned int cnt;
    
      if(argc != 3)
      {
        fputs("Usage: mycp <source file> <dest file>\n", stderr);
        exit(EXIT_FAILURE);
      }
    
      if(!(fpin = fopen(argv[1], "rb")))
      {
        fprintf(stderr, "%s: %s\n", argv[1], strerror(errno));
        exit(EXIT_FAILURE);
      }
    
      if(!(fpout = fopen(argv[2], "wb")))
      {
        fprintf(stderr, "%s: %s\n", argv[2], strerror(errno));
        fclose(fpin);
        exit(EXIT_FAILURE);
      }
    
      while((cnt = fread(buf, 1, sizeof(buf), fpin)))
        if(fwrite(buf, 1, cnt, fpout) != cnt)
        {
          fprintf(stderr, "Error while writing: %s\n", strerror(errno));
          break;
        }
    
      if(ferror(fpin))
        fprintf(stderr, "Error while reading: %s\n", strerror(errno));
    
      fclose(fpin);
      fclose(fpout);
    
      return EXIT_SUCCESS;
    }
    If you understand what you're doing, you're not learning anything.

  12. #12
    Registered User
    Join Date
    Aug 2006
    Posts
    8
    Also, instead of making the user specify the buffer size, you can do multiple reads/writes until the file is done.
    True - except I'm not developing this for myself, and the "client" insists on an option to specify byte/block size

  13. #13
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Quote Originally Posted by Jags
    Well, Zuk, that did the trick. Now why didn't I think of that...
    Probably because you ignored what I wrote. I told you you had to use the return value of fread.


    Quzah.
    Hope is the first step on the road to disappointment.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. added start menu crashes game
    By avgprogamerjoe in forum Game Programming
    Replies: 6
    Last Post: 08-29-2007, 01:30 PM
  2. send/recv binary files using sockets in C++
    By dafatdude in forum Networking/Device Communication
    Replies: 14
    Last Post: 07-25-2004, 11:00 AM
  3. MFC: CStrings & binary files question(s)
    By BrianK in forum Windows Programming
    Replies: 0
    Last Post: 06-24-2004, 05:41 PM
  4. Making Binary Output to files using VC++ 6's "fstream"
    By Perica in forum C++ Programming
    Replies: 5
    Last Post: 02-05-2003, 01:48 AM
  5. Binary files in C++
    By Unregistered in forum C++ Programming
    Replies: 3
    Last Post: 09-25-2001, 04:48 PM