Thread: dump the buffer read from binary file

  1. #1
    Registered User
    Join Date
    Nov 2009
    Posts
    46

    Exclamation dump the buffer read from binary file

    Sup guys,

    I am reading the binary file to the buffer(char array) and then want parse it to this struct array.
    Code:
    typedef struct
    {
      char key[4], name[20];
      int mark;
    }rec;
    int main()
    {
       rec recs[10];
       .........
    }
    when parsing, i read characters one by one to fill 'key' and 'name' fields, but how do i deal with integer?
    int is 4 bytes. so when i read the content of the binary file to char array(buffer) that integer is stored byte by byte in that array.
    How do I get that integer from buffer?

    Thanks!

    ps. or is there a way to read directly to the struct array?

  2. #2
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    You can scanf() it directly to the struct member, but fscanf() needs strictly compliant data to the format you request with it.

    You can also get it from the string buffer with sscanf(), <just scanf() from a string >

    You could maybe use something like memmove:

    Code:
    memmove   Copies a block of n bytes from src to dest.
    
     Syntax:
       void *memmove(void *dest, const void *src, size_t n);
    
     Prototype in:
     mem.h   string.h
    
     Remarks:
    memmove copies a block of n bytes from src to dest.
    
    Even when the source and destination blocks overlap, bytes in the
    overlapping locations are copied correctly.
    
     Return Value:
    memmove returns dest.
    
     Portability:
    memmove is available on UNIX System V systems and is compatible
    with ANSI C.
    
     Example:
    #include <string.h> 
    #include <stdio.h>
    
     int main(void)
     {
       char *dest = "abcdefghijklmnopqrstuvwxyz0123456789";
       char *src = "******************************";
       printf("destination prior to memmove: %s\n", dest);
       memmove(dest, src, 26);
       printf("destination after memmove:    %s\n", dest);
       return 0;
     }
    I'm not very familiar with memmove, but when I read "4 bytes", in your post, it came to mind, right away.

    What do you think?

  3. #3
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Quote Originally Posted by Adak View Post
    Code:
       char *dest = "abcdefghijklmnopqrstuvwxyz0123456789";
       char *src = "******************************";
       printf("destination prior to memmove: %s\n", dest);
       memmove(dest, src, 26);
       printf("destination after memmove:    %s\n", dest);
       return 0;
     }
    I'm not very familiar with memmove, but when I read "4 bytes", in your post, it came to mind, right away.

    What do you think?
    I think you're not allowed to modify string literals, that's what I think.


    Quzah.
    Hope is the first step on the road to disappointment.

  4. #4
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    Quote Originally Posted by quzah View Post
    I think you're not allowed to modify string literals, that's what I think.


    Quzah.
    Well, you'd think wrong in the case of Turbo C. This program is part of the help file from Turbo C, and I tested it, just in case. AFAIK, string literals can be put into read only memory or not, depending on the compiler, OS, hardware architecture, and phase of the moon.

    The odd thing to me, is that the source string - src, remains unchanged after the call to memmove().

    I thought it would be 26 char's shorter. So what's the difference between memmove() and memcpy() then?

    Inquiring minds want to know!
    Last edited by Adak; 11-21-2009 at 06:12 PM.

  5. #5
    C++ Witch laserlight's Avatar
    Join Date
    Oct 2003
    Location
    Singapore
    Posts
    28,413
    Quote Originally Posted by Adak
    Well, you'd think wrong in the case of Turbo C. This program is part of the help file from Turbo C, and I tested it, just in case. AFAIK, string literals can be put into read only memory or not, depending on the compiler, OS, hardware architecture, and phase of the moon. p
    Sure, but the undefined behaviour can be easily fixed by changing dest to be an array instead of a pointer. src should be changed to be a pointer to const char for good measure.

    Quote Originally Posted by Adak
    So what's the difference between memmove() and memcpy() then?
    memmove() can be safely used on overlapping buffers whereas memcpy() cannot.
    Quote Originally Posted by Bjarne Stroustrup (2000-10-14)
    I get maybe two dozen requests for help with some sort of programming or design problem every day. Most have more sense than to send me hundreds of lines of code. If they do, I ask them to find the smallest example that exhibits the problem and send me that. Mostly, they then find the error themselves. "Finding the smallest program that demonstrates the error" is a powerful debugging tool.
    Look up a C++ Reference and learn How To Ask Questions The Smart Way

  6. #6
    Registered User
    Join Date
    Nov 2009
    Posts
    46
    I think I better post the whole code, for you guys to get the idea of whats reall going on.
    here we go:
    Code:
    #include <stdio.h>
    #include <string.h>
    #include <stdlib.h>
    
    typedef struct
    {
    	char key[4], Name[21];
    	int Mark;
    }rec;
    void merge(rec * rm, rec * r1, rec * r2, int n1, int n2);
    void printRecs(rec * r, int n);
    void parseF(char * filename, rec * r, int *n);
    
    int main(void)
    {
    	int i, n1=0, n2=0;
    
    	rec recs1[10], recs2[10], recs[20];
    
    	char s[27];
    	
        	parseF("bin1", recs1, &n1);
            parseF("bin2", recs2, &n2);
    	merge(recs, recs1, recs2, n1, n2);
    	printRecs(recs, n1+n2);
    		
    	return 0;
    }   
    void parseF(char* filename, rec *r, int *n)
    {
    	FILE *f;
    	char *buffer;
    	unsigned long fileLen;
    	int i, ri, part, ci;
    
    	f = fopen(filename, "rb");
    	if ( f == NULL )
    	{
    		printf("Cannot open record file %s", filename);   
    		exit(1);
    	}
    	fseek(f, 0, SEEK_END);
    	fileLen = ftell(f);
    	fseek(f, 0, SEEK_SET);
    	
    	buffer = (char*)malloc(fileLen+1);
    	if (!buffer)
    	{
    		printf("Cannot allocate memory");
    		fclose(f);
    		exit(1);
    	}
    	fread(buffer, fileLen, 1, f);
    	//r = (rec*)malloc(sizeof(rec)*fileLen/27);
    	i  = 0;
    	ri = 0; // record count
    	ci = 0;; // character count;
    	part = 0;
    	while(i<fileLen)
    	{
    		if(buffer[i]=='\0') {continue;}
    		if (i%27==0)
    		{
    			ci=0;
    			part = 0;
    			ri++;
    		}
    		if (i%27==3)
    		{
    			ci=0;
    			part = 1;
    		}
    		if (i%27==23)
    		{
    			ci=0;
    			part = 2;
    		}
    		if (part == 0)
    		{
    			printf("%c\n", buffer[i]);
    			r[ri].key[ci++] = buffer[i];
    		} else if (part == 1)
    		{
    			printf("%c\n", buffer[i]);
    			r[ri].Name[ci++] = buffer[i];
    		} else if (part == 2)
    		{
    			// ??????
    		}
    		i++;
    	}
    	fclose(f);
    }
    void merge(rec * rm, rec * r1, rec * r2, int n1, int n2)
    {
    	int i1=0, i2=0, i=0;
    	while (i1<n1 && i2<n2)
    	{
    		if (r1[i1].Mark < r2[i2].Mark)
    		{
    			strcpy(rm[i].key, r1[i1].key);
    			strcpy(rm[i].Name, r1[i1].Name);
    			rm[i].Mark = r1[i1].Mark;
    			i1++;
    		}
    		else if (r1[i1].Mark > r2[i2].Mark)
    		{
    			strcpy(rm[i].key, r2[i2].key);
    			strcpy(rm[i].Name, r2[i2].Name);
    			rm[i].Mark = r2[i2].Mark;
    			i2++;
    		}
    		else
    		{
    			if (strcmp(r1[i1].Name, r2[i2].Name)<1)
    			{
    				strcpy(rm[i].key, r1[i1].key);
    				strcpy(rm[i].Name, r1[i1].Name);
    				rm[i].Mark = r1[i1].Mark;
    				i1++;
    			}
    			else
    			{
    				strcpy(rm[i].key, r2[i2].key);
    				strcpy(rm[i].Name, r2[i2].Name);
    				rm[i].Mark = r2[i2].Mark;
    				i2++;
    			}
    		}
    		i++;
    
    	}
    	while (i1<n1){ rm[i].Mark = r1[i1].Mark; i1++; i++;}
    	while (i2<n2){ rm[i].Mark = r2[i2].Mark; i2++; i++;}
    
    }
    void printRecs(rec * r, int n)
    {
    	int i;
    	for (i=0;i<n ;i++ )
    	{
    		printf("%3s  %20s %4d\n", r[i].key, r[i].Name, r[i].Mark);
    	}
    }
    What my code is supposed to do:
    1)Reads two binary files which contain [key][Name][Mark] records which are sorted by [Mark] field
    2)puts them in two different struct arrays
    3)merges them, so we get third sorted array of structs
    4)prints it

    In parseF method there is commented line with question marks... that's the place where I am stuck...

    One more thing:

    records in bin file are stored in this way
    key Name Mark
    [3 bytes][20 bytes][4 bytes]
    - no delimeters between the fields
    - the Name field which is 20 chars long can be less than 20 chars and ends with null character, the rest of the bytes can contain anything and
    (# bytes before null)+(# the rest of bytes) = 20

    Any suggestions are welcome!

  7. #7
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    If the input file is strictly formatted, then use fscanf() - that's what it was made for.

    For instance, I wrote a program to build some basic stats for my folding team (Folding@Home). Every 3 hours I can get a data file from Stanford's FAH lab servers.

    It's a binary file, with their name, team number, total points, total work units, etc., for about 60,000 ?? folders.

    Since they send out the data file with exactly the same format each time, I use fscanf(), with no problems, and separate out my teammates 7,100 folders, and put the active one's right into an array of structs. Unfortunately, I have to sort through a large amount of other folders first, so an intermediate data file has to be used.

    This program was written in Turbo C, so making an array of 60k structs is out of the question. (600-700 structs is the most I've been able to get out of Turbo C in Windows XP).

    From there the arrays of structs are re-sorted by a different key, and various reports are generated, ready for posting on our team's forum software.

    The whole thing takes about 10 seconds. I'll post up some code for a part of the program.

    This is bringing in a maximum amount of 64 char's for the persons folding handle, and just their current points, which is an unsigned long.

    Code:
       k = 0;
       do   {    
          i = fscanf(innew, "%64s %U", folders[k].nfield, &folders[k].pfield);     
          k++;   
        }while (i > 0);
    This is one report for my team, showing it's output:
    The Milestone Thread 4.0 - Overclockers Forums

    All the colors, formatting, underlining, etc,, are all made by the program.

    This is the second report it generates:
    Our Monkeys Are Even Doing It! - Overclockers Forums

    A bit splashier.

    if you'll post up 10 lines of actual input from the file (and if the data is strictly formatted!), I'll post up the code to put it directly into a struct.

    Please don't give me a description. I don't care about the description beyond a passing interest. I need actual data and I'll see what it needs from there.

    No offense, but I don't trust descriptions from people who don't know how to use fscanf(), no more than I trust descriptions of potholes in the road, from the blind.
    Last edited by Adak; 11-21-2009 at 07:06 PM.

  8. #8
    Registered User
    Join Date
    Nov 2009
    Posts
    46
    I'll post up some code for a part of the program.
    That would be awesome, appreciate that!

  9. #9
    Registered User
    Join Date
    Nov 2009
    Posts
    46
    I can't put the contents of binary file because there are some characters which cannot be displayed in simple text editor. for example null character '\0'
    so, ok here is the example of bin file with '\0':
    Code:
    AAAJohn Doe\0abcdefghijk1BBBJack Doe\0abcdefghijk2CCCBenn Doe\0abcdefghijk3

  10. #10
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    What I need is a true binary file - everything just the same.

    You can post it at Swoopshare, (a file depot website) and then post the link, here.

  11. #11

  12. #12
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Quote Originally Posted by Adak View Post
    Well, you'd think wrong in the case of Turbo C. This program is part of the help file from Turbo C, and I tested it, just in case. AFAIK, string literals can be put into read only memory or not, depending on the compiler, OS, hardware architecture, and phase of the moon.
    Ah, so you're advocating modifying string literals because you might get lucky. Clearly you should be giving advise here...


    Quzah.
    Hope is the first step on the road to disappointment.

  13. #13
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    Quote Originally Posted by quzah View Post
    Ah, so you're advocating modifying string literals because you might get lucky. Clearly you should be giving advise here...


    Quzah.
    I made no statement advocating modifying string literals. That was the example by Borland.

    As was stated by laserlight, memmove (you do remember memmove() was the subject of my post, right?) works on char arrays.

    Clearly, you have no substantive point to make, and you should learn to spell "advice".

  14. #14
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Quote Originally Posted by Adak View Post
    I made no statement advocating modifying string literals.
    You wrote:

    "You could maybe use something like memmove:"

    Then illustrated its use by modifying a string literal. You didn't qualify your example as:
    Quote Originally Posted by Adak View Post
    That was the example by Borland.
    Therefore, it's not beyond reason that I tell you it's incorrect. Furthermore, you followed up with a reply, not acknowledging that it was incorrect, but rather said that it "works for me".

    The difference here is that I'm not on an English forum, teaching the English language.


    Quzah.
    Hope is the first step on the road to disappointment.

  15. #15
    Registered User
    Join Date
    Sep 2006
    Posts
    8,868
    Quote Originally Posted by quzah View Post
    You wrote:

    "You could maybe use something like memmove:"

    Then illustrated its use by modifying a string literal. You didn't qualify your example as:Therefore, it's not beyond reason that I tell you it's incorrect. Furthermore, you followed up with a reply, not acknowledging that it was incorrect, but rather said that it "works for me".

    The difference here is that I'm not on an English forum, teaching the English language.


    Quzah.
    A "maybe" is *NOT* the same thing as a recommendation. "What do you think?" is not advocating an action, it is asking a question.

    Turbo C has been modifying string literals for more than 20 years for me. You want me to lie about it?

    Laserlight already posted the change needed to use memmove() on a string.

    Instead of trying to make a point that has already been made and solved, why not try and post up a nice suggestion or code example for r00t?

    I'm much too old to be impressed by how correct you think you are.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. read a file of commands and execute them
    By surlyTomato in forum C Programming
    Replies: 5
    Last Post: 08-20-2009, 11:32 AM
  2. Can we have vector of vector?
    By ketu1 in forum C++ Programming
    Replies: 24
    Last Post: 01-03-2008, 05:02 AM
  3. efficient binary file read
    By Marv in forum C Programming
    Replies: 15
    Last Post: 11-19-2007, 04:42 PM
  4. C++ std routines
    By siavoshkc in forum C++ Programming
    Replies: 33
    Last Post: 07-28-2006, 12:13 AM
  5. Replies: 3
    Last Post: 03-04-2005, 02:46 PM