Need help with loading and extracting bytes from a packet -network byte order trouble

This is a discussion on Need help with loading and extracting bytes from a packet -network byte order trouble within the C Programming forums, part of the General Programming Boards category; Hi guys, I'm relatively new to C programming, and am having a hard time with the network byte order stuff. ...

  1. #1
    Registered User
    Join Date
    Jun 2011
    Posts
    5

    Need help with loading and extracting bytes from a packet -network byte order trouble

    Hi guys, I'm relatively new to C programming, and am having a hard time with the network byte order stuff. It seems like it should be simple, htons/htonl before writing, and ntohs/ntohl when reading. And yet I run into problems. I think it's the way I am handling my bytes. Can you guys look at my approach and see if there's something that raises any flags with you?

    Packing bytes for SENDING
    I'm implementing a protocol over TCP. And I want to send a packet that is 6 bytes.
    format: [4 bytes for IP address] [2 bytes for TCP PORT]

    Code:
    // variables I want to load into packet
    
    //addr is a struct sockaddr_in, filled in via getsockname(), getaddrinfo(), etc
    uint32_t state_to_load = htonl(addr.sin_addr.s_addr);
    uint16_t length_to_load = htons(addr.sin_port)
    
    char packet[6];
    uint16_t *state = (uint32_t *)(pack);
    uint16_t *len = (uint16_t *)(pack + 4);
    *state = state_to_load;
    *len = length_to_load;
    write(fd, packet, 6);
    Extracting Bytes after READING
    format: [4 bytes for IP address] [2 bytes for TCP PORT]
    Code:
    char packet[6];
    read(fd, packet, 6);
    
    int extracted_state = (packet[0] << 24) | (packet[1] << 16) | (packet[2] << 8) | packet[3];
    int extracted_length = (packet[4] << 8) | packet[5];
    I extract data from packets different from how I pack them in, because I can't think of a good way to pack them. I heard that extracting bytes this way avoids network byte order.

    From your expert opinions, is this a good way to pack and extract bytes for sending/receiving?

  2. #2
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Why does "packet" become "pack"? Also your state is schizophrenic about 16 v. 32.

    Why do you talk about ntohl and ntohs at the beginning, and then at the end say "I can't think of a good way to pack them"? Isn't that what ntohl and ntohs are for?

  3. #3
    Algorithm Dissector iMalc's Avatar
    Join Date
    Dec 2005
    Location
    New Zealand
    Posts
    6,308
    Here' the reverse of that unpacking:
    Code:
    char packet[6] = {(state >> 24) & 0xFF, (state >> 16) & 0xFF, (state >> 8) & 0xFF, state & 0xFF,
                      (length >> 8) & 0xFF, length & 0xFF};
    Yes this is a good way to do it.
    My homepage
    Advice: Take only as directed - If symptoms persist, please see your debugger

    Linus Torvalds: "But it clearly is the only right way. The fact that everybody else does it some other way only means that they are wrong"

  4. #4
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    Code:
    union u
    {
        char packet[6];
        struct {
            unsigned char s0, s1, s2, s3;
            unsigned char l0, l1;
        } oneway;
        struct {
            unsigned char s3, s2, s1, s0;
            unsigned char l1, l0;
        } theotherway;
    } packetin, packetout;
    
    read( here, &packetin.packet, 6 );
    packetout.theotherway.s0 = packetin.oneway.s0;
    packetout.theotherway.s1 = packetin.oneway.s1;
    packetout.theotherway.s2 = packetin.oneway.s2;
    packetout.theotherway.s3 = packetin.oneway.s3;
    packetout.theotherway.l0 = packetin.oneway.l0;
    packetout.theotherway.l1 = packetin.oneway.l1;
    Unions are fun. Assuming that's the byte order you want. Otherwise mix the numbers around.


    Quzah.
    Last edited by quzah; 07-27-2011 at 01:53 AM. Reason: brb food
    Hope is the first step on the road to disappointment.

  5. #5
    Registered User
    Join Date
    Jun 2011
    Posts
    5
    Thanks for your replies! Sorry about the bad code examples. Those are typos. I was trying to whip up some quick samples highlighting my problem. My actual code is pretty bloated currently, and there are all sorts of packets - and I'm extracting/packing bytes all the time, etc.

    Thing is, when I wrap one side in htonl() before sending, and then extract it on the other side with ntohl(), I get different values. I think this is because of the way I pack them too. So I remove htonl or ntohl from one of the sides, and then it works. Only I don't know why, since I thought you needed both. Stretch this over a long project, where I'm constantly flipping htonl/ntohl off/on to get the right values, and that's my current mess. Somewhere, something was packed incorrectly - and I don't know where.

    iMalc, if I pack bytes into a buffer your way before writing, does that negate the need for the htonl/htons?

    Sorry, to clarify, I can't assume anything about the Endianess of the machines. They could be different across server/client. So is using ntohl / htonl unavoidable?
    Last edited by Jupiter; 07-27-2011 at 01:35 PM.

  6. #6
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,676
    You're packing the IP address and port into your packet. Are you sure they're not already in network byte order from getsockname(), getaddrinfo(), etc? htons/l and ntohs/l don't know what the current byte order is. They just always switch the byte order so if you were in network, and call htons/l, you're now in host, so the corresponding ntohs/l will re-reverse it (assuming, of course, that host byte order != network byte order -- if they're the same, the functions have no effect). Removing one of them will make it appear to work, but the real issue is to only call htons/l if the data is not yet in network byte order.

  7. #7
    Registered User
    Join Date
    Jun 2011
    Posts
    5
    I see. So how does one know if the data is not yet in network byte order?

  8. #8
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Quote Originally Posted by Jupiter View Post
    I see. So how does one know if the data is not yet in network byte order?
    Where did you get the data from? If you got it from a function, read the documentation for the function. If you just did, say,
    Code:
    short port_number = 42;
    then you know the port number is in host order, and hton will convert it to network order as necessary.

  9. #9
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,676
    Well, generally, an short/int wont ever be in network order until you say so:
    Code:
    char packet[10];
    int x = 42;  // plain old int -- this is stored in host order of course
    uint32_t y = htonl(x);  // switch it to network order
    memcpy(packet, &y, sizeof(y));  // stuff it in a packet for sending
    But your address and port actually come from a specific network function, so it may have already put the bytes in the right order for you. I can't actually tell how you obtained the info in addr since you didn't post that code. Also, I didn't see anything about byte order when glancing through the man page for getaddrinfo, but you should take a thorough look through there and through the man page of any other relevent functions and see if it mentions byte order. You could also do a quick test by stepping through your code in GDB or by memcpy'ing the addr.sin_addr.s_addr and sin_port to a char buffer and printing out each byte in order to verify.

    EDIT: Too slow! Great minds think alike though...

  10. #10
    Registered User
    Join Date
    Jun 2011
    Posts
    5
    Thanks guys! This is starting to make more and more sense! anduril462, to further elaborate on your example, after I write it out, how should I extract it?

    Code:
    char packet[10];
    int x = 42;  // plain old int -- this is stored in host order of course
    uint32_t y = htonl(x);  // switch it to network order
    memcpy(packet, &y, sizeof(y));  // stuff it in a packet for sending
    write(fd1, packet, 10);
    
    // now assume we want to read
    
    char read_buf[10];
    read(fd2, read_buf, 10);
    
    //TODO: How should I get the x/y int from this packet?

  11. #11
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    You can use memcpy to move things from read_buf back to y (assuming you know that it's one int worth of data), and then you can use ntoh to get it back to host order.

  12. #12
    Registered User
    Join Date
    Jun 2011
    Posts
    5
    Thanks guys! This has been illuminating.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Extracting data from byte buffers
    By WDT in forum C# Programming
    Replies: 8
    Last Post: 05-16-2009, 08:00 PM
  2. Complex struct organization (network/packet-related)
    By brooksbp in forum C Programming
    Replies: 6
    Last Post: 08-16-2008, 11:38 PM
  3. Extracting info from a packet trace file
    By Ho ming in forum C Programming
    Replies: 1
    Last Post: 03-31-2008, 12:54 PM
  4. byte order change
    By onebrother in forum C Programming
    Replies: 1
    Last Post: 08-06-2007, 06:40 AM
  5. byte and bit order question
    By chunlee in forum C Programming
    Replies: 7
    Last Post: 11-07-2004, 01:50 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21