Thread: write() and read() int values from socket returning wrong values.

  1. #1
    Registered User
    Join Date
    Nov 2012
    Posts
    4

    write() and read() int values from socket returning wrong values.

    I have tried writing to a socket using write() command. The data I need to write is of structure type and it contains both char and int datatype values. But when I write the data to the server socket, the value of the int variable is getting corrupted. I am receiving a value as 16777216 always.
    I am writing from a Solaris machine to a Linux machine. Solaris is a 32 bit machine and Linux is a 64 bit machine. Do this have any impact on my problem.
    I used htonl/ntohl and vice versa for passing the integer values from and to the server. But how can I use this if I am writing the whole structure type into the socket?? Can anyone help me to sort this out??

    PCHAR data = NULL;
    int size = sizeof(WSMsg);
    int retry = 10;
    data = new char [sizeof(WSMsg)];
    memcpy(data, msg, sizeof(WSMsg));
    int n = write(fd, &data[0], size);

    This is my code part. Here WSMsg is of structure type and has int and char type values.

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,656
    > I used htonl/ntohl and vice versa for passing the integer values from and to the server. But how can I use this if I am writing the whole structure type into the socket?
    You can't send a whole structure "as is", because of data size and alignment issues (not to mention, endian issues).

    You have to use htonl/ntohl etc on each member of the struct separately.
    You can then pack each result into a char array, using memcpy, before sending the whole thing to the peer.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  3. #3
    Registered User
    Join Date
    Nov 2012
    Posts
    4
    Thanks Salem for your reply.

    Actually I am using this to communicate between applications on two different servers(on Linux and Solaris.). I am passing the structure which is holding the database values[including numbers and characters]. This is used in many places in my application for passing the information.

    When I transfer the data, only my integer vales are getting corrupted. Also I am using this to transfer objects corresponding to database structures across two applications. So it is not practically possible for me to convert each and every value using htonl/ntohl etc.. Is there any other way by which I can sort out this issue?

  4. #4
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,656
    There might be a specific fix, if you can post a specific example (and mention which compiler(s) you're using).

    But if you want the problem to go away permanently, then there is no other option than to serialise each member.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  5. #5
    Registered User
    Join Date
    Nov 2012
    Posts
    4
    I am writing a particluar row from the database table which contains number, char type values. And I am writing different rows of different tables like this. So if I need to use htonl/ntohl then I need to convert all of these numbers to network bytes. As I have 20 or more tables and its structures which is having integer type values, it will be a time consuming job for me to convert all of these and send across.

    I am using CC compiler in Solaris[SunOS 5.9 Generic_122300-10 sun4u sparc SUNW,Sun-Fire-V210] and g++ compiler in Linux[Linux 2.6.18-194.el5 #1 SMP x86_64 x86_64 x86_64 GNU/Linux].

  6. #6
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    In a well-designed program, it should not be that time consuming. Boring, crappy and tedious yes, but time consuming, not so much. These sorts of mundane tasks are to be expected in any sizable project. It would have taken less time to write your serialization/deserialization routines than the time you spent waiting for replies on this board. Unfortunately there is no other way around it. When dealing with multiple architectures, you must deal with size and endian issues.

    SPARC is big-endian, x86_64 is little endian. You must use an agreed upon byte order for the two machines to talk to each other sensibly. Network byte order (big-endian) is fine, so stick with htonl/ntohl and the like.

    Also, I believe your SPARC is 64-bit, but I'm not sure. It would be a good idea to use fixed-width types (int32_t, uint64_t, etc) instead of the typical int and long, since they may be different sizes on different architectures/implementations. For example, on a 32-bit machine, an int is typically 32 bits, but on a 64-bit machine, it's 64 bits. If you transmit an int from the 64-bit system to the 32-bit system, and it's only using it's native int size, you have 4 extra bytes in your data stream that will throw everything off.

  7. #7
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,656
    Perhaps something like this?
    External Data Representation - Wikipedia, the free encyclopedia

    If you've got a lot of structures to get through, you might be able to dig up some library support and/or tools to at least semi-automate the generation of the serialisation wrappers.

    Or if your structs are particularly consistent and well-formatted, you might be able to do something with script tools like perl/awk.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.

  8. #8
    Registered User
    Join Date
    Nov 2010
    Location
    Long Beach, CA
    Posts
    5,909
    Quote Originally Posted by Salem View Post
    Or if your structs are particularly consistent and well-formatted, you might be able to do something with script tools like perl/awk.
    Or my preference, one badass vim regular expression in line-wise visual mode. Continue straight to coding, or use the undo capabilities if you botched it.

  9. #9
    Ticked and off
    Join Date
    Oct 2011
    Location
    La-la land
    Posts
    1,728
    The way I've solved similar issues, is to use the specific-size integer types (int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t and uint64_t, and Binary32 (float) and Binary64 (double) for floating-point data, with each sender using their native format, and each receiver converting read data to native on the fly. To determine the byte order conversion necessary, the endpoints use a "handshake" which contains a prototype value of each type (with different byte values of each component).

    For example, assume you transfer float values. A prototype value I've used is 721409.0 / 1048576.0 = 0.68798923492431640625 , which has the bit pattern 0011111100110000001000000001000016, which as an integer is 3F30201016 in hexadecimal. Thus, each byte in the received prototype value must be either 3F16=63, 3016=48, 2016=32, or 1016=16. Only their order can vary. All the float values will have that same byte order, so figuring out what order change you need to do to get the prototype value correct, also tells you the order change you need to do on any float to interpret it correctly.

    The same approach works for integers, too, of course.

    The receiver only needs to find the byte order permutation that yields the same byte order as the native prototype value. Currently, you can expect there to be only two: either no permutation (same byte order), or reverse byte order. In the past, there have been different architectures, and you could support all four (1234, 4321, 2143, 3412) for 32-bit and 64-bit types, but I haven't seen anything except same (1234) or reverse (4321) byte order in practice.

    On current architectures integers and floats have the same byte order, but that may not necessarily be true in the future.

    There are many ways to manipulate the byte order, but I prefer the arithmetic one. For example:
    Code:
    uint32_t get_u32(const void *const dataptr, const unsigned char byte_order)
    {
        uint32_t  data = *(const uint32_t *)dataptr;
    
        if (byte_order & 1)
            data = ((data & 0x00FF00FFU) << 8U)
                 | ((data >> 8U) & 0x00FF00FFU);
    
        if (byte_order & 2)
            data = ((data & 0x0000FFFFU) << 16U)
                | ((data >> 16U) & 0x0000FFFFU);
    
        return data;
    }
    
    float get_float(const void *const dataptr, const unsigned char byte_order)
    {
        uint32_t  data = *(const uint32_t *)dataptr;
    
        if (byte_order & 1)
            data = ((data & 0x00FF00FFU) << 8U)
                 | ((data >> 8U) & 0x00FF00FFU);
    
        if (byte_order & 2)
            data = ((data & 0x0000FFFFU) << 16U)
                | ((data >> 16U) & 0x0000FFFFU);
    
        return *(float *)&data;
    }
    Note that you can simply try all possible byte_order values (four for 32-bit values), until the received prototype matches the expected prototype value. If none of them work, then the sender did not use the same representation, and the communication would not work anyway.

    I've found that for typical messages, the overhead incurred to adjust the byte order for received messages is minimal, truly neglible. One reason for that is that the data received tends to be cache-hot, and any operations done on them will usually not cause any cache-misses. The overhead tends to be just a few clock cycles per element.

    The approach works very well even when messages are broadcast. The recipient only needs to check the sender to remember which byte order swizzle has to be done to the message. If the 2, 4, or 8-byte overhead per data type per message is not significant, you can even include the prototype value for each type in the message itself.

    (Note that the situation is NOT symmetric. You cannot do the conversion on the sender side, and achieve the same flexibility. The sender would have to target each receiver separately. Since each message can have only one sender, but multiple recipients, the recipient can always adapt to the sender but not vice versa. So, this is only really feasible on the recipient side.)

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Returning Values in C
    By cuba06 in forum C Programming
    Replies: 6
    Last Post: 11-21-2007, 01:23 AM
  2. FFT returning values in wrong order
    By DavidP in forum C# Programming
    Replies: 3
    Last Post: 10-25-2007, 01:15 PM
  3. Returning values from functions
    By g001 in forum C++ Programming
    Replies: 3
    Last Post: 10-01-2007, 03:42 AM
  4. Read and Write, not getting same values.
    By Wraith-Lunati in forum C++ Programming
    Replies: 9
    Last Post: 09-13-2006, 10:41 AM
  5. Returning Values
    By incognito in forum C++ Programming
    Replies: 2
    Last Post: 03-17-2002, 09:31 AM