Thread: Problem with asynchronous read/write

  1. #1
    Registered User
    Join Date
    Jun 2007

    Problem with asynchronous read/write


    I have discovered that a certain feature of aio in Linux does not work as I expect. From what I understand, it is possible to start read and then write asynchronous operations, and expect them to complete in ANY order. For example, suppose that when a client connects I want to send it a greeting as well as start listening for any data asynchronously. If I start async write and then async read everything works as expected. However, if I start async read and then async write, the write operation does not proceed until the read operation completes. I do not understand why this happens. If a client does not send any data, I still should expect the write operation to complete, despite that it was started second. Even though there is no data available from the client, there still should be possibility to send it something.

    Below I have a sample program to test what I mean. The program listens on port 7777. When connection arrives it starts async read and then initiates async write to send "HELLO" greetings to the client. When I use telnet to connect to this, I expect to see "HELLO" right after the connection completes, even though I haven't sent any data. However, this is not what happens. I have to send some data to the server and only then does the write operation proceeds and I receive my long awaited "HELLO". If I reverse the two IO operations (write followed by a read), everything works as expected. I tested this on Debian and Suse linux distributions with the same result (kernel 2.6.18 ).

    Please, if somebody could clarify to me why this happens and how I should go about it, I would be extremely grateful!
    Thank you in advance.

    #include <iostream>
    using namespace std;
    #include <aio.h>
    #include <unistd.h>
    #include <string.h>
    #include <sys/socket.h>
    #include <netinet/in.h>
    #include <errno.h>
    int main()
      sockaddr_in addr;
      memset(&addr, 0, sizeof(addr));
      addr.sin_port = htons(7777);
      addr.sin_family = AF_INET;
      addr.sin_addr.s_addr = INADDR_ANY;
      int fd = socket(AF_INET, SOCK_STREAM, 0);
      assert(fd > 0);
      int opt = 1;
      assert(setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) == 0);
      assert(bind(fd, (struct sockaddr*)&addr, sizeof(addr)) == 0);
      assert(listen(fd, 5) == 0);
      int cfd = accept(fd, NULL, 0);
      assert(cfd > 0);
      //  READING:
      aiocb read_cb;
      char readBuf[1024];
      memset(&read_cb, 0, sizeof(read_cb));
      read_cb.aio_fildes = cfd;
      read_cb.aio_offset = 0;
      read_cb.aio_buf = readBuf;
      read_cb.aio_nbytes = 1023;
      // start asynch read first
      assert(aio_read(&read_cb) == 0);
      //  WRITING:
      aiocb write_cb;
      char strToSend[] = "HELLO\n";
      size_t strLen = strlen(strToSend)+1;
      memset(&write_cb, 0, sizeof(write_cb));
      write_cb.aio_fildes = cfd;
      write_cb.aio_offset = 0;
      write_cb.aio_buf = strToSend;
      write_cb.aio_nbytes = strLen;
      // start asynch write second
      assert(aio_write(&write_cb) == 0);
      size_t const numIO = 2;
      aiocb* cba[numIO] = {0};
      cba[0] = &write_cb;
      cba[1] = &read_cb;
      cout <<"Start waiting for IO completion" <<endl;
      for (size_t done=0; done < numIO; ) {
        //int ret = aio_suspend(cba, numIO, NULL);  // uncommenting this does not help
        //assert(ret == 0);
        // look for completed io:
        done = 0;
        for (size_t c=0; c < numIO; ++c) {
          aiocb *currIO = cba[c];
          if (currIO == NULL) {
          int errorIO = aio_error(currIO);
          // IO has completed:
          if (errorIO == 0) {
            ssize_t retIO = aio_return(currIO);
            if (currIO == &write_cb) {
              cout <<"WRITE COMPLETED: ret=" <<retIO <<endl;
            else if (currIO == &read_cb) {
              readBuf[retIO] = 0;
              cout <<"READ COMPLETED : ret=" <<retIO <<" buf=" <<readBuf <<endl;
            else {
            cba[c] = NULL;
            assert(errorIO == EINPROGRESS); // don't deal with other errors
      return 0;
    (To compile this you have to link against librt: 'g++ -Wall -o server server.cpp -lrt')

  2. #2
    Registered User
    Join Date
    Jan 2011

    librt actually uses synchronous operations

    And thus the read operation indefinitely blocks the write operation.

    I've come across the same problem when porting an ACE Proactor application from VxWorks/Windows to Linux.

    Attached is a sample program that tries to use a non-blocking socket. It still doesn't work though, busy-looping with EAGAIN/EWOULDBLOCK status from the read operation.

    I'd appreciate help if you were able to solve this full-duplex socket problem without resorting to something different than AIO.

    P.S. The attached program has the same behavior on Centos5.5, WindRiver Linux 1.4 and WindRiver Linux 3.0.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Memory problem with Borland C 3.1
    By AZ1699 in forum C Programming
    Replies: 16
    Last Post: 11-16-2007, 11:22 AM
  2. Someone having same problem with Code Block?
    By ofayto in forum C++ Programming
    Replies: 1
    Last Post: 07-12-2007, 08:38 AM
  3. A question related to strcmp
    By meili100 in forum C++ Programming
    Replies: 6
    Last Post: 07-07-2007, 02:51 PM
  4. WS_POPUP, continuation of old problem
    By blurrymadness in forum Windows Programming
    Replies: 1
    Last Post: 04-20-2007, 06:54 PM
  5. beginner problem
    By The_Nymph in forum C Programming
    Replies: 4
    Last Post: 03-05-2002, 05:46 PM