non-blocking sockets

So I found a select server off of a website and modified it. I am using the server with non-blocking clients.

If I connect 2 clients to the select server the clients work in blocking mode just fine. When I set the clients to non-blocking they echo to them selves. Whether connected to the server or not. As long as 2 or more clients are connected. For example if I connect cli1 & cli2 (no server), type hey in cli1 and it will send hey to itself.

I fixed the EAGAIN issue with read, and EINOROGRESS with connect.

What could be causing this issue?

Also, the select server works while the clients are connected. I ran strace with the server and it worked fine.

Nearly a year in, and 160 posts, and you still aren't providing us with your code...

Sorry anduril, I wrote that in a rush. Its just getting nice in Jersey. Super nice actually! Ive been planning a camping trip to the Appalachian mountains. And my gas gauge is broke so I've spent the day replacing my fuel sending unit on my 94 Ranger. I tried writing that post on the way to mechanic shop to pick up the part.

I'm getting gas now and figured I'd respond real quick. Ill re-write everything in a more elaborate manner along with the code when I get home tonight. This is from my android phone. Thanks and sorry for the cracked out post!

OK just got home. So here it goes.....

I have a select server in working order. I got the select server from a website and got it working with minimal tweaking. I wrote the clients myself. Now if I connect my 2 blocking clients to the select server, they communicate synchronously just fine. The problem occurs when i set non-blocking mode.

I took care of the expected EAGAIN issue.

Code:

---

if((readfd <= 0) && (readfd == EAGAIN))
{
readfd = read(sockfd, buffer, sizeof(buffer)); 
}

                else
                {
                fprintf(stdout, "%s", buffer);
                }

---

As well as the connect issue:

Code:

---


        int connector = connect(sockfd, (const struct sockaddr *) &serv_addr, sizeof(serv_addr));
        if((connector < 0) && (!EINPROGRESS))
        {
        fprintf(stderr, "CONNECT(-1) ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }
                else
                {
                  fprintf(stdout, "Made a connection to %s\n", inet_ntoa(serv_addr.sin_addr));
                }

---

setsockopt:

Code:

---

setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));

---

non-blocking mode:

Code:

---


int x = fcntl(sockfd, F_SETFL, O_NONBLOCK);
        if (x < 0)
        {
        printf("FCNTL(-1) ---> %s.\n", strerror(errno));
        //close(sockfd);
        //exit(EXIT_FAILURE);
        }

        if(x == 0)
        {
        printf("FCNTL(0) ---> SUCCESS.\n",);
        }

---

NOW FOR THE WEIRD ISSUE.
When i enable non-blocking mode, the clients connect to the server just fine. "EXCEPT" anything i type is just echoed back to the client that sent the string. For example, if I type hello on client 1, client 1 will immediately recieve a hello string from itself.

I am using strace and tcpdump to monitor all processes.

Another issue is that even if do not have the server running, the clients will work and echo to them selves. As long as 2 or more clients are connected. So no need for the server at all. The problem is non-existant as long as the clients are not set to non-blocking.


LET ME KNOW IF THE WHOLE CLIENT AND/OR SERVER CODE NEEDED. THANKS!

Yes, the full code would be helpful, though there are some issues with what you posted. You need to re-read the man page for the read function. It will not return EAGAIN, and you need to check EWOULDBLOCK too. Also, readfd is not being used the way I would expect. Either it's a file descriptor for something else, and you're overwriting it with a bogus value (the return value of read), which could cause your problem, or it's just really badly named, and should be called something sensible like bytes_read.

Here is the full code.

CLIENT:

Code:

---

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netdb.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <errno.h>

int main(int argc, char *argv[]) {
int sockfd, portno, readfd, writefd, yes = 1;
char buffer[1024];
struct hostent *server;
struct sockaddr_in serv_addr/*, cli_addr*/;


        if(argc < 3)
        {
        fprintf(stderr, "Ussage: %s IP Address port #", argv[0]);
        exit(EXIT_FAILURE);
        }

sockfd = socket(AF_INET, SOCK_STREAM, 0);
        if(sockfd < 0) 
        {
        fprintf(stderr, "SOCKET(-1) ---> %s.", strerror(errno));
        exit(EXIT_FAILURE);
        }

        if(sockfd == 0) 
        {
        fprintf(stderr, "SOCKET(0) ---> %s.", strerror(errno));
        exit(EXIT_FAILURE);
        }

int x = fcntl(sockfd, F_SETFL, O_NONBLOCK);
        if (x < 0)
        {
        printf("FCNTL(-1) ---> %s.\n", strerror(errno));
        /*close(sockfd);
        exit(EXIT_FAILURE);*/
        }

        if(x == 0)
        {
        printf("FCNTL(0) ---> %s.\n", strerror(errno));
        }

setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes));

bzero((char *) &serv_addr, sizeof(serv_addr));
server = gethostbyname(argv[1]);
        if(server == NULL)
        {
        fprintf(stderr, "No such host.");
        printf("SERVER(NULL) ---> %s", strerror(errno));
        exit(EXIT_FAILURE);
        }

portno = atoi(argv[2]);
serv_addr.sin_family = AF_INET;
memcpy(&serv_addr.sin_addr.s_addr, server->h_addr, server->h_length);
serv_addr.sin_port = htons(portno);

        int connector = connect(sockfd, (const struct sockaddr *) &serv_addr, sizeof(serv_addr));
        if((connector < 0) && (!EINPROGRESS))
        {
        fprintf(stderr, "CONNECT(-1) ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }
                else
                {
                  fprintf(stdout, "Made a connection to %s\n", inet_ntoa(serv_addr.sin_addr));
                }

for( ; ; ) {
int i = sizeof(buffer)-1; if(i > 0) bzero(buffer, sizeof(buffer));
          fprintf(stdout, "Message: ");
        fgets(buffer, sizeof(buffer), stdin);
       
writefd = write(sockfd, buffer, strlen(buffer)-1);
        if(writefd > 0)
        {
        printf("Waiting for %s\n", inet_ntoa(serv_addr.sin_addr));
        }
                else
                {
                        if(writefd < 0)
                        {
                        fprintf(stderr, "WRITE(c) ---> %s.\n", strerror(errno));
                        printf("errno = %d.\n", errno);
                        exit(EXIT_FAILURE);
                        }
               
                        if(writefd == 0)
                        {
                        printf("WRITE(0). ---> %s.\n", strerror(errno));
                        exit(EXIT_FAILURE);
                        }
                }


//i = sizeof(buffer); if(i > 0) bzero(buffer, sizeof(buffer)-1);
if((readfd <= 0) && (readfd == EAGAIN))
{
readfd = read(sockfd, buffer, sizeof(buffer)); 

        /*if(readfd < 0)       
        {
        fprintf(stderr, "Error reading message from %s\n", inet_ntoa(cli_addr.sin_addr));
        printf("READ(-1) ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }
       
        //Test to see if the buffer is blank. Uncomment to test.
        if(readfd == 0)
        {
        printf("READ(0) ---> %s. Null buffer.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }*/
               
        //else if((readfd == EAGAIN) && (readfd != EAGAIN))
        //{
        //fprintf(stdout, "%s", buffer);
        //}

}
                else
                {
                fprintf(stdout, "%s", buffer);
                }

}

close(sockfd);

return 0;
}

---

SELECT SERVER:

Code:

---

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <netdb.h>

int main(int argc, char *argv[]) {
/* master file descriptor list */
fd_set master;
/* temp file descriptor list for select() */
fd_set read_fds;
/* server & client address */
struct sockaddr_in serv_addr, cli_addr;
struct timeval timeout;
/* maximum file descriptor number */
int fdmax;
/* listening socket descriptor */
int sockfd;
/* newly accept()ed socket descriptor */
int newsockfd;
/* buffer for client data */
char buf[1024];
int nbytes;
/* for setsockopt() SO_REUSEADDR, below */
int yes = 1;
socklen_t clilen;
int i, j;
/* clear the master and temp sets */
FD_ZERO(&master);
FD_ZERO(&read_fds);
 
if(argc < 2)
{
printf("USAGE: %s + <portno>\n", argv[0]);
exit(EXIT_FAILURE);
}

timeout.tv_sec = 300;
timeout.tv_usec = 0;

/* get the listener */
sockfd = socket(AF_INET, SOCK_STREAM, 0);
        if(sockfd < 0)
        {
        printf("%s. SOCKET()", strerror(errno));
        exit(EXIT_FAILURE);       
        }
                else if(sockfd)
                {
                        do
                        {
                          {
                          printf("Waiting for a connection.\n");
                          }                       
                        } while(!accept);
                }

                printf("Server-socket() is OK...\n");

        /*"address already in use" error message */
        if(setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int)) < 0)
        {
        printf("SETSOCKOPT() ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }

        printf("Server-setsockopt() is OK...\n");
 
/* bind */
int portno = atoi(argv[1]);

serv_addr.sin_family = AF_INET;
serv_addr.sin_addr.s_addr = htonl(INADDR_ANY);
serv_addr.sin_port = htons(portno);
memset(&(serv_addr.sin_zero), '\0', 8);

int binder;
binder = bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr));

        if(binder == -1)
        {
        printf("BIND() ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }
       
        printf("Server-bind() is OK...\n");
 
/* listen */
int listener = listen(sockfd, 10);
        if(listener < 0)
        {
        printf("LISTEN() ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }

        printf("Server-listen() is OK...\n");
 
/* add the listener to the master set */
FD_SET(sockfd, &master);
/* keep track of the biggest file descriptor */
fdmax = sockfd; /* so far, it's this one*/

/* Main loop */
for( ; ; )
{
/* copy it */
read_fds = master;
 
        int selector = select(fdmax+1, &read_fds, NULL, NULL, &timeout);
        if(selector < 0)
        {
        printf("SELECT(-1) ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }

        if(selector == 0)
        {
        printf("SELECT(0) ---> %s.\n", strerror(errno));
        exit(EXIT_FAILURE);
        }

       
        else
        {
        printf("Server-select() is OK...\n");
        printf("Waiting for data...\n");
        }


 
/*run through the existing connections looking for data to be read*/
for(i = 0; i <= fdmax; i++)
{//2nd for loop
            if(FD_ISSET(i, &read_fds))
            { /* we got one... */

                    if(i == sockfd)
                    {
                    /* handle new connections */
                    clilen = sizeof(cli_addr);

                        if((newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen)) == -1)
                        {
                        printf("ACCEPT() ---> %s.\n", strerror(errno));
                        exit(EXIT_FAILURE);
                        }
                                else
                                {
                                printf("Server-accept() is OK...\n");
                                FD_SET(newsockfd, &master); /* add to master set */

                                        if(newsockfd > fdmax)
                                        {
                                        /* keep track of the maximum */
                                        fdmax = newsockfd;
                                        }

                                printf("%s: New connection from %s on socket %d\n", argv[0], inet_ntoa(cli_addr.sin_addr), newsockfd);
                                }
                }

        else
        {
        /* handle data from a client */
                if((nbytes = recv(i, buf, sizeof(buf), 0)) <= 0)
                {
                /* got error or connection closed by client */
                if(nbytes == 0)
                /* connection closed */
                printf("%s: socket %d hung up\n", argv[0], i);
 
                        else
                        printf("RECV() ---> %s.\n", strerror(errno));
                        /* close it... */
                        close(i);
                        /* remove from master set */
                        FD_CLR(i, &master);
                        }

                                else
                                {
                                /* we got some data from a client*/
                                        for(j = 0; j <= fdmax; j++)
                                        {//3rd for loop
                                                /* send to everyone! */
                                                if(FD_ISSET(j, &master))
                                                {
                                                /* except the listener and ourselves */
                                                        if(j != sockfd && j != i)
                                                        {
                                                                if(send(j, buf, nbytes, 0) == -1)
                                                                {
                                                                printf("SEND() ---> %s.\n", strerror(errno));
                                                                exit(EXIT_FAILURE);
                                                                }       
                                                        }
                                                }       
                                        }//3rd for loop
                                }
                }
        }
}//2nd for loop
}//Main loop
return 0;
}

---

In the meantime i will re-read up on the read function(Non-blocking). Also could you explain more about how i am using read in a bogus way?

Your use of read() is fine. It's the variable readfd that is confusing. The fd implies it's a file descriptor. If it's actually a file descriptor, then readfd = read() will overwrite the file descriptor with some other value, so when you try to use it as a descriptor, it will likely fail. If it's not a file descriptor, it's simply very badly named. I have to leave work now, so not sure when I'll get back to this. When you read the man page for read(), pay careful attention to the return value, and the section about what to check for EAGAIN/EWOULDBLOCK.

Oh ok, good. Yeah I named the variable like that of an FD, but it is not. It does confuse a lot of people though. I am self taught and that is how I learned. Just a bad habit and sometimes bad habits are hard to break ya know.

I'll read up on some documentation tonight though. Thanks Anduril.

Quote:

---

Originally Posted by anduril462

Also, readfd is not being used the way I would expect. Either it's a file descriptor for something else, and you're overwriting it with a bogus value (the return value of read), which could cause your problem, or it's just really badly named, and should be called something sensible like bytes_read.

---

I think you (Annonymous) got the point here but I'm gonna reiterate because I'm sure I've made this (exact same) observation before (as in, about your use of the identifier "readfd" is networking code). It also turns out to be central to the problem. The thing is, you can write your code how the hell you like, if you don't care whether or not anyone else ever looks at it, or what they'll think if they do. Otherwise, try to take criticism about style and readability to heart, even if you don't think these things affect function. One basic point about style is: make your identifiers meaningful. And don't reuse identifiers in ways inappropriate to their name. Eg, something I used to do was, if I'd declared an int somewhere for whatever purposes, and it remained in scope at some later point where I needed another int temporarily, and the original purpose of that first int was complete, I'd just re-use it. So I might end up with an int readfd that really was a file descriptor at some point and then use it temporarily in a loop to get (eg) the return value of read. Or something.

Eventually I'd come back and look and my own code and recognize this and think, that's kind of stupid, so I'd change the name of the variable to something like "tmp" or "x". That would be like, my general purpose int for the block. Better than declaring 3 ints if you don't have to, right? No, it isn't. That "savings" is totally irrelevant. Eventually I realized it's much better to have three ints with three specific names than one with a name that could be anything. Much easier to read.

Quote:

---

NOW FOR THE WEIRD ISSUE.
When i enable non-blocking mode, the clients connect to the server just fine. "EXCEPT" anything i type is just echoed back to the client that sent the string. For example, if I type hello on client 1, client 1 will immediately recieve a hello string from itself.

---

Let's look at somt code that could be responsible for this from the server:

Code:

---

    /* handle data from a client */
        if((nbytes = recv(i, buf, sizeof(buf), 0)) <= 0)
        {
        /* got error or connection closed by client */
        if(nbytes == 0)
        /* connection closed */
        printf("%s: socket %d hung up\n", argv[0], i);
 
            else
            printf("RECV() ---> %s.\n", strerror(errno));
            /* close it... */
            close(i);
            /* remove from master set */
            FD_CLR(i, &master);
            }
                else
                {
// Actually let's start again, because your indenting is
// obviously wrong at this point...

---

No offence, Annon, but that is pathetic. It looks like we are moving into a nest THAT DOESN'T EXIST. This is at least as misleading as misnaming variables. Are you trying to confuse people, lol? And I don't care if it was because of the forum widget or whatever. You should check the post and if the indenting is wrong do something to fix it. I cut n' paste here all the time, I don't have any problems.

Here's how it should look:

Code:

---

/* handle data from a client */
        if((nbytes = recv(i, buf, sizeof(buf), 0)) <= 0)
        {
                /* got error or connection closed by client */
                if(nbytes == 0)
                        /* connection closed */
                        printf("%s: socket %d hung up\n", argv[0], i);
                else
                        printf("RECV() ---> %s.\n", strerror(errno));
                /* close it... */
                close(i);
                /* remove from master set */
                FD_CLR(i, &master);
        } else {
// Now it makes sense:
// Let's say we're here now , because recv returned > 0.
                /* we got some data from a client*/
                for(j = 0; j <= fdmax; j++)
                {//3rd for loop <-- seen below about naming loops
                        /* send to everyone! */
                        if(FD_ISSET(j, &master))
                        {
                        /* except the listener and ourselves */
                                if(j != sockfd && j != i)
                                {
// on the surface it looks like this can't be the problem, but let's make sure:
fprintf(stderr,"Server send: j = %d\n", j);
                                        if(send(j, buf, nbytes, 0) == -1)
                                        {
                                                printf("SEND() ---> %s.\n", strerror(errno));
                                                exit(EXIT_FAILURE);
                                        }

---

About naming loops in comments: that's fine, but again, give them a meaningful name, not a number. That number implies nesting THAT DOESN'T EXIST. AGAIN.

All your code is like this -- very haphazardly indented. Stop it. You know how to indent, do it. The fact that you don't implies you are rushing and figuring you can tidy up later. That is a "shoot myself in the foot" repeatedly approach. We're going to see the consequence in a moment.

So, actually that section of code is not responsible. I'll give you a clue about what is. It's in the client code:

Code:

---

Line 107:
if((readfd <= 0) && (readfd == EAGAIN))
{
// Guess what?  Those conditions can never be true together.
[...]
}
// another zany indent
Line 131
        else 
        {
        fprintf(stdout, "-->%s<--", buffer);
        }

---

If your code weren't such a mess, this would have been obvious in < ten minutes.

So, several days later, do you think you saved much time being lazy on style? Anduril462 actually pointed this out in post #6, but that got lost in the confusion about what exactly readfd referred to.

In theory, I guess that condition could be true, except:

A) Guess what? Readfd hasn't necessarily even been initialized at that point. Which is a good reason to always initialize variables when they are declared.

B) I guess EAGAIN could be less than 0 depending on the implementation. It almost certainly isn't, but it could be. However, if EAGAIN is implementation dependent, then you shouldn't assume it is less than 0. This goes for ALL such defines and error codes.


That condition might be sort of fulfilled with:

Code:

---

if((readfd < 0) && (errno == EAGAIN))

---

Except, of course, readfd is still, potentially, at that point, uninitialized, and if not, what does it refer to? And the only reason I put errno there is because that's probably what you'd want to check -- if there is anything this check is supposed to apply to at that point in the code. I don't see anything.

So you get the "else". Notice before you get the echo back you get another message:

FCNTL(0) ---> Success.
Made a connection to 127.0.0.1
Message: what
Waiting for 127.0.0.1
-->what<--

If you wade through that haphazard mess of braces, you'll see what's going on.

Please, please, please don't just go and shim in some correction here. First, format everything correctly, and as you write, keep it so. What tool do you use to code, BTW?

If you post code with wacko freestyle indentation here again, don't expect to get any help from me :mad: Programming is not just about figuring out how to do something. It's also about keeping your tish together in the process. If you can't you're going to end up up a creek (again ;)).

MK27 is right. We aren't picky about what indentation/formatting style you choose, but it ought to be consistent. I usually recommend one of the first 3 styles mentioned here: Indent style - Wikipedia, the free encyclopedia, since they're common and easily read by pretty much anybody.

Now, let's see what happens when I compile your code:

Code:

---

$ make server
gcc -Wall -g -std=c99  -lm -lpthread -lcurses -lefence  server.c  -o server
server.c: In function ‘main’:
server.c:14: error: ‘fd_set’ undeclared (first use in this function)
server.c:14: error: (Each undeclared identifier is reported only once
server.c:14: error: for each function it appears in.)
server.c:14: error: expected ‘;’ before ‘master’
server.c:16: error: expected ‘;’ before ‘read_fds’
server.c:19: error: storage size of ‘timeout’ isn’t known
server.c:34: warning: implicit declaration of function ‘FD_ZERO’
server.c:34: error: ‘master’ undeclared (first use in this function)
server.c:35: error: ‘read_fds’ undeclared (first use in this function)
server.c:60: warning: the address of ‘accept’ will always evaluate as ‘true’
server.c:104: warning: implicit declaration of function ‘FD_SET’
server.c:114: warning: implicit declaration of function ‘select’
server.c:139: warning: implicit declaration of function ‘FD_ISSET’
server.c:182: warning: implicit declaration of function ‘FD_CLR’
server.c:19: warning: unused variable ‘timeout’
make: *** [server] Error 1

$ make client
gcc -Wall -g -std=c99  -lm -lpthread -lcurses -lefence  client.c  -o client
client.c: In function ‘main’:
client.c:54: warning: implicit declaration of function ‘bzero’
client.c:65: error: ‘struct hostent’ has no member named ‘h_addr’
make: *** [client] Error 1

---

Egad! Look at all that mess. Neither one of those builds without errors, let alone without warnings. Fix all of those. Your code should compile without any errors or warnings, even on the highest settings. Use the -Wall option with gcc. You should get no output if your code is clean.

Quote:

---

Originally Posted by anduril462

Egad! Look at all that mess. Neither one of those builds without errors, let alone without warnings. Fix all of those. Your code should compile without any errors or warnings, even on the highest settings. Use the -Wall option with gcc.

---

Umm, in Annon's defence, if you use -Wall but get rid of the unnecessary -std=c99, all you get is:

gcc -Wall -g anon_serve.c -o server
anon_serve.c: In function ‘main’:
anon_serve.c:60:21: warning: the address of ‘accept’ will always evaluate as ‘true’


Presumably because:

Quote:

---

Originally Posted by man select

/* According to POSIX.1-2001 */
#include <sys/select.h>

/* According to earlier standards */
#include <sys/time.h> <-- only if you are using the timeout, actually
#include <sys/types.h>
#include <unistd.h>

---

So to use c99 with that code all you need to do is add:

Code:

---

#include <sys/select.h>

---

Which gets rid of all the errors and warnings except the one. To be fair, this is not made very clear in any documentation I've seen, including that man page, which is more of a obscure hint.

@anduril462, i usually compile with -g and -Wall and it displays no errors. I have never seen someone compile the way you did in the previous post you made.

I have read a lot of documentation on EWOULDBLOCK & EAGAIN and a lot of people have trouble with non-blocking sockets. There was no solution or very little help regarding my situation/issue. Although the documentation on EWOULDBLOCK was very small, there is a decent amount of information/help pertaining to EAGAIN. I just have not been reading much on it. Kind of busy.

@MK, I use gedit and the command line to write code. Sometimes nano, but never vim. I could never get used to it and have become comfortable with the ease of nano.

MK, i have the utmost respect for you and your opinion! The last thing i want to do make you mad and turn away your help. What i am going to do is re-read on the documentation that pertains to my data handling. Fix the indentation, and apply the correction you have supplied for me.

See the problem with the select function is there really isn't that great of documentation available. If there is, it is written very sloppy, very poorly explained with the assumption of the readers understanding, If there is well documented tutorials/files on the function they are much to advanced!

Check out where i got the select server from ---> The select() and TCP server Linux socket and network programing tutorial with C code and sample outputs There are others but again, well you get the point.

I'll be back in a weeks time, give or take. and see what i have. Hopefully my issue is solved!

Thanks Anduril and MK.