accept() Hangs

Hullo everyone,

My code hangs on the following line of code:

Code:

CSocket = accept(LSocket, NULL, NULL);

CSocket and LSocket are both of type SOCKET.

LSocket is the listening socket. It gets created to listen on a specific port for incoming connections.

When accept() is called, CSocket will be setup to handle any incoming connection that LSocket receives. However, if no incoming connection is detected by LSocket, then the code will "hang" on the accept() function until there IS a connection? And, essentially, once a connection is received, LSocket becomes useless (in this example), which is why it's closed, and any further contact between the connection will be done through CSocket?

Sorry, that's the least confusing way I could find to write it out. I'm coming at this from Winsock under Visual Basic, and boy, can I say that it was a whole lot easier

EDIT:
Search saved me:
Dumb question time - Winsock & accept()

I'm pretty sure I'm understanding it correctly now...

EDIT2:
Does this mean, if I want multiple connections, coming in at any time, I'll need multiple threads?

You could do multiple threads but you can also tell it to accept x number of connections and this is much easier, but if you are going to be accepting a bunch of connections then you might want to think about doing threads.

Threads increase the complexity of a C application dramaticaly. Increased complexity == more chances to make errors. Conclusion being, don't use threads in C, it is not productive.

I/O Multiplexing is probably best in C. However one should note that somethign such as Erlang or Concurrent ML is probably going to produce better results when it comes to concurrency and networking. Finally, a URL that gives you solutions to how to handle a large number of connections efficiently in C can be seen here:
http://www.kegel.com/c10k.html

Also, a website on threading:
http://threading.2038bug.com/index.html

Cool, thanks for the help guys...

I've come up with this general idea for handling everything. In games, the way they are generally setup is to perform the same tight loop over and over. Being something along the lines of:

Code:

//Pseudo
Loop While Running
{
GetInputFromUser();
UpdateVariables();
RenderScene();
}

Perhaps slightly more complicated

So, I've decided to go in the Win32 direction with this project of mine and incorporate a similar loop:

Code:

In Main.cpp
while(MainWindow.bRunning == true)
{
	MainWindow.MessageTick();
	MyWinsock.LoopTick();
}

Pretty fancy eh? MainWindow is my class created for handling, you guess it, the window. And MyWinsock handles all Winsock related issues. Here are the "Ticks":

Code:

MessageTick()
void cWindow::MessageTick(void)
{
	MSG Msg;

	SetFocus(hWnd); //Set Focus on the Main Window

	if(PeekMessage(&Msg, NULL, 0, 0, PM_REMOVE) != 0) //DO WE HAVE ANY MESSAGES WAITING
	{
		TranslateMessage(&Msg);	//CONVERT OUR VIRTUAL-KEY MESSAGE INTO A CHARACTER MESSAGE
		DispatchMessage(&Msg);	//SEND THE MESSAGE TO OUR MISC. MESSAGE HANDLER
	}
	else
	{
		//THERE WAS NO MESSAGE
		//
		//UPDATE THE WINDOW
		UpdateWindow(hWnd);
	}
}

Code:

LoopTick()
void cWinsock::LoopTick(void)
{
	/*
	PingExistingConnections(); //Closes any useless connections
	CheckForNewConnections(); //Accept any new connections
	ReceiveMessages(); //Interpret Incoming Data
	*/
}

Now, the order of those may be shuffled around a bit depending on the time that it'll take/how important they end up being (I'm relatively new with this, so I'm not sure yet). And of course, more functions may be added.

I guess my biggest question right now would be, since accept() is such a jerk and causes the machine to hang if there are no incoming connections, is there a way, kind of how like PeekMessage() tells me if a message is waiting, to check quickly whether or not there is an incoming connection waiting?

My whole plan here is relying on there actually being such a function. I sure hope somebody can shed some light here...

Conclusion being, don't use threads in C, it is not productive.

I couldn't possibly disagree more. I find it a lot easier to have each connected client processed on its own thread. There's a reason why most servers are multithreaded instead of multiplexed.

Code:

/* For God's sake, dont try pasting this code into a compiler */
void HandleConnection(void *p)
{
   SOCKET s = (SOCKET)p;
   /* Handle your connected client here */
}

int main()
{
   socket();
   bind();
   while(1) {
      SOCKET s = accept();
      THREADHANDLE th = newThread(HandleConnection,s);
      saveThreadHandleInSomeDataStructure(th);
   }
   closesocket();
}

If you are coding something like a ftp or http server, then you dont even need to keep track of the thread handles since they will never interact. If you are coding something like a chat server, then you just need to iterate through the threads to find the one you need to send a message to, then call send() on that thread's socket. Piece of cake. The only thing you should need to synchronize access around is the data structure that holds the thread handles.

> I couldn't possibly disagree more.
I disagree with your disagreement.

Your assertion may well be true, but you have to be a damn good designer and programmer to pull of any reasonably complex program which uses threads. Get it wrong and it either crashes totally randomly (you thought segfaults were tricky), or your code will be so full of locks as to make any concept of 'concurrency' completely meaningless.

The last project I worked on was a chaotic mix of race conditions caused by slap happy abuse of threads.

Well then I guess I disagree with your disagreement to my disagreement.

Threading only becomes tricky when you have multiple threads accessing shared global objects. That's just poor design. It is extremely easy (in my experience) to make a multi-threaded server that doesn't have race conditions or deadlocks. I don't consider myself an exceptional software engineer, but I've never had trouble with threaded servers before.

Hmmm....

Well, the example you showed bithub makes it look pretty simple. I do understand though that it will be more complicated, but it's not the craziness I was expecting.

As for being a "damn good designer and programmer"...well...I guess I'm just gonna have to become one Thanks for the heads up though Salem, I'll make sure to put extra care and time into making sure it's going together how it should.

I'll probably be back here soon

then call send() on that thread's socket. Piece of cake.

Sounds like a good idea. I sure hope you are serializing access to that socket. Not to mention, shouldn't your thread be aware of data being sent on it's socket and what that data is so it can manage it's state? What if the message sent requires a response? Shouldn't the thread need to know how to respond? That sounds like some inter-thread-communication to me...

Ok, let's assume for a moment that throwing each connection in it's own thread is easier to design and use. And lets also assume you are using pthreads, so each one is most likely going to correspond to a kernel thread. So you have 1 connection per 1 kernel thread. How many connections do you plan on handling here? 20? Your server is going to be brought to it's knees with 1000 concurrent connections.
Most decent servers/clients that use threads generally use 1 thread per CPU and then multiplex on those.
Now, the best threading systems I've seen on C revolve around message passing. This allows you to put all serialzation in one portion of your program, which would be the locking between message queues. The problem with this is it requires copying all data sent between threads for it to really work. Sending pointers doesn't help since you are just passing shared data around, which is going to break things. On top of that, it is fairly difficult to send arbitrary objects around in C. You have to write some sort of typeid layer on top of that. In which case, why are we still using C for this when all this has been written before and most likely better.

Now you also stated that any design where threads have to interact is inherently poor. Please show me any reasonable industrial strength application that is multithreaded where threads arn't going to need to interact. On the most basic level, you are going to need startup/shutdown messages between threads. If you wish to exit cleanly a thread needs to know it's time to exit don't they? So let's assume you use 1 connection per thread. If that thread is stuck in an accept() call, there is no way for it to get that message, unless someone connections and then it can stop blocking on it and check for messages. So the solution? Well make each thread 'select' On the accept socket, then it can periodicly check for messages after a timeout or something else. Ok, but now you are just multiplexing so why don't you just do that in the main thread in the first place? Secondly, any application is going to need to manage it's threads in some way. This may involve quering it to get it's state, or telling it to wait or what not. Yet more communication. most applications also log data. See just about any FTP server or HTTP server. If you have a lage number of threads logging a large amount of data this is going to be relativly time consuming, a lot of blocking will happen. So generally you'd want a single thread handling the logging and communicating the message to log to it some how. Yet more thread communication. So threads communicating with each other is going to fairly common in even a well designed application.

So, from the sounds of it, you've exchanged thread communication for a less powerful application. Not to mention, in hopes of avoiding multiplexing your application will scale fairly poorly due to all the resources each thread will use (yes it adds up). You can set hard limits if you want to avoid using up too much resources. And limits are fine, but not when they are far below the limits that obtainable by other designs. I would suggest reading the URL's I pasted in my previous post.

I sure hope you are serializing access to that socket

Yep. Anytime a thread must interact with another thread's data structure, you must serialize that operation. I'm not saying it shouldn't be done, I'm just not making it out to be the rocket science that you are.

Not to mention, shouldn't your thread be aware of data being sent on it's socket and what that data is so it can manage it's state?

Yes, but this is difficult how?

What if the message sent requires a response?

Then you screwed up in your server design.

Shouldn't the thread need to know how to respond?

Nope.

Your server is going to be brought to it's knees with 1000 concurrent connections

Think again, there are many multithreaded servers with support beyond 1000 connections. You may have to reduce the thread's stack size based upon the amount of RAM in the server though. I worked on a inhouse multithreaded mail server which had support for over 10k connections.

Please show me any reasonable industrial strength application that is multithreaded where threads arn't going to need to interact

Apache for windows. This application has very minimal thread interaction.

So, from the sounds of it, you've exchanged thread communication for a less powerful application

Only if you think a context switch is less overhead than the code required to multiplex. In some applications it probably is, but I can guarantee you it isn't true in every case.

If multiplexing is ALWAYS the better choice as you claim, then why are there so many multithreaded servers out there? Saying threads should never be used is a ridiculous generalization which undercuts one a powerful tool available to programmers. But hey, that's just my opinion, and everyone is entitled to their own.

I'm just not making it out to be the rocket science that you are.

I wouldn't say I'm making it out to be rocket science, simply more error prone in C/C++.

Then you screwed up in your server design.

Could you elaborate on that? Many protocols are request and response base. And in many cases a request or response has to be formed and the code which processes it has to be aware of it's state when it makes the request. Not to mention, why leak implementation details all around the application. Shouldn't the thread know how to handle the data and present it to the rest of the application? Otherwise you get a lot of coupling. However your statement is far to vague to jump to that conclusion of course.

Think again, there are many multithreaded servers with support beyond 1000 connections. You may have to reduce the thread's stack size based upon the amount of RAM in the server though. I worked on a inhouse multithreaded mail server which had support for over 10k connections.

We are talking concurrent connections here right? Does it run spawn a new thread for each incomming connection? Does no multiplexing happen? What sort of hardware did it run? What threading library? I am a bit hesitant to take that statement at face value since I don't know of a kernel which can decently schedule over 10k threads.

Only if you think a context switch is less overhead than the code required to multiplex. In some applications it probably is, but I can guarantee you it isn't true in every case.

I didnt' say performance, I said power. In terms of expressiveness and ability. I have already mentioned base cases for minimal thread communication (shutdown's in particular).

However, since you bring it up, context switches in kernel threads (such as ones you are probably going to be getting in most C threading API's) don't scale all too well. The more threads you get the more overhead it will take up, context switches add up. But even that doesn't bother me much.

Apache is a fairly poor example of a "good multithreaded" application. For starters, apache mostly involves spawning completely new processes. There are threaded workers for it, however they have been unstable for the most part. I have had many experiences where apache fails under high load on it's threading worker. Apache is also known for it's quality of service going severely down hill the more load it's under.

http://www.sics.se/~joe/apachevsyaws.html

In that experiment, apache begins to fall apart after 4000 connections. These values arn't set in stone but illustrate mine and many others experience with apache.

If multiplexing is ALWAYS the better choice as you claim, then why are there so many multithreaded servers out there?

I never stated multiplexing is always better. However when it comes to handling network connections in C/C++, multiplexing is generally less error prone. However, I often suggest people use a language such as Erlang, in which case you would probably have about 2 or 3 processes (threads) per connection.

As far as why there are so many multithreaded applications. To be blunt, like any hobby or profession the number of people that are proficient at what they do is far outnumbered by those who blunder around unknowingly. Not to mention, once a number of projects reach a certain point the cost of compeltely rewriting it in a more sane way generally is far more expensive then trying to get it's broken design a bit more managable.

I'm willing to admit the possibility that you are one of the people to not have any problems with threading in C, or atleast none of the problems you have had have shown themselves (using valgrind might show otherwise). However, a number of peopel do see problems with threading, many of us throuhg personnel experiences. If you're willing to use "look at all the multithreaded software" as an argument for threading (which I don't think holds much water), then look at all of the work to prevent the necesity of threading in C/C++, perhaps that says something.

If one wants to use threads, they are more than free to, but one should not be ignorant to the fact that threading in C/C++ will generally provide more error prone code should be known and taken into account. Going into concurrent programming without knowledge of it's pitfalls is a recipe for disaster. DISASTER!!!!!!!!

Moof

Could you elaborate on that? Many protocols are request and response base.

Yes, but the requests and responses should take place by the thread which "owns" the particular socket the requests and responses are being done on. A thread should never call recv() on another thread's socket. If you are in a situation where this is being done, you either need to rethink your threaded design, or use a different approach (as in multiplexing).

We are talking concurrent connections here right? Does it run spawn a new thread for each incomming connection? Does no multiplexing happen? What sort of hardware did it run? What threading library? I am a bit hesitant to take that statement at face value since I don't know of a kernel which can decently schedule over 10k threads.

Yes, it was concurrent connections, and a new thread was spawned for each connection. The hardware was quad pentium 3s, with 2GB of ram. I dont know if the kernel was modified as I came into the project after the infastructure was already set up.

I didnt' say performance, I said power. In terms of expressiveness and ability.

My mistake for misinterpreting. In complex applications which would require a threaded application to communicate extensively between threads, then I would agree that a threaded approach is not the easier (or best) one. On the other hand, an application like a mail server or http server, you don't run into this problem. In this case, I dont find the threaded approach suffers from a lack of expressiveness and ability.

I never stated multiplexing is always better.

True, but that was my interpretation of your statement when you said:

Conclusion being, don't use threads in C, it is not productive.

Regardless, your assertion seems to be (correct me if I'm wrong) that threads should never be used. That is the main belief that I'm arguing against.

I'm willing to admit the possibility that you are one of the people to not have any problems with threading in C

I've run into my fair share of deadlocks and race conditions. Most of those were due to poor program design or complex client applications. I'll never be satisfied with the conclusion that threads are inherently bad design though.

Check out Concurrent ML, or Erlang, or Mozart. They make threading in C look like the stone age. Extensive work has been done in acknowledging the many drawbacks to threading in non concurrent orientated languages and addressed with these. I'd suggest seriously looking at them.

Yeah, I'll take a look at those.

http://ortdotlove.net/docs/HPL-2004-209.ps

A paper on why threading cannot be part of a library.