I have been working on an application that is basically a server, a client library and a handful of clients. The interface between the clients and the server is functional, meaning that the clients don't worry about the socket connections they just call API functions to read or write information to the server. The client library would then send/recv a request/response over the already connected socket, and return the result to the calling client. The purpose of the API is to allow other developers to build clients quickly and easily without too much trouble. This already works quite well. I am using a local domain stream socket if the client is on the same host as the server and if it's on another machine it uses TCP/IP.
I would like to add the capability for the server to send messages to the clients asynchronously. Something like event or data change notification. It's causing me some grief because of resource contention issues surrounding the socket. I'm trying to keep this thing as efficient as possible so I'm trying to avoid excessive system calls.
I have a couple of ideas. I don't know if either is any good or which is better so I was hoping to get some advice on the potential pitfalls or benefits of each one.
First is to start another thread from within the library and have it handle communication. The server already requires pthreads so from a dependency perspective this isn't that big of a deal, but it seems like bad form to start threads from within a library. Maybe not. It seems to me that I'll wind up having to use a bunch of system calls and signals to coordinate all this.
The other idea is to simply open a second socket and give the client developer a select/poll kind of interface to the asynchronous messaging. This would probably still require some multi-threading in most clients but it would be at the discretion of the client developer not hidden away inside the library.
I guess a third option is to do both. :-) I'm basically looking for advice from those who have gone before.
