I have a program which needs to connect multiple servers at the same time. The program has to collect data from each of servers and then make a decision regarding to the data received.
There are several requirements.
1. Server (s) may shutdown anytime without any ack (e.g.power off).
2. Client has to connect several servers.
3. Every decision should "discuss" with servers; if one of them failed, the decision will based on others which still alive.
4. The discussion should be fast, more than 200ms is not acceptable.
My design principle is like that:
1. Create threads which every one connect to a server.
2. Creating a sending queue.
3. If the queue doesn't empty, then connect the server and send the data from queue to server.
4. Wait message from server.
5. Close the connection
There are also problems in the design.
1. If I sent the message to server, how about if the server dead and no response? (receive timeout )
2. If we create and initiating socket in a short time, a lot of TIME_WAIT since the client will close the socket when send and receive operation finished.
3. Since I have to connect multiple servers, if one of them are failed. I have to reconnect them, may say every 1 second a try?
Would you please give me some ideas that how to handle such kind of application? Thanks a lot!