Hi all,
I have a fairly simple server running on system A with several clients running on systems B (inside LAN) and C (outside LAN). Now the client on system C was running great for hours making all sorts of calls to connect(), but then timed-out once, and ever since, can never connect() to the server on A ever again. The thing is, the client on B and even running a client on A continue to work fine, so A is responding. They can open a connection, talk, close it, wait X minutes, reopen, etc. However C just times-out on connect() forever.

There are at least 3 firewalls between A and C, but none between A and B, so I could imagine it's a firewall issue. However the communications all take place over port 80 (http), and I can view C and get feedback, so I don't think it's a firewall issue. C is actually 152.20.76.100, so if you go there you get the outline of a webpage -- which makes me think it's not a firewall issue -- but C will never communicate with A.

Some important notes:
1) C WAS communicating great for a few hours, then hit a timeout on connect() ONCE and could never re-connect()
2) restarting the process on C does not fix the problem.
3) the client does other things and responds great, so it's not "frozen". For example, if you go to 152.20.76.100 you can submit queries to the graph and get feedback, so the client DOES respond to a user, it just can no longer reach A.

Any ideas? Do you think it could be some very intelligent firewall that looks at HTTP headers (like the 'User-Agent' field) or something, and filters the packets on that level? I figured all firewalls would just shut down an entire port or not and if I can view the website it's not a firewall issue.

I'm out of ideas. I've done tons of load testing internal to our LAN and no problems, so I don't know if C being outside our LAN is coincidence or not.

Any ideas or suggestions would be greatly appreciated.
Thanks.

This is called every time C has data to send to A, then calls another method to actually write() to the socket, but it never gets that far:
Code:
  
  
 Boolean RemoraServer::OpenCDMOSocket() 
 { 
 _ _struct sockaddr_in ServerAddress; // home base server address_ _ 
  
 _ _//open our socket 
 _ _if ((CDMOSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) 
 _ _{ 
 _ _ _ Log("While trying to open a socket to the CDMO, it failed"); 
 _ _ _ return (FALSE); 
 _ _} //end if 
 _ _  
 _ _//resolve our hostname into a host struct to get the IP address 
 _ _struct hostent *hostStructure = gethostbyname(repositoryURL); 
 _ _if (hostStructure == NULL) 
 _ _{ 
 _ _ _ Log("gethostbyname() failed with this exit code: " + itoa(h_errno)); 
 _ _ _ close(CDMOSocket); 
 _ _ _ CDMOSocket = -1; 
 _ _ _ return (FALSE); 
 _ _} //end if hostStructure failed 
  
 _ _//construct the server address structure  
 _ _memset(&ServerAddress, 0, sizeof(ServerAddress));_ _ _// zero out the structure to begin with 
 _ _ServerAddress.sin_family_ _ _ = AF_INET;_ _ _ _ _ _  
 _ _ServerAddress.sin_addr.s_addr = inet_addr(inet_ntoa( *(struct in_addr *) hostStructure->h_addr_list[0] ) ); //just try the first address  
 _ _ServerAddress.sin_port_ _ _ _ = htons(repositoryPort);  
 _ _int connect_value = 0; 
 _ _ 
 _ _// now establish the actual connection to home base 
 _ _if ((connect_value = connect(CDMOSocket, (struct sockaddr *) &ServerAddress, sizeof(ServerAddress))) < 0) 
 _ _{ 
 _ _ _ Log("Connect failed - (" + itoa(errno) + ") " + (JString)strerror(errno)); 
 _ _ _ Log((JString)hostStructure->h_addr_list[0]); 
 _ _ _ close(CDMOSocket); 
 _ _ _ CDMOSocket = -1; 
 _ _ _ return (FALSE); 
 _ _} //end if 
  
 _ _return (CDMOSocket != -1 ? TRUE : FALSE); 
  
 } //end RemoraServer::OpenCDMOSocket()


These are the msgs that are logged, which show the exact same error (timeout) over and over and over:
Fri Oct 17 12:14:32 2003
SERVER Our host successfully added the requested data
Fri Oct 17 12:14:32 2003
SERVER We received this command:PUT NOCRC CR10-150 1066380300 11.24800 97.84900 23.76600 1019.09998 1.65390 312.00000
Fri Oct 17 12:14:32 2003
SERVER Our host successfully added the requested data
Fri Oct 17 12:14:32 2003
SERVER CDMOSocket was -1 so we're going to try and open it
Fri Oct 17 12:17:41 2003
SERVER Connect failed - (110) Connection timed out
Fri Oct 17 12:17:41 2003
SERVER ¿®
Fri Oct 17 12:17:41 2003
SERVER ERROR: Could not open CDMO socket
Fri Oct 17 12:17:41 2003
SERVER We were unable to write our data string to the CDMO
Fri Oct 17 12:17:41 2003
SERVER We were not able to transmit this data to the CDMO
Fri Oct 17 12:17:41 2003
SERVER We could not fully process the command 'PUT NOCRC CR10-150 1066380300 11.24800 97.84900 23.76600 1019.09998 1.65390 312.00000 '
Fri Oct 17 12:17:41 2003
SERVER We received this command:PUT NOCRC CR10-150 1066359600 12.77800 95.02100 25.98500 1020.00000 1.92210 309.00000
Fri Oct 17 12:17:41 2003
SERVER Our host successfully added the requested data
Fri Oct 17 12:17:41 2003
SERVER CDMOSocket was -1 so we're going to try and open it

etc....................