HTTPS POST with C Socket
Hi guys, so here's my problem. Basically I need to make a socket connection to a HTTPS server and send a POST request to it. At the moment when I do this I get an error once I send() my POST data:
error: 10054, translating to:
It is my belief that this error is caused due to the nature of HTTPS. My program works fine when I post to a HTTP server on port 80, however soon as I attempt it on a HTTPS server on port 443 I get this error.
Connection reset by peer.
I'm having an extremely difficult time finding any resources or help to this problem on Google. I found someone with a similar problem (except he's using PHP) and he just resulted in using cURL, the PHP library. (Thread is here for those of you are interested).
Can anyone please point me in the right direction or tell me how I can get around this... Thank you!!!
The indirect answer (for a lot of network problems) is to use www.wireshark.org to compare a known working POST (say from a browser) with yours.
Or you could download the source for libcurl and look how it's been done there.
I think that's a rather large reference to "look" at :P
The post works fine if I do it to a non-HTTPS server :/
Maybe so, but unless you post code (which seems like a lot of code to me), simply posting an error message and asking us to guess your problem is a crap-shoot.
You're correct it's a lot of code, and it would be a complete waste of time posting my code as I've already stated it does a HTTP post over a socket, and it works fine for any non-HTTPS server, therefore it is not a problem with my integration, it is a question specific to the HTTPS protocol and how I should develop to incorporate the use of it.
I don't see how this is asking to guess a problem, it's asking if you have ever had experience developing an application that requires the use of a HTTPS connection and how you did so without the use of a 3rd party library, for example what was the different technique used in the method in which you sent the POST request to the server.
Hopefully someone has had experience with this situation before (I'd like to assume so), so I'll wait on any forum members to perhaps give their thoughts.
I have the same problem posting using port 443, so I'm trying to use openssl to post. but the c compiler complains:
/usr/ccs/bin/ld: Unsatisfied symbols:
SSL_connect (first referenced in sal443.o) (code)
SSL_write (first referenced in sal443.o) (code)
SSL_new (first referenced in sal443.o) (code)
SSLv23_client_method (first referenced in sal443.o) (code)
SSL_load_error_strings (first referenced in sal443.o) (code)
SSL_set_fd (first referenced in sal443.o) (code)
SSL_read (first referenced in sal443.o) (code)
SSL_CTX_new (first referenced in sal443.o) (code)
SSL_library_init (first referenced in sal443.o) (code
Are you simply connecting through the socket directly? Of course, since it's HTTPS, you'll have to use an SSL layer (that doesn't sound right, as the L itself stands for layer, but "have to use an SSL" doesn't make much sense either).
So do you use something like OpenSSL or not? If not, that's obviously your problem. Posting raw HTTP requests to HTTPS doesn't make any sense... And yes, it may close the connection, I guess.
You seem to got it right. Though you have to link with the proper libraries. I don't know which now, you might try -lopenssl, but look it up.
Thanks EVOEx, OpenSSL did the trick.
Are you (or anyone else) aware of any service or website I can send a HTTPS POST to that returns the data I sent (for troubleshooting/verification purposes)? I'm having some troubles because I think the data I'm sending is for some reason not what it should be when it's being received by the server.
Could it perhaps be using a different type of encryption algorithm or something and it's causing it to be decrypted into garbage therefore giving invalid output? The SSL_write function tells me I'm sending the correct number of bytes but the response from the HTTPS server isn't what it should be...
Edit: Let me explain the situation. My program is communicating with a secure payment server. This server reads a string sent to it and strips it of its contents. Basically the string I send to the server is something like this:
Normally when you submit a transaction the server will respond with the values you sent to it for a receipt:
POST /paypage HTTP/1.1
However the response from the server is something like so:
HTTP/1.1 200 OK
Date: Thu, 22 Apr 2010 12:22:24 GMT
Expires: Sun, Jul 1990 00:00:00 GMT
P3P: CP="NOI DSP COR CURa ADMa TA1a OUR BUS IND UNI COM NAV INT"
Keep-Alive: timeout=120, max=100
set-cookie: citrix_ns_id=ujS8Knmlb/NAAHx59/ilZXQ3gO0A0; domain=.server.com; path=/
Which indicates to me that the data (for some reason) is not being received as it should be, for example the amount should have been returned as 100 since I've sent it as 100, and so fourth. And that error shouldn't appear because I was sending the field it is saying is missing.
amount=0&identifier=0&message=required field blah is missing
Well... set-cookie isn't a legal header. To get back exactly what you sent, you can probably use netcat for that.
The server is returning that header. Not sure why.
Originally Posted by EVOEx
Because it's a valid header for a response. You'll have to parse it and set it to the right header ("Cookie"). See Client Side State - HTTP Cookies
Originally Posted by pobri19
So you think this would be causing the problem?
Originally Posted by EVOEx
How am I supposed to know what the cookie is if I haven't received the response header... I only receive the response header once I send the request line, but obviously it's too late at that point.
I.e. I send the following:
POST /paypage HTTP/1.0
User-Agent: C Example
But then it blocks on the recv() call after that because I haven't sent the request line yet... But if I send the request line to get the response it will respond before I have a chance to send the cookie header.
Okay I figured it out after a long time of messing around...
The problem was a header that wasn't specified in the POST request:
Hopefully if anyone else is encountering this problem then applying the Content-Type header will fix the problem.
If anyone else making a client to communicate with a HTTP/HTTPS server is receiving a response that doesn't appear to be correct or it indicates that the data you sent to it is incorrect or blank then this is a good indication that this is the problem.
And here's the working POST request for anyone else interested:
And in an array:
POST /uri HTTP/1.1
User-Agent: C Example Client
char * array = "POST /uri HTTP/1.1\r\n"
"User-Agent: C Example Client\r\n"