Trying to capture POST method from Web Page

I have a C++ application that listens on a local port for incomming requests from web pages. We used to only listen for GET methods, which were easy, but now we want to listen for POST methods aswell, to get the data that was inserted into the text box on the web page. For some reason, when I click the submit button on the web page all I get is this :

POST /login.htm HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Referer: http://localhost:8080/connections.htm
Accept-Language: en-ca
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Host: localhost:8080
Content-Length: 14
Connection: Keep-Alive
Cache-Control: no-cache

What I'm not getting is the userName=Hello that is supposed to be comming through. Is anyone familiar enough with HTTP to know how to get the content of the webpage? The solution is going to have to work for at least Internet Explorer and Firefox. I've already discovered that by using IE to load up a cached page, I can click the submit button and successfully get the whole POST message, but since other browsers can't do this it's not a solution.

You sure that's all you're getting? It's showing that there's 14 bytes of content, and 'userName=Hello' is 14 bytes long.

When I was first learning how to use CGI, I just echoed the input to see what it looked like. I forget if I ended up using GET or POST, but I remember one who give me a whole bunch of junk, and hidden amongst it was the variable I wanted. You should be able to just search the input for "variable_name=something", though.

Could you show a bit of your code? The part where you get the data from the socket.
I don't know much C++, but I think your problem might have something to do with the fact that the last line (the one with your "userName=Hello") doesn't end with a newline. I've had the same problem in Perl. Try using recv() (or some C++ equivalent) to get the remaining data.

Well you could try http://curl.haxx.se/ as a library to remove some of the grubby details from your code.

You are probably not getting the rest of the POST request, because all the information isn't being copied to your recv() buffer. Try doing another call to recv(), and see if you get anything else.

Code:

---

QString pageReq;
QSocket* socket = (QSocket*)sender();
if( socket->canReadLine() && m_HttpPageInfo.size() != 0) {
        QStringList tokens = QStringList::split(QRegExp("[ \r\n][ \r\n]*"), socket->readLine());
        QTextStream os(socket);
        os.setEncoding(QTextStream::UnicodeUTF8);
        if (tokens[0] == "GET") {
                pageReq = tokens[1];
                //Performs a search for the requested page
                //and returns it using the textStream "os"
        } else if( tokens[0] == "POST" ) {
                QStringList tokens2;
                while (socket->canReadLine()) {
                        pageReq = socket->readLine();
                }
        }
}

---

Here's the code that listens and reads the socket information. As you can see I make use of the QT libraries from http://doc.trolltech.com/3.3/classes.html. I use a while loop for the POST because I knew that when the POST message comes through, the actual content is always the last piece. But when I started to realize that it wasn't comming I added test code to output every readLine to a text file which is where this came from :

POST /login.htm HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Referer: http://localhost:8080/connections.htm
Accept-Language: en-ca
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Host: localhost:8080
Content-Length: 14
Connection: Keep-Alive
Cache-Control: no-cache

It's definetly knows about the content because it sees that it is 14 characters long, but it's definetly not sending the content. I've even tried adding extra lines of pageReq = socket->readLine(); after exiting the while loop, but it's just not there to read.

On the HTML side it's just a test page, so it's just the bare basics plus the following:

Code:

---

<form>
  <input type="text" name="userName" value="Hello">
  <input type="submit" value="Submit">
</form>

---

Personally I think the most odd thing is that the POST doesn't send the content for a normal page, but it does send the content if it was a cached page, and then every other submit after a cached page submit also works fine. Does anyone know of a difference when posting through a normal vs cached page? Whatever the difference is, it's the reason why I either do or don't get the data.

No the problem isn't that the content isn't being sent (because it has been sent). Your problem is the fact you use "readLine()". This only reads something when it has a \n at the end of it. The POST data doesn't have a \n at the end of it.

Try something like:

Code:

---

else if( tokens[0] == "POST" ) {
  char buffer[1000];
  socket->readBlock(buffer,1000);
  printf("%s",buffer);
}

---

If I'm right buffer should have everything in it, including the POST data

Great job guys, that's the problem. Cached pages apparently were sending the extra \r\n that I needed to readLine(). I simply put an extra <input type="hidden" name="End" Value="\r\n"> at the end of every web page so that \r\n is the last characters sent. It was either that or add readBlock() at the end of the readLine() loop, and check the buffer for content.