Trying to capture POST method from Web Page

This is a discussion on Trying to capture POST method from Web Page within the Networking/Device Communication forums, part of the General Programming Boards category; I have a C++ application that listens on a local port for incomming requests from web pages. We used to ...

  1. #1
    Registered User
    Join Date
    Mar 2005
    Posts
    3

    Trying to capture POST method from Web Page

    I have a C++ application that listens on a local port for incomming requests from web pages. We used to only listen for GET methods, which were easy, but now we want to listen for POST methods aswell, to get the data that was inserted into the text box on the web page. For some reason, when I click the submit button on the web page all I get is this :

    POST /login.htm HTTP/1.1
    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
    Referer: http://localhost:8080/connections.htm
    Accept-Language: en-ca
    Content-Type: application/x-www-form-urlencoded
    Accept-Encoding: gzip, deflate
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
    Host: localhost:8080
    Content-Length: 14
    Connection: Keep-Alive
    Cache-Control: no-cache

    What I'm not getting is the userName=Hello that is supposed to be comming through. Is anyone familiar enough with HTTP to know how to get the content of the webpage? The solution is going to have to work for at least Internet Explorer and Firefox. I've already discovered that by using IE to load up a cached page, I can click the submit button and successfully get the whole POST message, but since other browsers can't do this it's not a solution.

  2. #2
    C++ Developer XSquared's Avatar
    Join Date
    Jun 2002
    Location
    Ontario, Canada
    Posts
    2,718
    You sure that's all you're getting? It's showing that there's 14 bytes of content, and 'userName=Hello' is 14 bytes long.
    Naturally I didn't feel inspired enough to read all the links for you, since I already slaved away for long hours under a blistering sun pressing the search button after typing four whole words! - Quzah

    You. Fetch me my copy of the Wall Street Journal. You two, fight to the death - Stewie

  3. #3
    Senior Member joshdick's Avatar
    Join Date
    Nov 2002
    Location
    Phildelphia, PA
    Posts
    1,146
    When I was first learning how to use CGI, I just echoed the input to see what it looked like. I forget if I ended up using GET or POST, but I remember one who give me a whole bunch of junk, and hidden amongst it was the variable I wanted. You should be able to just search the input for "variable_name=something", though.
    FAQ

    "The computer programmer is a creator of universes for which he alone is responsible. Universes of virtually unlimited complexity can be created in the form of computer programs." -- Joseph Weizenbaum.

    "If you cannot grok the overall structure of a program while taking a shower, you are not ready to code it." -- Richard Pattis.

  4. #4
    return 0;
    Join Date
    Jan 2005
    Location
    Netherlands
    Posts
    89
    Could you show a bit of your code? The part where you get the data from the socket.
    I don't know much C++, but I think your problem might have something to do with the fact that the last line (the one with your "userName=Hello") doesn't end with a newline. I've had the same problem in Perl. Try using recv() (or some C++ equivalent) to get the remaining data.

  5. #5
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,588
    Well you could try http://curl.haxx.se/ as a library to remove some of the grubby details from your code.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  6. #6
    Registered User
    Join Date
    Sep 2004
    Location
    California
    Posts
    3,246
    You are probably not getting the rest of the POST request, because all the information isn't being copied to your recv() buffer. Try doing another call to recv(), and see if you get anything else.

  7. #7
    Registered User
    Join Date
    Mar 2005
    Posts
    3

    Unhappy more info

    Code:
    QString pageReq;
    QSocket* socket = (QSocket*)sender();
    if( socket->canReadLine() && m_HttpPageInfo.size() != 0) {
    	QStringList tokens = QStringList::split(QRegExp("[ \r\n][ \r\n]*"), socket->readLine());
    	QTextStream os(socket);
    	os.setEncoding(QTextStream::UnicodeUTF8);
    	if (tokens[0] == "GET") {
    		pageReq = tokens[1];
    		//Performs a search for the requested page
    		//and returns it using the textStream "os"
    	} else if( tokens[0] == "POST" ) { 
    		QStringList tokens2;
    		while (socket->canReadLine()) {
    			pageReq = socket->readLine();
    		}
    	}
    }
    Here's the code that listens and reads the socket information. As you can see I make use of the QT libraries from http://doc.trolltech.com/3.3/classes.html. I use a while loop for the POST because I knew that when the POST message comes through, the actual content is always the last piece. But when I started to realize that it wasn't comming I added test code to output every readLine to a text file which is where this came from :

    POST /login.htm HTTP/1.1
    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
    Referer: http://localhost:8080/connections.htm
    Accept-Language: en-ca
    Content-Type: application/x-www-form-urlencoded
    Accept-Encoding: gzip, deflate
    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
    Host: localhost:8080
    Content-Length: 14
    Connection: Keep-Alive
    Cache-Control: no-cache

    It's definetly knows about the content because it sees that it is 14 characters long, but it's definetly not sending the content. I've even tried adding extra lines of pageReq = socket->readLine(); after exiting the while loop, but it's just not there to read.

    On the HTML side it's just a test page, so it's just the bare basics plus the following:
    Code:
    <form>
       <input type="text" name="userName" value="Hello">
       <input type="submit" value="Submit">
    </form>
    Personally I think the most odd thing is that the POST doesn't send the content for a normal page, but it does send the content if it was a cached page, and then every other submit after a cached page submit also works fine. Does anyone know of a difference when posting through a normal vs cached page? Whatever the difference is, it's the reason why I either do or don't get the data.

  8. #8
    return 0;
    Join Date
    Jan 2005
    Location
    Netherlands
    Posts
    89
    No the problem isn't that the content isn't being sent (because it has been sent). Your problem is the fact you use "readLine()". This only reads something when it has a \n at the end of it. The POST data doesn't have a \n at the end of it.

    Try something like:

    Code:
    else if( tokens[0] == "POST" ) { 
      char buffer[1000];
      socket->readBlock(buffer,1000);
      printf("%s",buffer);
    }
    If I'm right buffer should have everything in it, including the POST data

  9. #9
    Registered User
    Join Date
    Mar 2005
    Posts
    3

    Solved

    Great job guys, that's the problem. Cached pages apparently were sending the extra \r\n that I needed to readLine(). I simply put an extra <input type="hidden" name="End" Value="\r\n"> at the end of every web page so that \r\n is the last characters sent. It was either that or add readBlock() at the end of the readLine() loop, and check the buffer for content.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Downloading HTML Files from Web Page
    By Unregistered in forum A Brief History of Cprogramming.com
    Replies: 13
    Last Post: 07-18-2002, 05:59 AM
  2. Free Web Page Design Templates
    By Nutshell in forum A Brief History of Cprogramming.com
    Replies: 3
    Last Post: 07-01-2002, 05:59 PM
  3. Creating a web page.............
    By Nutshell in forum A Brief History of Cprogramming.com
    Replies: 8
    Last Post: 04-08-2002, 01:33 AM
  4. web page hosting
    By qwertiop in forum A Brief History of Cprogramming.com
    Replies: 3
    Last Post: 03-29-2002, 11:32 PM
  5. Dialog app interface with web page...
    By Unregistered in forum Windows Programming
    Replies: 5
    Last Post: 01-29-2002, 07:41 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21