Thread: Submitting HTML form and parse respons

  1. #1
    Registered User
    Join Date
    Feb 2006
    Location
    Leiden, NL
    Posts
    12

    Submitting HTML form and parse respons

    Hi everybody,

    I'm currently working on a project which requires data which can be retreived by filling in a HTML on the Internet and submitting it. The server then sends back something like this:

    Code:
    #### Begin of results ####
    
    108
    107
    101
    107
    102
    106
    90
    100
    106
    94
    101
    93
    
    #### End of results (lines produced: 14) ####
    This is not in a HTML format, but raw code (a string is sent back with the data).

    My problem is as follows: I have written a program in C which needs this data. Therefore I want the C program to retrieve that information automatically. I think I have to write a piece of code which opens a socket, sends the appropriate HTML headers to the server, waits for the response and parse it, am I correct?

    Requesting the information is done by sending an URL such as http://server/retrieve.cgi?parameter1=xx&parameter2=xx to the server.

    Does anyone know a tutorial about a similar problem or a piece of example code which I can alter?

    by the way, I'm writing my programs on my Linux box (don't know if that's important information).

  2. #2
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    Or you could use the command line tools like wget and curl to send appropriate HTML requests and save the responses to a file.

    Use ethereal to observe the message sequence between client and server, if you intend to write this yourself.

    Also, moved to the networking forum, who's intro threads you should read.

  3. #3
    Registered User
    Join Date
    Feb 2006
    Posts
    5

    post message

    sprintf(request, "GET %s HTTP/1.0\r\n"
    "Host: %s\r\n"
    "User-Agent: BlaBla/1.0\r\n\r\n",
    "webpage.php",
    "www.somehost.com");

    s_socket.send((BYTE*)request, strlen(request)+1)
    where the socket class is located at:
    www.zalsoft.com getnet source code /baselib/ folder

  4. #4
    Registered User
    Join Date
    Feb 2006
    Location
    Leiden, NL
    Posts
    12

    getting there

    Hi, thanks for the response!

    I've created a string such as:

    Code:
      data = "GET / HTTP/1.1\r\n"
      "Host: www.servername.nl\r\n"
      "Connection: close\r\n"
      "\r\n";
    I'm getting the right response, so that's great!

    In my case I have to have permission to access the data. This is done by a username and password. If I don't provide this information I get a
    Code:
    HTTP/1.1 401 Authorization Required
    response.

    Is there a way to provide the username and password in the http headers? My instinct says this isn't possible and I have to talk to the server several times (to provide the name and password) before I can retrieve the page. If this is the case, I have no idea how this is done through HTTP headers.

    btw, I've tried ethereal (actually, tehtereal) but haven't made any good progress so far.

    edit: I also see a
    Code:
    WWW-Authenticate: Basic realm="Monica"
    in the server response....
    Last edited by IdunnO; 03-07-2006 at 07:17 AM.

  5. #5
    and the hat of int overfl Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    39,660
    Rather than just guessing, why not wander over to http://www.rfc-editor.org/
    and grab the RFC's which define HTTP.

Popular pages Recent additions subscribe to a feed