Thread: Http Client

  1. #1
    Registered User
    Join Date
    Feb 2008

    Question Http Client

    Hi Everyone

    I am developing for Linux and am looking for a way to retrieve HTTP pages from the internet, I have found a couple of tutorials that have shown me how to get the basics sorted out:

        int sockid;
        int bufsize;
        char buffer[2048];
        struct sockaddr_in socketaddr;
        struct hostent *hostaddr;
        struct servent *servaddr;
        struct protoent *protocol;
        bool threadActive = true;
    	while(threadActive) {
    	    if (!(hostaddr = gethostbyname(my_data->seed))) {
                fprintf(stderr, "Error resolving host: %s", my_data->seed);
            } // if
            else printf("%s", hostaddr->h_name);
            /* clear and initialize socketaddr */
            memset(&socketaddr, 0, sizeof(socketaddr));
            socketaddr.sin_family = AF_INET;
            /* setup the servent struct using getservbyname */
            servaddr = getservbyname(SERVICE, PROTOCOL);
            socketaddr.sin_port = servaddr->s_port;
            memcpy(&socketaddr.sin_addr, hostaddr->h_addr, hostaddr->h_length);
            protocol = getprotobyname(PROTOCOL);
            sockid = socket(AF_INET, SOCK_STREAM, protocol->p_proto);
            if (sockid < 0) {
                fprintf(stderr, "Error creating socket.");
            } // if
            if(connect(sockid, (struct sockaddr*)&socketaddr, sizeof(socketaddr)) == -1) {
                fprintf(stderr, "Error connecting.");
            } // if
            /* send our get request for http */
            if (send(sockid, GET, strlen(GET), 0) == -1) {
                fprintf(stderr, "Error sending data.");
            } // if
            /* read the socket until its clear then exit */
            while ( (bufsize = read(sockid, buffer, sizeof(buffer) - 1))) {
                write(1, buffer, bufsize);
            } // while
    Whilst this works to some degree it does not seem to actually get the http page I am after (for example Google and I get:

    HTTP/1.1 302 Found
    Location: Google
    Cache-Control: private
    Content-Type: text/html; charset=UTF-8
    Set-Cookie: PREF=ID=d861ae030e872f6e:TM=1279574897:LM=12795748 97:S=nfk6koUtrUvXVYpI; expires=Wed, 18-Jul-2012 21:28:17 GMT; path=/;
    Set-Cookie: NID=36=a1WKldAuUvvHIjy_sBH3aI0iH1J-0KEXW5fRisfGG9jDNj_fEDs9wn3x4sF3STR6D0lbxAGWwxu7Ju Rujy4OJyfYoPGG7XaaiODMvRizwLz5CnDUL0d3YjRS62Bp4l8J ; expires=Tue, 18-Jan-2011 21:28:17 GMT; path=/;; HttpOnly
    Date: Mon, 19 Jul 2010 21:28:17 GMT
    Server: gws
    Content-Length: 221
    X-XSS-Protection: 1; mode=block

    <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
    <TITLE>302 Moved</TITLE></HEAD><BODY>
    <H1>302 Moved</H1>
    The document has moved
    <A HREF="">here</A>.
    returned instead of the actual google homepage. I think It's got something to do with the way I cam creating and directing the socket to the google address, in that its using it as a address, where in actual fact I want it to be using the http address / http protocol (perhaps).

    Any suggestions would be most welcome!


  2. #2
    Registered User
    Join Date
    Jul 2010
    Its an HTTP 302 response redirecting to libcURL can be set to follow HTTP redirects to the page you actually want.
    Last edited by qwertylurker; 07-19-2010 at 04:16 PM.

  3. #3
    Registered User
    Join Date
    Feb 2008

    I found the problem, I was not forming my GET request properly

    now: GET Google HTTP/1.1\n\n

    Returns what I need

    Tho thanks for the response qwertyLurker, ill have a look at libcURL as it may well be less hassle!


  4. #4
    Join Date
    Dec 2007
    WAY, WAY, WAY less hassle.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Socket file descriptor valid and then not valid
    By Florian in forum C Programming
    Replies: 3
    Last Post: 05-22-2010, 08:23 AM
  2. Memory Leak in AppWizard-Generated Code
    By jrohde in forum Windows Programming
    Replies: 4
    Last Post: 05-19-2010, 04:24 PM
  3. reading a file to send to an http client
    By xixpsychoxix in forum Networking/Device Communication
    Replies: 4
    Last Post: 06-22-2008, 09:05 AM
  4. Socket Programming Problem!!!!
    By bobthebullet990 in forum Networking/Device Communication
    Replies: 2
    Last Post: 02-21-2008, 07:36 PM
  5. Client works on a LAN but don't send all the data
    By Niara in forum Networking/Device Communication
    Replies: 9
    Last Post: 01-04-2007, 04:44 PM