Thread: Http Client

  1. #1
    Registered User
    Join Date
    Feb 2008
    Posts
    58

    Question Http Client

    Hi Everyone

    I am developing for Linux and am looking for a way to retrieve HTTP pages from the internet, I have found a couple of tutorials that have shown me how to get the basics sorted out:

    Code:
        
        int sockid;
        int bufsize;
        char buffer[2048];
    
        struct sockaddr_in socketaddr;
        struct hostent *hostaddr;
        struct servent *servaddr;
        struct protoent *protocol;
    
        bool threadActive = true;
    
    	while(threadActive) {
    
    	    if (!(hostaddr = gethostbyname(my_data->seed))) {
                fprintf(stderr, "Error resolving host: %s", my_data->seed);
                exit(1);
            } // if
    
            else printf("%s", hostaddr->h_name);
    
            /* clear and initialize socketaddr */
            memset(&socketaddr, 0, sizeof(socketaddr));
            socketaddr.sin_family = AF_INET;
    
            /* setup the servent struct using getservbyname */
            servaddr = getservbyname(SERVICE, PROTOCOL);
            socketaddr.sin_port = servaddr->s_port;
    
            memcpy(&socketaddr.sin_addr, hostaddr->h_addr, hostaddr->h_length);
    
            protocol = getprotobyname(PROTOCOL);
    
            sockid = socket(AF_INET, SOCK_STREAM, protocol->p_proto);
    
            if (sockid < 0) {
                fprintf(stderr, "Error creating socket.");
                exit(1);
            } // if
    
            if(connect(sockid, (struct sockaddr*)&socketaddr, sizeof(socketaddr)) == -1) {
                fprintf(stderr, "Error connecting.");
                exit(1);
            } // if
    
            /* send our get request for http */
            if (send(sockid, GET, strlen(GET), 0) == -1) {
                fprintf(stderr, "Error sending data.");
                exit(1);
            } // if
    
            /* read the socket until its clear then exit */
            while ( (bufsize = read(sockid, buffer, sizeof(buffer) - 1))) {
                write(1, buffer, bufsize);
            } // while
    
            close(sockid);
    Whilst this works to some degree it does not seem to actually get the http page I am after (for example Google and I get:

    HTTP/1.1 302 Found
    Location: Google
    Cache-Control: private
    Content-Type: text/html; charset=UTF-8
    Set-Cookie: PREF=ID=d861ae030e872f6e:TM=1279574897:LM=12795748 97:S=nfk6koUtrUvXVYpI; expires=Wed, 18-Jul-2012 21:28:17 GMT; path=/; domain=.google.com
    Set-Cookie: NID=36=a1WKldAuUvvHIjy_sBH3aI0iH1J-0KEXW5fRisfGG9jDNj_fEDs9wn3x4sF3STR6D0lbxAGWwxu7Ju Rujy4OJyfYoPGG7XaaiODMvRizwLz5CnDUL0d3YjRS62Bp4l8J ; expires=Tue, 18-Jan-2011 21:28:17 GMT; path=/; domain=.google.com; HttpOnly
    Date: Mon, 19 Jul 2010 21:28:17 GMT
    Server: gws
    Content-Length: 221
    X-XSS-Protection: 1; mode=block

    <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
    <TITLE>302 Moved</TITLE></HEAD><BODY>
    <H1>302 Moved</H1>
    The document has moved
    <A HREF="http://www.google.co.uk/">here</A>.
    </BODY></HTML>
    returned instead of the actual google homepage. I think It's got something to do with the way I cam creating and directing the socket to the google address, in that its using it as a address, where in actual fact I want it to be using the http address / http protocol (perhaps).

    Any suggestions would be most welcome!

    David

  2. #2
    Registered User
    Join Date
    Jul 2010
    Posts
    26
    Its an HTTP 302 response redirecting http://google.co.uk/ to http://www.google.co.uk/. libcURL can be set to follow HTTP redirects to the page you actually want.
    Last edited by qwertylurker; 07-19-2010 at 04:16 PM.

  3. #3
    Registered User
    Join Date
    Feb 2008
    Posts
    58
    Hi

    I found the problem, I was not forming my GET request properly

    now: GET Google HTTP/1.1\n\n

    Returns what I need

    Tho thanks for the response qwertyLurker, ill have a look at libcURL as it may well be less hassle!

    David

  4. #4
    Registered User
    Join Date
    Dec 2007
    Posts
    2,675
    WAY, WAY, WAY less hassle.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Socket file descriptor valid and then not valid
    By Florian in forum C Programming
    Replies: 3
    Last Post: 05-22-2010, 08:23 AM
  2. Memory Leak in AppWizard-Generated Code
    By jrohde in forum Windows Programming
    Replies: 4
    Last Post: 05-19-2010, 04:24 PM
  3. reading a file to send to an http client
    By xixpsychoxix in forum Networking/Device Communication
    Replies: 4
    Last Post: 06-22-2008, 09:05 AM
  4. Socket Programming Problem!!!!
    By bobthebullet990 in forum Networking/Device Communication
    Replies: 2
    Last Post: 02-21-2008, 07:36 PM
  5. Client works on a LAN but don't send all the data
    By Niara in forum Networking/Device Communication
    Replies: 9
    Last Post: 01-04-2007, 04:44 PM