sending an http GET request in C

This is a discussion on sending an http GET request in C within the C Programming forums, part of the General Programming Boards category; here is my code: Code: /* * File: webClient.c * Author: Adam * * Created on November 3, 2011, 1:26 ...

  1. #1
    Registered User
    Join Date
    Nov 2009
    Posts
    16

    sending an http GET request in C

    here is my code:
    Code:
    /* 
     * File:   webClient.c
     * Author: Adam
     *
     * Created on November 3, 2011, 1:26 AM
     */
    
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <string.h>
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <netinet/in.h>
    #include <netdb.h>
    
    /*
     * 
     */
    int main(int argc, char** argv) {
        
        char arg[500];
        char firstHalf[500];
        char secondHalf[500];
        char request[1000];
        struct hostent *server;
        struct sockaddr_in serveraddr;
        int port = 80;
        
        strcpy(arg, argv[1]);
        
        int i;
        for (i = 0; i < strlen(arg); i++)
        {
            if (arg[i] == '/')
            {
                    strncpy(firstHalf, arg, i);
                    firstHalf[i] = '\0';
                    break;
            }     
        }
        
        for (i; i < strlen(arg); i++)
        {
            strcat(secondHalf, &arg[i]);
            break;
        }
        
        printf("\nFirst Half: %s", firstHalf);
        
        printf("\nSecond Half: %s", secondHalf);
        
        int tcpSocket = socket(AF_INET, SOCK_STREAM, 0);
        
        if (tcpSocket < 0)
            printf("\nError opening socket");
        else
            printf("\nSuccessfully opened socket");
        
        server = gethostbyname(firstHalf);
        
        if (server == NULL)
        {
            printf("gethostbyname() failed\n");
        }
        else
        {
            printf("\n%s = ", server->h_name);
            unsigned int j = 0;
            while (server -> h_addr_list[j] != NULL)
            {
                printf("%s", inet_ntoa(*(struct in_addr*)(server -> h_addr_list[j])));
                j++;
            }
        }
        
        printf("\n");
    
        bzero((char *) &serveraddr, sizeof(serveraddr));
        serveraddr.sin_family = AF_INET;
    
        bcopy((char *)server->h_addr, (char *)&serveraddr.sin_addr.s_addr, server->h_length);
        
        serveraddr.sin_port = htons(port);
        
        if (connect(tcpSocket, (struct sockaddr *) &serveraddr, sizeof(serveraddr)) < 0)
            printf("\nError Connecting");
        else
            printf("\nSuccessfully Connected");
      
        bzero(request, 1000);
    
        sprintf(request, "Get %s HTTP/1.1\r\n Host: %s\r\n \r\n \r\n", secondHalf, firstHalf);
        
        printf("\n%s", request);
        
        if (send(tcpSocket, request, strlen(request), 0) < 0)
            printf("Error with send()");
        else
            printf("Successfully sent html fetch request");
        
        bzero(request, 1000);
        
        recv(tcpSocket, request, 999, 0);
        printf("\n%s", request);
        printf("\nhello");
        
        close(tcpSocket);
        
        return (EXIT_SUCCESS);
    }
    i get hung up on the send while the recv code is included. but if i comment the recv line out then the send works successfully. what am i doing wrong?

  2. #2
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by c++guy View Post
    here is my code:
    Code:
        sprintf(request, "Get %s HTTP/1.1\r\n Host: %s\r\n \r\n \r\n", secondHalf, firstHalf);
    Technically, that is a bad request, because you have a space before Host:, but per RFC2616, there should not be any whitespace in a header field name.

    Worse: you do not want a space between your /r/n's. Of which two should separate the header from the body ("/r/n/r/n"), not three. That may not seem like such a big issue because there is no body, but you still need an "/r/n/r/n" to indicate the header is complete, and you do not have that. Hence, I think many or most web servers will not respond in kind, and will instead terminate the connection immediately because the request is malformed. That termination takes place via tcp/ip, not http, and if you are doing the wrong thing with the socket (eg, waiting on recv) at the time, it could be fatal to the process.*

    i get hung up on the send while the recv code is included. but if i comment the recv line out then the send works successfully.
    How are you are deducing this? Do you mean in the former case "Error with send()" appears? Or nothing? If it is nothing, I would bet that the send did succeed, but the stdout buffer is not flushed before your process recieves a SIGPIPE or SIGHUP while waiting on the recv. Try it with:

    Code:
        if (send(tcpSocket, request, strlen(request), 0) < 0)
            fprintf(stderr, "Error with send()");
        else
            fprintf(stderr, "Successfully sent html fetch request");
    This will avoid buffering issues with stdout if the process has actually died prematurely while waiting on the recv(). Ie, the the connection was terminated via tcp/ip by the connected server, without sending any http reponse. How servers deal with non-conforming clients is...undefined.

    Of course, if you do not wait on the recv(), you will get "Successfully sent html fetch request"; you sent a bad request and so the server (on a tcp/ip level) killed the connection (just as before), but it does not matter because your program already ended (and flushed the stdout buffer on exit).

    * you almost certainly want to use a signal handler for SIGPIPE because of that.
    Last edited by MK27; 11-03-2011 at 02:16 PM.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  3. #3
    Registered User
    Join Date
    Nov 2009
    Posts
    16
    Quote Originally Posted by MK27 View Post
    Technically, that is a bad request, because you have a space before Host:, but per RFC2616, there should not be any whitespace in a header field name.

    Worse: you do not want a space between your /r/n's. Of which two should separate the header from the body ("/r/n/r/n"), not three. That may not seem like such a big issue because there is no body, but you still need an "/r/n/r/n" to indicate the header is complete, and you do not have that. Hence, I think many or most web servers will not respond in kind, and will instead terminate the connection immediately because the request is malformed. That termination takes place via tcp/ip, not http, and if you are doing the wrong thing with the socket (eg, waiting on recv) at the time, it could be fatal to the process.*



    How are you are deducing this? Do you mean in the former case "Error with send()" appears? Or nothing? If it is nothing, I would bet that the send did succeed, but the stdout buffer is not flushed before your process recieves a SIGPIPE or SIGHUP while waiting on the recv. Try it with:

    Code:
        if (send(tcpSocket, request, strlen(request), 0) < 0)
            fprintf(stderr, "Error with send()");
        else
            fprintf(stderr, "Successfully sent html fetch request");
    This will avoid buffering issues with stdout if the process has actually died prematurely while waiting on the recv(). Ie, the the connection was terminated via tcp/ip by the connected server, without sending any http reponse. How servers deal with non-conforming clients is...undefined.

    Of course, if you do not wait on the recv(), you will get "Successfully sent html fetch request"; you sent a bad request and so the server (on a tcp/ip level) killed the connection (just as before), but it does not matter because your program already ended (and flushed the stdout buffer on exit).

    * you almost certainly want to use a signal handler for SIGPIPE because of that.
    you are awesome! it was because of the spaces and the fact that i had 3 \r\n's. my teacher had 3 in his provided code for some reason so i had no idea that had a chance at being incorrect. thanks a lot!

  4. #4
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,535
    Two threads on this forum, and cross-posted as well.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Sending http get request
    By LeeVerr in forum C Programming
    Replies: 4
    Last Post: 09-07-2010, 09:02 PM
  2. HTTP GET request
    By aosmith in forum C Programming
    Replies: 4
    Last Post: 03-21-2010, 07:00 AM
  3. http head request
    By nishanthaMe in forum C Programming
    Replies: 5
    Last Post: 10-01-2007, 11:57 PM
  4. Sending an http POST request
    By jaxen in forum C++ Programming
    Replies: 5
    Last Post: 11-24-2006, 11:35 AM
  5. Sending http request to httpd
    By Encrypted in forum C Programming
    Replies: 3
    Last Post: 03-30-2003, 03:57 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21