Thread: fetching html pages in C. output only shows html source sometimes

My output usually is something like:
Second Half New Way: /finance
First Half: Google
Google = 74.125.47.10574.125.47.10674.125.47.14774.125.47.9 974.125.47.
10374.125.47.104

GET /finance HTTP/1.1
Host: Google

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Fri, 04 Nov 2011 02:02:48 GMT
Expires: Fri, 04 Nov 2011 02:02:48 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Transfer-Encoding: chunked

but every now and then the program will actually spit back some actual html code of the page. how come i only get the actual html source some of the time?

Code:

/* 
 * File:   webClient.c
 * Author: Adam
 *
 * Created on November 3, 2011, 1:26 AM
 */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netdb.h>

/*
 * 
 */
int main(int argc, char** argv) {
    
    char arg[500];
    char firstHalf[500];
    char secondHalf[500];
    char request[1000];
    struct hostent *server;
    struct sockaddr_in serveraddr;
    int port = 80;
    int tcpSocket;
    
    strcpy(arg, argv[1]);
    
    int i;
    for (i = 0; i < strlen(arg); i++)
    {
        if (arg[i] == '/')
        {
                strncpy(firstHalf, arg, i);
                firstHalf[i] = '\0';
                break;
        }     
    }
    
    strncpy(secondHalf, arg + i, strlen(arg) - i);
    secondHalf[strlen(arg) - i] = '\0';

    
    printf("\nSecond Half New Way: %s", secondHalf);
    
    printf("\nFirst Half: %s", firstHalf);
    
    tcpSocket = socket(AF_INET, SOCK_STREAM, 0);
    
    if (tcpSocket < 0)
        printf("\nError opening socket");
    
    server = gethostbyname(firstHalf);
    
    if (server == NULL)
    {
        printf("gethostbyname() failed\n");
    }
    else
    {
        printf("\n%s = ", server->h_name);
        unsigned int j = 0;
        while (server -> h_addr_list[j] != NULL)
        {
            printf("%s", inet_ntoa(*(struct in_addr*)(server -> h_addr_list[j])));
            j++;
        }
    }
    
    printf("\n");

    bzero((char *) &serveraddr, sizeof(serveraddr));
    serveraddr.sin_family = AF_INET;

    bcopy((char *)server->h_addr, (char *)&serveraddr.sin_addr.s_addr, server->h_length);
    
    serveraddr.sin_port = htons(port);
    
    if (connect(tcpSocket, (struct sockaddr *) &serveraddr, sizeof(serveraddr)) < 0)
        printf("\nError Connecting");
  
    bzero(request, 1000);

    sprintf(request, "GET %s HTTP/1.1\r\nHost: %s\r\n\r\n", secondHalf, firstHalf);
    
    printf("\n%s", request);
    
    if (send(tcpSocket, request, strlen(request), 0) < 0)
        fprintf(stderr, "Error with send()");
    
    bzero(request, 1000);
    
    recv(tcpSocket, request, 999, 0);
    printf("%s", request);
    
    close(tcpSocket);
    
    return (EXIT_SUCCESS);
}

i solved this by adding a print statement for every iteration of the loop and making sure i put null terminator at end of every recv:

Code:

do {
        
    iResult = recv(tcpSocket, request, 999, 0);
    
    if (iResult > 0)
    {
        request[iResult] = '\0';
        printf("%s", request);
        printf("\nBytes received: %d", iResult);
    }
    else if (iResult == 0)
        printf("\nConnection closed");
    else
        perror("recv failed");
    }while(iResult > 0);

also i needed to create a timeout so the loop would eventually exit:

Code:

    struct timeval tv;
    tv.tv_sec = 2;
    setsockopt(tcpSocket, SOL_SOCKET, SO_RCVTIMEO, (struct timeval *)&tv, sizeof(struct timeval));

however this solution doesn't exit loop in cygwin/windows environment like it does in ubuntu. anyone know why?

Originally Posted by Salem

More cross-posting buffoonery
Fetching Html Pages In C.why Does Code Only Return Html Code Sometimes - C And C++ | Dream.In.Code

Time to make a choice

i dont see the problem in it really. realistically there are some people on this board who aren't on dreamincode. and realistically there are some people on dreamincode that aren't on this board. so why not post in both and get answers from more people? i have posted the same question in multiple forums many times where i got answers in all of them but only one of them from one forum was the correct answer that actually helped me. therefore if i had only chosen one arbitrarily and that one i had chosen ended up being one without the helpful answer then i would have been set back. it's only logical to post in as many as possible to get the most helpful answer as quickly as possible. it's all about quickly furthering the knowledge of the human race, not setting arbitrary limits on slowing the gaining of knowledge down because you're not 'supposed to' post in more than one forum because of a pointless social standard.

why don't you guyz forget about who have to do what and focus on code problem please

fair enough. so anyone know why setsockopt() isn't having the same functionality in cygwin when im on windows as it is when i run it in ubuntu?