A socket connecting to a website, and fetching the data returned

This is a discussion on A socket connecting to a website, and fetching the data returned within the C++ Programming forums, part of the General Programming Boards category; Google wouldn't help me, and I didn't find anything on these boards either, so I'm just gonna go ahead and ...

  1. #1
    Registered User
    Join Date
    Oct 2007
    Posts
    9

    A socket connecting to a website, and fetching the data returned

    Google wouldn't help me, and I didn't find anything on these boards either, so I'm just gonna go ahead and make a new thread.

    I'm not quite that good with C++ yet, but I'm not that much of a beginner anymore either.
    Now, what I am especially interested in is working with websites, and I'm that type of person that learns best by looking at the source code. However, when searching I only found complex classes, or something for which I wasn't looking.

    I have a plea to you, dear advanced C++ programmers. Could you please write a simple connect to a website, fetching data and saving it in a variable? No big extras, just that. I know I can help myself to understand it, once I have the code.

    Thanks!

  2. #2
    Registered User
    Join Date
    Dec 2007
    Posts
    6
    Hi mate,

    You need to look into winsock programming. To do what you want is really quite simple. You just initialise winsock. Create a socket and give it the server information that you wish to connect to (ip, port number etc) and then simply connect to the server. Once connected you send data to it using the http protocol and read the response back and store it in a buffer. Then parse the buffer looking for what you are interested in.

    This code below should be roughly what you are looking for

    Code:
    WSADATA wsaData;  
    
     WSAStartup(MAKEWORD(1, 1), &wsaData) ;
    
    int sockfd, numbytes;  
    	char buf[MAXDATASIZE];
    
    	struct sockaddr_in their_addr;
    
    	sockfd = socket(PF_INET, SOCK_STREAM, 0) ;
    	
    	their_addr.sin_family = AF_INET; 
    	their_addr.sin_port = htons(PORT_HERE);   
    	their_addr.sin_addr = inet_addr("IP_HERE");
    
    	memset(their_addr.sin_zero, '\0', sizeof their_addr.sin_zero);
    
    	connect(sockfd, (struct sockaddr *)&their_addr, sizeof their_addr) ;
    	
    	send(sockfd, "GET /index.html HTTP/1.0\r\n\r\n", 28,0) ; //28 is the string length
    
    	numbytes=recv(sockfd, buf, MAXDATASIZE-1, 0) ; // buf now contains the index.html code
    
    	close(sockfd) ;
    Note i didnt compile or test the above code but it should be good enough for you to understand whats going on. Oh and dont forget your error checking

    Hope this helps, and merry christmas
    Last edited by Codeslinger; 12-25-2007 at 08:06 AM.

  3. #3
    Registered User
    Join Date
    Oct 2007
    Posts
    9
    why thank you! Merry christmas to you too!

  4. #4
    Frequently Quite Prolix dwks's Avatar
    Join Date
    Apr 2005
    Location
    Canada
    Posts
    8,046
    For a nice tutorial, see: http://beej.us/guide/bgnet/

    [Next time, consider posting threads like this in the Network Programming Forum.]
    dwk

    Seek and ye shall find. quaere et invenies.

    "Simplicity does not precede complexity, but follows it." -- Alan Perlis
    "Testing can only prove the presence of bugs, not their absence." -- Edsger Dijkstra
    "The only real mistake is the one from which we learn nothing." -- John Powell


    Other boards: DaniWeb, TPS
    Unofficial Wiki FAQ: cpwiki.sf.net

    My website: http://dwks.theprogrammingsite.com/
    Projects: codeform, xuni, atlantis, nort, etc.

  5. #5
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,489
    Or use a library, so you don't have to talk HTTP directly to the website.
    http://curl.haxx.se/
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  6. #6
    Malum in se abachler's Avatar
    Join Date
    Apr 2007
    Posts
    3,189
    or use the Win32 API, so you dont have to link in a library.


    Until you can build a working general purpose reprogrammable computer out of basic components from radio shack, you are not fit to call yourself a programmer in my presence. This is cwhizard, signing off.

  7. #7
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,489
    > or use the Win32 API, so you dont have to link in a library.
    So that isn't a library then huh?
    Of course it is, it isn't magic.

    More vendor lock-in.
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

Popular pages Recent additions subscribe to a feed

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21