HTTP Protocol :: Winsock

This is a discussion on HTTP Protocol :: Winsock within the Windows Programming forums, part of the Platform Specific Boards category; Hi. I posted a thread about adding a feature to a program that searches a websites via Google. I have ...

  1. #1
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348

    HTTP Protocol :: Winsock

    Hi.

    I posted a thread about adding a feature to a program that searches a websites via Google. I have the basic design and implementaton done. Now I am debugging it.

    I have successfully design and implement a chat program using TCP/IP and non-blocking mode via WAsynSelect(). I am using that same design with this website algorithm. I am running into a problem in this early debugging stage.

    The program is I am not sure if TCP/IP and HTTP are the same or are similar. With TCP/IP and using WAsynSelect() mode, the non-blocking model relies on the Windows messaging system. With HTTP protocol, will the server, i.e. Google, send messages about the read, write, and/or close status of a socket (the socket connects the user computer to Google server via an Internet).

    Thanks,
    Kuphryn

  2. #2
    Unregistered
    Guest
    It should'nt matter what protocol you're using, the FD_READ message should catch any incoming data whatever protocol is being used. I don't see any reason for it to be any different if you are using the Google API.

  3. #3
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Thanks.

    I am not using Google API.

    Kuphryn

  4. #4
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,600
    I am not using Google API.
    Funny...

    I don't understand you're problem? Can you explain it better?
    Personally, I have no current need for any of the advanced Winsock functions, so my insight may be irelevant to your non-blocking routines. Still, I am curious to know what you mean...
    Code:
    if( numeric_limits< byte >::digits != bits_per_byte )
        error( "program requires bits_per_byte-bit bytes" );
    24bbs.cpp

  5. #5
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Success! Thanks wrathgame of GameDev for the nice HTTP GET code and the other modifying HTTP handshake codes.

    The program I am working on can now do a search on Google and filter out sites. The only problem I face now is determine when to stop sending GET for addition pages. Google has a limit on the number of pages in a search. For example, if you do a search for "car," I believe Google returns about 800 or so pages. Google sends ten hits per GET command, so you send the GET command 80 times. With programming, I am not sure how to determine the last page Google will send back.

    Please post if there is some HTTP code that Google sends back indication that it is the last page for a particular searchkey.

    Thanks,
    Kuphryn

  6. #6
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,600
    Run the program and ask for more pages than allowed, each iteration printing the recieved page to file. Look in the file and note the message recieved when you max out on requests. Hard-code that message into the proggie
    Code:
    if( numeric_limits< byte >::digits != bits_per_byte )
        error( "program requires bits_per_byte-bit bytes" );
    24bbs.cpp

  7. #7
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Yeah. That is the same technique I am trying. The problem is Google continue sending new pages. I have no way of knowing when the last page comes around so I can stop the GET command and analyze the output data.

    I will keep track of all sites from the previous GET command and compare the new data. If the two are the same or there are no hits, then I will stop the GET command.

    Kuphryn

  8. #8
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,600
    Sounds reasonable...hey BTW, when you're done, why not post the EXE here?
    Code:
    if( numeric_limits< byte >::digits != bits_per_byte )
        error( "program requires bits_per_byte-bit bytes" );
    24bbs.cpp

  9. #9
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Sure. I have some minor tweaks I want to finish. I will release the first beta on my website. I should finish the beta sometime today.

    Kuphryn

  10. #10
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Okay. I have just compiled the first beta release. Please tell me if there is any bug especially in the keyword you use to search.

    http://www22.brinkster.com/rakryn/software.html

    Thanks,
    Kuphryn

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. HTTP Protocol
    By konartis_3000 in forum Tech Board
    Replies: 3
    Last Post: 01-18-2004, 05:09 PM
  2. HTTP Downloads :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 6
    Last Post: 01-13-2003, 10:12 PM
  3. Standard HTTP Characters :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 2
    Last Post: 01-12-2003, 11:21 AM
  4. HTTP GET Index Command :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 2
    Last Post: 12-17-2002, 09:45 AM
  5. http protocols through winsock
    By Unregistered in forum Windows Programming
    Replies: 1
    Last Post: 02-21-2002, 07:29 AM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21