Thread: HTTP Protocol :: Winsock

  1. 06-20-2002 #1
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348

    HTTP Protocol :: Winsock

    Hi.

    I posted a thread about adding a feature to a program that searches a websites via Google. I have the basic design and implementaton done. Now I am debugging it.

    I have successfully design and implement a chat program using TCP/IP and non-blocking mode via WAsynSelect(). I am using that same design with this website algorithm. I am running into a problem in this early debugging stage.

    The program is I am not sure if TCP/IP and HTTP are the same or are similar. With TCP/IP and using WAsynSelect() mode, the non-blocking model relies on the Windows messaging system. With HTTP protocol, will the server, i.e. Google, send messages about the read, write, and/or close status of a socket (the socket connects the user computer to Google server via an Internet).

    Thanks,
    Kuphryn
  2. 06-20-2002 #2
    Unregistered
    Guest
    It should'nt matter what protocol you're using, the FD_READ message should catch any incoming data whatever protocol is being used. I don't see any reason for it to be any different if you are using the Google API.
  3. 06-20-2002 #3
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Thanks.

    I am not using Google API.

    Kuphryn
  4. 06-20-2002 #4
    Sebastiani
    Sebastiani is offline
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    I am not using Google API.
    Funny...

    I don't understand you're problem? Can you explain it better?
    Personally, I have no current need for any of the advanced Winsock functions, so my insight may be irelevant to your non-blocking routines. Still, I am curious to know what you mean...
    Code:
    #include <cmath>
#include <complex>
bool euler_flip(bool value)
{
    return std::pow
    (
        std::complex<float>(std::exp(1.0)), 
        std::complex<float>(0, 1) 
        * std::complex<float>(std::atan(1.0)
        *(1 << (value + 2)))
    ).real() < 0;
}
  5. 06-21-2002 #5
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Success! Thanks wrathgame of GameDev for the nice HTTP GET code and the other modifying HTTP handshake codes.

    The program I am working on can now do a search on Google and filter out sites. The only problem I face now is determine when to stop sending GET for addition pages. Google has a limit on the number of pages in a search. For example, if you do a search for "car," I believe Google returns about 800 or so pages. Google sends ten hits per GET command, so you send the GET command 80 times. With programming, I am not sure how to determine the last page Google will send back.

    Please post if there is some HTTP code that Google sends back indication that it is the last page for a particular searchkey.

    Thanks,
    Kuphryn
  6. 06-21-2002 #6
    Sebastiani
    Sebastiani is offline
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Run the program and ask for more pages than allowed, each iteration printing the recieved page to file. Look in the file and note the message recieved when you max out on requests. Hard-code that message into the proggie
    Code:
    #include <cmath>
#include <complex>
bool euler_flip(bool value)
{
    return std::pow
    (
        std::complex<float>(std::exp(1.0)), 
        std::complex<float>(0, 1) 
        * std::complex<float>(std::atan(1.0)
        *(1 << (value + 2)))
    ).real() < 0;
}
  7. 06-21-2002 #7
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Yeah. That is the same technique I am trying. The problem is Google continue sending new pages. I have no way of knowing when the last page comes around so I can stop the GET command and analyze the output data.

    I will keep track of all sites from the previous GET command and compare the new data. If the two are the same or there are no hits, then I will stop the GET command.

    Kuphryn
  8. 06-21-2002 #8
    Sebastiani
    Sebastiani is offline
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Sounds reasonable...hey BTW, when you're done, why not post the EXE here?
    Code:
    #include <cmath>
#include <complex>
bool euler_flip(bool value)
{
    return std::pow
    (
        std::complex<float>(std::exp(1.0)), 
        std::complex<float>(0, 1) 
        * std::complex<float>(std::atan(1.0)
        *(1 << (value + 2)))
    ).real() < 0;
}
  9. 06-21-2002 #9
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Sure. I have some minor tweaks I want to finish. I will release the first beta on my website. I should finish the beta sometime today.

    Kuphryn
  10. 06-21-2002 #10
    kuphryn
    kuphryn is offline
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Okay. I have just compiled the first beta release. Please tell me if there is any bug especially in the keyword you use to search.

    http://www22.brinkster.com/rakryn/software.html

    Thanks,
    Kuphryn
« Previous Thread | Next Thread »
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. HTTP Protocol
    By konartis_3000 in forum Tech Board
    Replies: 3
    Last Post: 01-18-2004, 06:09 PM
  2. HTTP Downloads :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 6
    Last Post: 01-13-2003, 11:12 PM
  3. Standard HTTP Characters :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 2
    Last Post: 01-12-2003, 12:21 PM
  4. HTTP GET Index Command :: Winsock
    By kuphryn in forum Windows Programming
    Replies: 2
    Last Post: 12-17-2002, 10:45 AM
  5. http protocols through winsock
    By Unregistered in forum Windows Programming
    Replies: 1
    Last Post: 02-21-2002, 08:29 AM