Thread: Search Engines :: Winsock

  1. #1
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348

    Search Engines :: Winsock

    Hi.

    I will begin work on a feature in a program to do a search on search engines such as Google and Yahoo. For example, the user will type in whatever phrase he/she wants such as "race & cars," and the program will do a search via Google and print the responses.

    I am not sure what message I need to send and where to send it if I want to search from, for example, Google. Do I send this:

    // "http://www.google.com/search?hl=en&lr=&ie=UTF8&oe=UTF8&q=race+%26+cars//

    I pasted the code above directly from the URL when I did a search at Google.

    Secondly, what data will the search engine return? For example, the search above came back with more than ten pages at Google. Will Google send back all pages or one page at a time?

    Please add if you have any experience dealing with search engines and/or communicating with websites in general.

    Thanks,
    Kuphryn

  2. #2
    Unregistered
    Guest
    Using Telnet (in Windows) connect to google.com like this:

    telnet google.com 80

    Do that from the console prompt. Then you would use the HTTP command "GET" which retrieves stuff from the server. Like so:

    GET /search?hl=en&lr=&ie=UTF8&oe=UTF8&q=race+%26+cars

    This will get the the first page of the results. To get the next lot of results, you'd have to GET the data from page 2 of the results like so:

    GET /search?q=race+%26+cars&hl=en&lr=&ie=UTF8&oe=UTF8&s tart=10&sa=N

    (and so on...)

    The search only returns one page at a time so you can't receive all the data at once. You'd probably have to issue multiple GET commands. You should read up on the HTTP protocol.

  3. #3
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    Okay. Thanks.

    Kuphryn

  4. #4
    Registered User
    Join Date
    Nov 2001
    Posts
    1,348
    I found out Google offers Google API that will help with the search and make the programming process much, much easier and quicker. The only drawback is the fact that under the license agreement, you can only make 1,000 query. I think I have end up having to use the Google API because of the simplicity.

    I really want full control over the search via programming skill. In other words, I would like to do any searches via programming Win32 API instead of going through Google.

    Kuphryn

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Binary Search Trees Part III
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 10-02-2004, 03:00 PM
  2. Request for comments
    By Prelude in forum A Brief History of Cprogramming.com
    Replies: 15
    Last Post: 01-02-2004, 10:33 AM
  3. Anyone know how search engines work?
    By W.Churchill in forum A Brief History of Cprogramming.com
    Replies: 8
    Last Post: 03-20-2002, 08:07 PM
  4. search engines
    By iain in forum A Brief History of Cprogramming.com
    Replies: 16
    Last Post: 01-24-2002, 12:31 PM