Thread: HTTP Downloads :: Winsock

Hi.

I am working on a project that includes downloading data from webservers. There is a problem. Some webservers rejects the program request for data. In one example, the webserver responded with an error that the request was ignored because the client (program) is trying to access the data from another server. In terms of web browsers, this is like clicking on a link at www.google.com to download data from www.gamespot.com.

To my understanding, the server is incorrect because the program is communicating directly to the server just as a web browser does. I would like to know how do web server such as the example know where the client originally found the link to the data?

In the example above, is there any HTTP 1.1 code that I need to add to the request so that the server to process the request?

Thanks,
Kuphryn

Okay. Thanks.

Kuphryn

I'm pretty sure it's the referrer which gets checked. If it's an off-site referrer, you get the error. If not, the file is sent.

I'm not exactly sure where the referrer goes in your GET request... you could check the RFC.

I'll capture a few packets and see.

GET /images/hp0.gif HTTP/1.1
Accept: */*
Referer: http://www.google.ca
Accept-Language: en-us

That's what I got. Set the referer (hmm it's spelled wrong) to the website you're attempting to download from.

Okay. Thanks.

I will first try the full URL including HTML file, i.e.

www.google.com/something/index.html

You solution is actually easier to implement though.

Kuphryn

Okay. I have tried both solutions, but none worked. Here is an example of the request.

Code:

/GET http://visualparadox.com/images/no-l...roncall800.jpg HTTP/1.1
Host: visualparadox.com
Referer: http://visualparadox.com/wallpapers/coloroncall.htm
Accept: text/plain, text/html
Accept-Language: en-us
Accept-Encoding: gzip
Connection: Keep-Alive

Again, the request above does not work. Here is one website I tested use to test the request.

http://visualparadox.com/wallpapers/coloroncall.htm
http://visualparadox.com/images/no-l...roncall800.jpg

Kuphryn