![]() |
| | #1 |
| Registered User Join Date: Nov 2006
Posts: 83
| I can still download the file, but InternetReadFile() doesn't always download the entire file if I don't have the EXACT filesize. It's very strange. There must be something I'm doing wrone. Here's the function that I use to grab the files: Code: std::string Internet::getURL(const char *URL)
{
//get size of file
std::string temp = "";
hInternet = InternetOpen( "GINA: Version 0.1", INTERNET_OPEN_TYPE_DIRECT, NULL, 0,0);
hFile = InternetOpenUrl(hInternet, URL, NULL, 0, 0, 0);
DWORD sizeBuffer;
DWORD length = sizeof(sizeBuffer);
char *buffer = NULL;
bool succeeds = HttpQueryInfo(hFile, HTTP_QUERY_CONTENT_LENGTH | HTTP_QUERY_FLAG_NUMBER, (LPVOID)&sizeBuffer, &length, NULL);
if (!succeeds)
{//query fails at /maps/
int i = GetLastError();
char buf[25];
MessageBox(NULL,itoa(i,buf,10),"Error",0);
//MessageBox(NULL,URL,"URL Failed Query",0);
sizeBuffer = 65536; //read everything damn it
}
buffer = new char[sizeBuffer];
//get file
DWORD bytesRead = 0;
DWORD bytesToRead = sizeBuffer;
if (hFile)
{
do
{
InternetReadFile(hFile, buffer, bytesToRead, &bytesRead);
}while (bytesRead != 0);
}
else
throw "Unable to grab File";
temp = buffer;
InternetCloseHandle(hFile);
InternetCloseHandle(hInternet);
return temp;
}
The URLS that fail (all others work fine with this function) are: Code: http://www.imperialconflict.com/maps/ and: http://www.imperialconflict.com/rankings.php?type=topfamilies_score&g=11 hInternet,hFile are members of my class Internet Thanks for any help
__________________ Iphone/ITouch technical reference |
| A10 is offline | |
| | #2 |
| Sweet Join Date: Aug 2002 Location: Tucson, Arizona
Posts: 1,698
| The issue mainly is the use of your buffer. You create the buffer but you assume the buffer is big enough to hold the contents of the download. Which is not the case when you don't get the content length(Which can happen). I re-wrote your code not to be dependent on the content size and used a fixed buffer to download the file. Code: std::string GetUrl(const char *URL)
{
//Constant to hold the read size(1k).
const int DownloadBufferSize = 1024;
//Constant to hold an empty string.
const std::string errorString = "ERROR";
//
//Attempt to open a internet connection. Check for success.
//
HINTERNET hInternet = InternetOpen("GINA: Version 0.1", INTERNET_OPEN_TYPE_DIRECT, NULL, 0,0);
if(hInternet == NULL){
return errorString;
}//if
//
//Attempt to open the url. Check for success
//
HINTERNET hFile = InternetOpenUrl(hInternet, URL, NULL, 0, 0, 0);
if(hFile == NULL){
return errorString;
}//if
//
//These will be used to hold the content size(Potentially)
//
DWORD sizeBuffer;
DWORD length = sizeof(sizeBuffer);
//Attempt to grab the content length of the request.
bool succeeds = HttpQueryInfo(hFile, HTTP_QUERY_CONTENT_LENGTH | HTTP_QUERY_FLAG_NUMBER, &sizeBuffer, &length, NULL) == TRUE;
//Special Case: If the query fails handle failure case, Else output the content length.
if (!succeeds){
//Grab the last error that occured.
DWORD lastError = GetLastError();
//Special Case: Header not found is expected. Not all pages output their content length.
if(lastError != ERROR_HTTP_HEADER_NOT_FOUND){
std::stringstream outputStream;
outputStream<<"Error: "<<lastError;
MessageBox(NULL, outputStream.str().c_str(), "Query Info Error", MB_OK);
}//if
}//if
else{
std::stringstream outputStream;
outputStream<<"Content Length: "<<sizeBuffer;
MessageBox(NULL, outputStream.str().c_str(), "Query Info Error", MB_OK);
}//else
//Will be used to store all of the downloaded data.
std::string downloadedContents = "";
//This will be used to hold the buffer to download.
char *downloadBuffer = new char[DownloadBufferSize];
//Will be used to hold the number of bytes read.
DWORD bytesRead = 0;
do{
//Read the current bytes from the internet file.
InternetReadFile(hFile, downloadBuffer, DownloadBufferSize, &bytesRead);
//Append the buffer to the download contents
downloadedContents.append(downloadBuffer, DownloadBufferSize);
}while(bytesRead != 0);
//
//Close the handles.
//
InternetCloseHandle(hFile);
InternetCloseHandle(hInternet);
//Return the downloaded contents to the caller.
return downloadedContents;
}
You could however use the content size to do a progress bar or something if you wanted.
__________________ Woop? |
| prog-bman is offline | |
| | #3 |
| Registered User Join Date: Nov 2006
Posts: 83
| Thanks for your response. Your rewrite was very very helpful. Unfortunately It wasn't 100% successful. When I took a look at the file the function downloaded, It was chewed up and not necessarily continuous. After a little bit of thinking I came up with a theory and a solution. InternetReadFile() only downloads as much data as is available. Depending on the connection, there isn't always 1k of data available. The file would appear to be chewed up because old data still in the buffer wouldn't have been written over. I've rewritten your function to download and append only as much data is available. I also increased the buffer size to 4096 and commented out the size query as I don't yet have a use for it. The code works flawlessly. Thanks for your help ![]() Code: #include <winInet.h>
#include <string>
#include <windows.h>
//#include <sstream>
std::string getURL(const char *URL)
{
//Constant to hold the read size(4k).
const int downloadBufferSize = 4096;
//Constant to hold an empty string.
const std::string errorString = "ERROR";
//
//Attempt to open a internet connection. Check for success.
//
HINTERNET hInternet = InternetOpen("GINA: Version 0.1", INTERNET_OPEN_TYPE_DIRECT, NULL, 0,0);
if(hInternet == NULL){
return errorString;
}//if
//
//Attempt to open the url. Check for success
//
HINTERNET hFile = InternetOpenUrl(hInternet, URL, NULL, 0, 0, 0);
if(hFile == NULL){
return errorString;
}//if
/*
//
//These will be used to hold the content size(Potentially)
//
DWORD sizeBuffer;
DWORD length = sizeof(sizeBuffer);
//Attempt to grab the content length of the request.
bool succeeds = HttpQueryInfo(hFile, HTTP_QUERY_CONTENT_LENGTH | HTTP_QUERY_FLAG_NUMBER, &sizeBuffer, &length, NULL) == TRUE;
//Special Case: If the query fails handle failure case, Else output the content length.
if (!succeeds){
//Grab the last error that occured.
DWORD lastError = GetLastError();
//Special Case: Header not found is expected. Not all pages output their content length.
if(lastError != ERROR_HTTP_HEADER_NOT_FOUND){
std::stringstream outputStream;
outputStream<<"Error: "<<lastError;
MessageBox(NULL, outputStream.str().c_str(), "Query Info Error", MB_OK);
}//if
}//if
*/
//Will be used to store all of the downloaded data.
std::string downloadedContents = "";
//This will be used to hold the buffer to download.
char *downloadBuffer = new char[downloadBufferSize];
DWORD availableSize = 0;
//Will be used to hold the number of bytes read.
DWORD bytesRead = 0;
do{
//query how much data is available
InternetQueryDataAvailable(hFile,&availableSize,0,0);
//read only as much as is available or downloadBufferSize
if (availableSize > downloadBufferSize)
availableSize = downloadBufferSize;
//Read the current bytes from the internet file.
InternetReadFile(hFile, downloadBuffer, availableSize, &bytesRead);
//Append the buffer to the download contents
downloadedContents.append(downloadBuffer, availableSize);
}while(bytesRead != 0);
//
//Close the handles.
//
InternetCloseHandle(hFile);
InternetCloseHandle(hInternet);
//Return the downloaded contents to the caller.
return downloadedContents;
}
__________________ Iphone/ITouch technical reference |
| A10 is offline | |
| | #4 |
| Sweet Join Date: Aug 2002 Location: Tucson, Arizona
Posts: 1,698
| Perfect. I forgot about that. Looks good, glad I could help.
__________________ Woop? |
| prog-bman is offline | |
![]() |
| Thread Tools | |
| Display Modes | |
|
Similar Threads | ||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| making sprites | DavidP | Game Programming | 9 | 02-20-2010 07:00 AM |
| wininet | jopp3 | C Programming | 0 | 04-29-2007 09:27 AM |
| Making great graphics | MadCow257 | Game Programming | 1 | 02-20-2006 11:59 PM |
| Verbal abuse from school teachers -- how to get them caught or busted? | deltabird | A Brief History of Cprogramming.com | 17 | 02-13-2003 07:26 AM |
| About Unix Programming - Making a career desision | null | C Programming | 0 | 10-14-2001 07:37 AM |