Socket Programming: Retrieve file from HTTP

Hi. I am writing a program that simply downloads and saves a file over HTTP. See attached code.

I was wondering if anyone had any hints or tips on this matter or perhaps my code in general and point out any glaring problems.

Thanks for any advice.

I had a quick look. If you want you can look at some http stuff i made a few days ago (it can break apart a url and download stuff) and compare yours to mine if yours isnt working.

http://host-a.net/39ster/httpstuff.rar

The files are .cpp but they should compile in C also.

cool thanks. I'll have a look. I've gotten a bit further with mine. I've just got to use memcpy to save the file as binary rather than just text.

Why do you need memcpy()? Just use fwrite() - or are you trying to build a larger block of data before you save it to file?

--
Mats

No I need to skip the header, then write the block of data to a file. Sorry, what I said above sounds odd; I do use fwrite. My GetRidOfHeaders function used strcpy which I now need to change.

Why not just figure out where the headers end, and then fwrite that?

something like:

Code:

---

    char buf[...];
    size_t size;
    size_t totalsize;
    totalsize = getstuff(buf);
    size = headersize(buf);
    fwrite(&buf[size], totalsize - size, 1, file);

---

--
Mats

Yeah that's pretty much what I've been trying to do. I've attached my code, as it's a bit different now. Thanks for the suggestion, what you've got there looks sensible.

Edit: Finished!

What happens if the double \r\n is split between recv() calls?

Also, chunked encoding... when I finally got my first HTTP thing working, chunked encoding bit me in the butt. I now use libcurl... >.>

It's also been my experience that some servers send a "Continue" (namely IIS) before sending the actual data. (I think if the headers get separated on their way there...) Again, libcurl ftw.

If you look at mine, i have a function called recvLine(). This function goes into a loop and receives 1 byte at a time in the recv() function until it reaches a '\n' or until it has reached the buffer size limit. Once you have called this function, trim the end of the buffer (to remove the \n, if there is one) and if the strlen() is 0 than its a blank line....end of the header.

Code:

---

int recvLine(int pSockId, char* pOut, int pOutSize)
{
    int received = 0;
    char letter;
    memset(pOut, 0, pOutSize);
    for(; received < pOutSize-1; received++)
    {
        if(recv(pSockId, (char*)&letter, 1, 0) > 0)
        {
            pOut[received] = letter;
            if(letter == '\n')
                break;
        } else break;
    }
    return received;
}

---

Code:

---

    while((length = recvLine(sockId, buffer, sizeof(buffer))) > 0)
    {
        //trim line
        for(int i = length-1; i >= 0; i--)
        {
            if(buffer[i] < ' ')
                buffer[i] = 0;
            else break;
        }

        //if end of header
        if(!strlen(buffer))
            break;
       
        //Process header
    }
    //receive file data

---

You mean, what if the header doesn't all come out in the first recv call? Yeah you're right, that is something I need to look at. As for libcurl, I wanted to write my own rather than use someone elses library.