Sending binary data via HTTP Response

Hello,

I'm trying to send some binary data, such as an animated gif, via an HTTP response for a simple web server.

I am having issues having the browser close the connection after receipt of the data.

I create the header such as:

"HTTP/1.1 200 OK
Content-Length: <file size>
Content-Type: <file type>

<binary data>"

This is stored in a malloc'ed char *.
To add the binary data I'm using fread:
fread(request + strlen(request), 1, size, fp)

I then write 'total' bits via connfd:
write(connfd, request, total)
where total = strlen(headerRequest) + size + 3 (for ending \r\n\0).

The writing seems to be okay however the image doesn't load and the browser still seems to be waiting for data. I was wondering if you could assist.

If I added "Connection: close" to the header and closed connfd myself the page loads fine. Obviously, for efficiency purposes, I'd rather only close connfd once read() returns 0.

If anyone could be of assistance that would be great.

Further information available on request.

Thank you for your time.

The header creation code is below:
=======

Code:

---

        /* Try to get file */
        strcat(cwd, file);
        if(endsWith (cwd, ".html") || endsWith(cwd, ".htm")) {
                strcpy(fileType, "text/html");
                binary = 0;
        } else if (endsWith(cwd, ".txt")) {
                strcpy(fileType, "text/plain");
                binary = 0;
        } else if (endsWith(cwd, ".jpg") || endsWith(cwd, ".jpeg")) {
                strcpy(fileType, "image/jpeg");
                binary = 1;
        } else if (endsWith(cwd, ".gif")) {
                strcpy(fileType, "image/gif");
                binary = 1;
        } else {
                strcpy(fileType, "application/octet-stream");
                binary = 1;
        }
        if (!binary) {
                if ((fp = fopen(cwd, "r")) == NULL) {
                        return errorFileNotFound(connfd);
                }
        } else {
                if ((fp = fopen(cwd, "rb")) == NULL) {
                        return errorFileNotFound(connfd);
                }
        }
        size = getFileSize(fp);
        /* Total malloc is header length + size + 3 (3 for terminating \r\n\0) */
        printf("%s has size %ld bytes.\n", cwd, size);
        headerRequest = getHeader(fileType, size);
        total = strlen(headerRequest) + size + 3;
        request = malloc(total);
        if (request == NULL) {
                return internalError(connfd);
        }
        request[0] = '\0';
        strcat(request, headerRequest);
        free(headerRequest);
        char filepart[MAXLINE];
        filepart[0] = '\0';
        if (!binary) {
                while (fgets(filepart, MAXLINE, fp) != NULL) {
                        strcat(request, filepart);
                }
                strcat(request, "\r\n");
        } else {
                read = fread(request+strlen(request), 1, size, fp);
                request[total - 3] = '\r';
                request[total - 2] = '\n';
                request[total - 1] = '\0';
        }
        printf("%lu bytes read\n", read);
        fclose(fp);
        printf("\nResponse\n----\n%s\n---\n", request);
        if ((written = write(connfd, request, total)) == -1) {
                printError("Error writing to client.");
                free(request);
                return internalError(connfd);
        }
        printf("%lu written, told server I'd send %lu\n", written, total);
        free(request);
        return 0;

---

you don't need \r\n\0 after the binary data. your packet should look like this. maybe that is what you have. its not clear from your post
[code]
"HTTP/1.1 200 OK\r\n
Content-Length: <file size>\r\n
Content-Type: <file type>\r\n
\r\n
<binary data>

from the spec
Response = Status-Line ; Section 6.1
*(( general-header ; Section 4.5
| response-header ; Section 6.2
| entity-header ) CRLF) ; Section 7.1
CRLF
[ message-body ] ; Section 7.2
[\code]


it would help if you had shown what your header actually contained since your example calls functions that aren't shown.
i recommend that you write out your entire response dataa to a file, then inspect it to see what it really contains and that all the \r\n's are in the right place and that the body actually contains the length you specified in the header.

edit: the browser is probably confused by the trailing /r/n/0 so it is waiting for something else. but when you close the connection yourself, the browser is smart enough to ignore those extra bytes.

Thank you for your response.
A printf of the response looks like everything between the '----' and '---'

Code:

---

Response
----
HTTP/1.1 200 OK
Content-Length: 751635
Content-Type: image/gif


GIF89a�
---

---

The removal of trailing \r\n\0 (and subsequent reduction of request size by 3 if binary data is being sent) has made no change.
Yeah if I close the connection myself the browser is happy however it is wasteful of a connection and I'd rather keep it open if possible.

Are you also suppose to leave 3 new lines or 2? You have 3 here from "gif" to "GIF"

Code:

---

HTTP/1.1 200 OK
Content-Length: 751635
Content-Type: image/gif
 
 
GIF89a�

---

But 2 here

Code:

---

HTTP/1.1 200 OK
Content-Length: <file size>
Content-Type: <file type>

<binary data>"

---

Is also �correct?

It also seems that there is an extra null-terminating string as %s doesn't print it yet you send it

Code:

---

printf("\nResponse\n----\n%s\n---\n", request);
if ((written = write(connfd, request, total)) == -1) {

---

Though I guess that problem went away when you removed /r/n/0. Though if you had a reason to have that trailing /r/n maybe you should just remove the /0

Code:

---

        total = strlen(headerRequest) + size;
        if(!binary) {
                total += 3;
        }
        request = malloc(total);
        if (request == NULL) {
                return internalError(connfd);
        }
        request[0] = '\0';
        strcat(request, headerRequest);
        free(headerRequest);
        char filepart[MAXLINE];
        filepart[0] = '\0';
        if (!binary) {
                while (fgets(filepart, MAXLINE, fp) != NULL) {
                        strcat(request, filepart);
                }
                strcat(request, "\r\n\0");
        } else {
                fread(request+strlen(request), 1, size, fp);
        }
        fclose(fp);
        printf("\nResponse\n----\n%s\n---\n", request);
        if (write(connfd, request, total) == -1) {
                printError("Error writing to client.");
                free(request);
                return internalError(connfd);
        }

---

I've changed the code slightly (albeit to no avail).

Yes I assumed that the reason the printf wasn't printing out screeds and screeds of characters was due to random \0 in a binary file.

There are now the correct number of lines between header head and body (again to no avail).

Code:

---

Response----
HTTP/1.1 200 OK
Content-Length: 14352
Content-Type: image/jpeg


????
---

---

Code:

---

Response----
HTTP/1.1 200 OK
Content-Length: 751635
Content-Type: image/gif


GIF89a??
---

---

https://www.wireshark.org/
Use it for diagnosing all your network programming issues.

Use it to observe what a known functioning browser does.

Use it to observe what your code is doing.
In particular, compare the differences between what works and what doesn't.

another way to test is to use wget to make the request. you can capture the entire response to a file and see what it looks like.

Thanks, I'll try that!