HTTP Headers

This is a discussion on HTTP Headers within the Networking/Device Communication forums, part of the General Programming Boards category; Hi everyone. I'm trying to code a Wikipedia bot using libcurl. After a request, the following is in my 'header' ...

  1. #1
    Registered User
    Join Date
    Feb 2011
    Posts
    126

    HTTP Headers

    Hi everyone. I'm trying to code a Wikipedia bot using libcurl. After a request, the following is in my 'header' buffer. Now, I want to parse this for errors, which is not easy because I'm trying to use C99 and <string.h>

    1. Is it normal to get 3 or more than 1 header in reply?

    2. If I do, should I simply strstr the lot of them for all the common MediaWiki API errors? Can I assume that if there's an error, it will be in the last one?

    Code:
    HTTP/1.0 200 OK
    Date: Sun, 06 Mar 2011 07:54:39 GMT
    Server: Apache
    Cache-Control: private
    Vary: Accept-Encoding
    Content-Length: 513
    Content-Type: text/xml; charset=utf-8
    X-Cache: MISS from sq33.wikimedia.org
    X-Cache-Lookup: MISS from sq33.wikimedia.org:3128
    X-Cache: MISS from sq63.wikimedia.org
    X-Cache-Lookup: MISS from sq63.wikimedia.org:80
    Connection: close
    
    HTTP/1.0 200 OK
    Date: Sun, 06 Mar 2011 07:54:40 GMT
    Server: Apache
    Cache-Control: public, s-maxage=0, max-age=2678400
    Last-Modified: Sun, 06 Mar 2011 07:51:25 GMT
    Vary: Accept-Encoding
    Content-Length: 386
    Content-Type: text/x-wiki; charset=UTF-8
    X-Cache: MISS from sq63.wikimedia.org
    X-Cache-Lookup: MISS from sq63.wikimedia.org:3128
    X-Cache: MISS from sq71.wikimedia.org
    X-Cache-Lookup: MISS from sq71.wikimedia.org:80
    Connection: close
    
    HTTP/1.0 200 OK
    Date: Sun, 06 Mar 2011 07:54:41 GMT
    Server: Apache
    Cache-Control: private
    Vary: Accept-Encoding
    Content-Length: 933
    Content-Type: text/html; charset=utf-8
    X-Cache: MISS from sq34.wikimedia.org
    X-Cache-Lookup: MISS from sq34.wikimedia.org:3128
    X-Cache: MISS from sq65.wikimedia.org
    X-Cache-Lookup: MISS from sq65.wikimedia.org:80
    Connection: close

  2. #2
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,490
    It could be anything at all.

    The only thing you can rely on for HTTP is getting the double \r\n at the end of the header. That's the only clue you have that you've got the whole header.

    Whether you get the whole header in a single recv() call, or one byte at a time is entirely down to the network transport at that moment in time. You need to reassemble the received message into a string you can parse.

    Or are you asking why you have 3 headers to begin with?
    What sort of request did you send?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  3. #3
    Registered User
    Join Date
    Feb 2011
    Posts
    126
    Well, libcurl claims that it won't return until the whole message is received. I'm not sure how it tells that the whole message has been received, but it does.

    This was the result of an HTTP_POST request.

    Richard

  4. #4
    and the hat of wrongness Salem's Avatar
    Join Date
    Aug 2001
    Location
    The edge of the known universe
    Posts
    32,490
    Ah - missed that bit - sorry.

    Well it seems you've got 3 headers, with 3 different content lengths.

    What was the POST request?
    If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.
    If at first you don't succeed, try writing your phone number on the exam paper.
    I support http://www.ukip.org/ as the first necessary step to a free Europe.

  5. #5
    Registered User
    Join Date
    Feb 2011
    Posts
    126

    Don't mind me

    Actually, I think this is my mistake. I failed to reset the pointers sent to libcurl, so the headers were written after each other instead of over the top of each other. My bad. I'm a newb.

    Richard

  6. #6
    Banned
    Join Date
    Aug 2010
    Location
    Ontario Canada
    Posts
    9,547
    Hi Richard... you might find this helpful...

    RFC Sourcebook

    Look up the HTTP Protocal...

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Function call from another .c module
    By Ali.B in forum C Programming
    Replies: 14
    Last Post: 08-03-2009, 11:45 AM
  2. Small HTTP proxy
    By zacs7 in forum Networking/Device Communication
    Replies: 8
    Last Post: 09-11-2007, 03:53 AM
  3. Read HTTP headers
    By zacs7 in forum Networking/Device Communication
    Replies: 14
    Last Post: 07-13-2007, 01:48 PM
  4. Writing all HTTP requests from a website to a log file
    By goomyman in forum C# Programming
    Replies: 1
    Last Post: 07-29-2005, 09:18 AM
  5. Headers that use each other
    By nickname_changed in forum C++ Programming
    Replies: 7
    Last Post: 10-03-2003, 04:25 AM

Tags for this Thread


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21