Thread: Timing HTTP GET requests

  1. #1
    Registered User
    Join Date
    Jul 2008
    Posts
    17

    Timing HTTP GET requests

    I have a need to time the various GET requests in viewing a web page.

    Each time I "get" a web page, it involves many GET requests with each one returning a response code.

    One web page may have multiple GET requests for the CSS, for each of the graphics files, etc.

    I need to count and time each GET request.

    I've looked at cURL library and looked deeply into sockets, but I can't seem to find out how to find each GET request. Then I could add a timer.

    Can anyone point me in the right direction?

    Please help.

    Thank you in advance.

  2. #2
    Registered User
    Join Date
    Dec 2007
    Posts
    2,675
    If you can do it in a web browser, then Firefox and Firebug will give you this info.

  3. #3
    Registered User
    Join Date
    Jul 2008
    Posts
    17
    I need to do this in a C program

  4. #4
    Hurry Slowly vart's Avatar
    Join Date
    Oct 2006
    Location
    Rishon LeZion, Israel
    Posts
    6,788
    how to find each GET request
    you need to parse the html and see - what "child" files are mentioned
    All problems in computer science can be solved by another level of indirection,
    except for the problem of too many layers of indirection.
    – David J. Wheeler

  5. #5
    Registered User
    Join Date
    Jul 2008
    Posts
    17

    Using wireshark

    I see the original GET request:

    GET / HTTP/1.1\r\n

    Then multiple other GET requests:

    GET /Files/styles.css HTTP/1.1\r\n
    GET /Files/functions.js HTTP/1.1\r\n
    GET /Images/shadow_Left.gif HTTP/1.1\r\n

    ...

    If I use libcurl or sockets, I can create the first GET (GET / HTTP/1.1\r\n) but I never create the other GETs.

    I want to see each of the GETs being sent so I can set a timer to see how long each GET takes.

    Any ideas?

  6. #6
    Registered User
    Join Date
    Aug 2008
    Posts
    67
    I want to see each of the GETs being sent so I can set a timer to see how long each GET takes.

    Any ideas?
    Wireshark (formally known as Ethereal)?

  7. #7
    Chinese pâté foxman's Avatar
    Join Date
    Jul 2007
    Location
    Canada
    Posts
    404
    Well, if you are on Linux, you could do something like
    Code:
    wget -p -nv -e robots=off --delete-after webpagerurl
    which will print on stdout every file you need to download to "see" correctly webpagerurl.

    Then, for every file wget just printed, you could
    Code:
    time wget -q --delete-after fileurl
    But I'm not sure if this is what you want. Do you want the time it take to download a specific file or just the time it takes for the server to answer back to a GET request ?
    I hate real numbers.

  8. #8
    Registered User
    Join Date
    Jul 2008
    Posts
    17
    Yes I can use Wireshark. But I want these numbers from a C program. I'd like the same stats as Wireshark shows, but I can't figure out how to get them.

    I'd like the time it takes to connect, time to first byte and time for download.

    I thought about getting a list of all the files from the web page and doing them one at a time, but I see that Wireshark and HttpWatch Professional show the individual times like I would like, but they don't do it independently - as far as I can tell.

    Anyone else have any ideas?

  9. #9
    Banned master5001's Avatar
    Join Date
    Aug 2001
    Location
    Visalia, CA, USA
    Posts
    3,685
    Sure. Why not just poll your most accurate way of telling time on your computer prior to beginning your HTTP transfer and poll it again after you have completed all transfers. Getting stuff off of a web-page is not the most difficult network programming. So are you having trouble parsing HTML or are you just plain not knowing how to do time stamps?

  10. #10
    Registered User
    Join Date
    Aug 2008
    Posts
    67
    Quote Originally Posted by master5001 View Post
    Sure. Why not just poll your most accurate way of telling time on your computer prior to beginning your HTTP transfer and poll it again after you have completed all transfers. Getting stuff off of a web-page is not the most difficult network programming. So are you having trouble parsing HTML or are you just plain not knowing how to do time stamps?
    That's a good idea. For windows, you could compare GetTickCount before and after.

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Persistent HTTP Requests
    By reelbigtim in forum Networking/Device Communication
    Replies: 9
    Last Post: 10-08-2008, 12:06 AM
  2. http requests
    By xixpsychoxix in forum Networking/Device Communication
    Replies: 3
    Last Post: 05-04-2008, 01:13 PM
  3. viewing HTTP requests
    By DavidP in forum A Brief History of Cprogramming.com
    Replies: 6
    Last Post: 04-09-2007, 06:44 PM
  4. Writing all HTTP requests from a website to a log file
    By goomyman in forum C# Programming
    Replies: 1
    Last Post: 07-29-2005, 09:18 AM