URL help!

This is a discussion on URL help! within the C Programming forums, part of the General Programming Boards category; Forgive me for being a newbie, but here goes... If I enter the following at a unix prompt; $ wget ...

  1. #1
    Registered User
    Join Date
    May 2009
    Posts
    27

    URL help!

    Forgive me for being a newbie, but here goes...
    If I enter the following at a unix prompt;

    $ wget "http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY/200901/ 20090101/narr-a_221_20090101_0000_000.ascii?tmpprs[0][3][94][327]"

    and then do this...

    $ vi "narr-a_221_20090101_0000_000.ascii?tmpprs[0][3][94][327]"

    I get this:

    tmpprs, [1][1][1][1]
    [0][0][0], 272.8044

    time, [1]
    733409.0
    lev, [1]
    925.0
    lat, [1]
    35.25
    lon, [1]
    -97.375


    Which is great.


    To make sure I get the same thing when I compile and run a script, is what I do something along these lines?:


    Code:
     
    #include <stdio.h>
    int main()
    
    /* URL example */
    /*http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY/200901/20090101/ */
    /*        narr-a_221_20090101_0000_000.ascii?tmpprs[0][3][94][327]*/
    
    const char text1 = "http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY/"
    const char text2 = "/narr-a_/221_{%d /n, ymd}_0000_000.ascii?"
    const char tmpprs;
    
    /* ym = year, month (i.e., 200901) */
    /* ymd = year, month, day (i.e., 20090101) */
    const int ym = 200901;
    const int ymd = 20090101;
    const int hour = 0;
    
    int date;
    
    const int pressure_level = 3;
    
    const int longitude = 327;
    const int latitude = 94;
    
    {
    sscanf( buff, "text1","ym","ymd","text2","tmpprs","hour","pressure_level","latitude","longitude");
    }
    
    return ();
    I'm sure this isn't exactly right. Iíve never dealt with URLs in C before. I am only wanting to script something so I can see the output. I don't know what to do about a printf statement in this case because there's a URL. The data will be written to a file, eventually.

    What am I missing?
    "Do. Or do not. There is no try."
    - Yoda

  2. #2
    ATH0 quzah's Avatar
    Join Date
    Oct 2001
    Posts
    14,826
    There are no standard "give me a URL and I'll go fetch that page" functions in C, if that's what you're trying to do. You're also missing your opening and closing braces for your main function.


    Quzah.
    Hope is the first step on the road to disappointment.

  3. #3
    Registered User
    Join Date
    May 2009
    Posts
    27
    Quote Originally Posted by quzah View Post
    There are no standard "give me a URL and I'll go fetch that page" functions in C, if that's what you're trying to do. You're also missing your opening and closing braces for your main function.


    Quzah.

    Thanks on the 'main' function.

    And not the 'URL page'... Just the output. Is that not possible? How come it works at the command prompt?
    "Do. Or do not. There is no try."
    - Yoda

  4. #4
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    >> I'm sure this isn't exactly right.

    Far from it. Have you considered picking up a good C-programming book?
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  5. #5
    Registered User
    Join Date
    May 2009
    Posts
    27
    Quote Originally Posted by Sebastiani View Post
    >> I'm sure this isn't exactly right.

    Far from it. Have you considered picking up a good C-programming book?


    Yes, and I've been through one semester of programming for non-majors (4 years ago). Didn't cover it.


    So how far am I?
    "Do. Or do not. There is no try."
    - Yoda

  6. #6
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    Far enough to require further study, anyway.

    Besides helping you solve your current problem, it will take most of the guesswork out of the ones you'll encounter in the future.
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  7. #7
    Registered User
    Join Date
    May 2009
    Posts
    27
    Quote Originally Posted by Sebastiani View Post
    Far enough to require further study, anyway.

    Besides helping you solve your current problem, it will take most of the guesswork out of the ones you'll encounter in the future.
    I am a geographer, not a programmer, but I am trying to learn. This is a small part of major project using steps I've never encountered before and I'm sitting here with notes, books, etc open feeling just a little too lost.

    Thanks for your time.
    "Do. Or do not. There is no try."
    - Yoda

  8. #8
    Guest Sebastiani's Avatar
    Join Date
    Aug 2001
    Location
    Waterloo, Texas
    Posts
    5,708
    >> This is a small part of major project using steps I've never encountered before and I'm sitting here with notes, books, etc open feeling just a little too lost.

    I understand. In that case, it may help to develop the program in an extremely stepwise fashion, eg: line by line; After each line, compile and run, and verify that everything is working properly before moving forward. You should grab a good C library API reference as well (there are many online) to ensure you are using the functions correctly (sscanf, for example, is not). You might avoid explicit usage, for now, of pointers, to avoid the many associated pitfalls (you appear to be trying to use them in assigning 'text1', but you have allocated only a single char). At any rate, a good programming text will help clear things up, to be sure.
    Code:
    #include <cmath>
    #include <complex>
    bool euler_flip(bool value)
    {
        return std::pow
        (
            std::complex<float>(std::exp(1.0)), 
            std::complex<float>(0, 1) 
            * std::complex<float>(std::atan(1.0)
            *(1 << (value + 2)))
        ).real() < 0;
    }

  9. #9
    Registered User
    Join Date
    May 2009
    Posts
    27
    Can someone please explain this to me?

    http://nomads.ncdc.noaa.gov/dods/NCE..._DAILY/200901/ 20090101/narr-a_221_20090101_0000_000.ascii?tmpprs[0][3][94][327]


    I honestly do not know what to do with this. There are about 8 different parts that I need to make loops for.

    I need "200901"/ "20090101" and "_20090101_ " (each place there's a date)
    I need "tmpprs"
    and I need each of these... "[0][3][94][327]"

    So I break it all apart first and then make loops? Where do I break it? Do I break every section apart where there is something between the backslashes? Do I break it apart by something else and declare ints/chars? I'm guessing this: "http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY" is the host? I got that from a website at least. So is that assigned a constant? That's ALL I know to do with it!

    I have been to three bookstores and none of those people seem to know about programming let alone parsing (once I checked last week, once this week) and there is nothing on how to parse this kind of URL.

    I've tried using this website: Cross Reference: /onnv/onnv-gate/usr/src/common/net/wanboot/parseURL.c But I guess my link is going to apply differently. This one has parts of the URL called something but I can't even tell what it was to begin with. I don't see what part of the URL they are trying to declare!

    I seriously don't know what to do with this and I've tried different things but they are very clearly WRONG. I don't see anything in the "programming texts" about parsing. I've found "parsing errors" but nothing about "parsing a URL." Some google searches bring up pages on parsing but not all of them mention C (C is all I know and barely at that!) and when they do, I don't see how mine would work. I don't know what these parts are or if this one is like them or it's an exception to the rules. It really looks that way to me. I keep guessing but it's getting me nowhere.

    PLEASE! You can see I'm trying here but I'm getting nowhere. Could someone help me break this thing apart and tell me what it means?
    Last edited by Liz320; 05-29-2009 at 10:15 AM.
    "Do. Or do not. There is no try."
    - Yoda

  10. #10
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Why do you want to parse it? Don't you just want to print it, and/or navigate to that site?

    Edit: That is, I seem to recall another thread about using wget etc. And for that matter, unless you are ncdc.noaa.gov, I don't see why you would need to parse it (that's their job, to use that URL to get you the file/data you want).

  11. #11
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Quote Originally Posted by Liz320 View Post
    I don't see anything in the "programming texts" about parsing. I've found "parsing errors" but nothing about "parsing a URL." Some google searches bring up pages on parsing but not all of them mention C and when they do, I don't see how mine would work. I don't know what these parts are or if this one is like them or it's an exception to the rules. It really looks that way to me. I keep guessing but it's getting me nowhere.
    Probably because this is not something most people would do in C unless they already knew the language fairly well, in which case they would probably go about it very differently.

    I'm not telling you to give up, but you should consider that you are trying to do something that seems like it would be simple when in fact it is slightly beyond your abilities right now. I think you would be better off pursing some more straightforward exercises until you have a better grasp on programming, because there are some elementary issues with the task that will prevent you from learning much, because you really don't have the groundwork to tackle those issues, and you won't develop that groundwork by trying to tackle these issues. You will have to learn the basics first.

    That said, there might be a way to make this simple enough. If I were you, I would just use wget on the command-line to create the file you mention that you open in vi. I presume what you really want to do is process the data in that file. So concentrate on that. In other words, create the file manually using wget, and then write a C program to do what you want with the text in the file. This way you will not have to worry about parsing the url, sorting out the filename, etc.

    It is not clear that you understand file I/O. Do you know how to open a file in C and print it out?
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  12. #12
    Registered User
    Join Date
    May 2009
    Posts
    27
    Quote Originally Posted by MK27 View Post

    That said, there might be a way to make this simple enough. If I were you, I would just use wget on the command-line to create the file you mention that you open in vi. I presume what you really want to do is process the data in that file. So concentrate on that. In other words, create the file manually using wget, and then write a C program to do what you want with the text in the file. This way you will not have to worry about parsing the url, sorting out the filename, etc.

    It is not clear that you understand file I/O. Do you know how to open a file in C and print it out?
    I'm okay with opening a file (I/O and printf). To me, this link looks more like a directory with multiple sub-directories.

    I'm just not sure how to treat this link. It has so many components to it that I'm possibly just feeling overwhelmed by it. It may very well be as simple as a file. I wish I had the time to do a bunch of parsing exercises. This is for work. No, my boss isn't breathing over my neck about it, but I can't throw my hands up and say I give up. I'd lose too much. I'd rather tell him it's going to take some time, find an approach that works for me and then spend a few hours, days, late nights doing that and get it done. I just don't know what 'that' is.

    *edit* Do I use WGET on the beginning part of that link? On all of it? I need to change those parts out and that's the problem.


    Quote Originally Posted by tabstop View Post
    Why do you want to parse it? Don't you just want to print it, and/or navigate to that site?

    Edit: That is, I seem to recall another thread about using wget etc. And for that matter, unless you are ncdc.noaa.gov, I don't see why you would need to parse it (that's their job, to use that URL to get you the file/data you want).
    I need the data in a format so that I only pull what I need. Otherwise, we're talking massive amounts of data and I'd probably be shot by our IT department if I tried to pull everything from that site.
    Last edited by Liz320; 05-29-2009 at 11:30 AM.
    "Do. Or do not. There is no try."
    - Yoda

  13. #13
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Quote Originally Posted by Liz320 View Post
    I'm okay with opening a file (I/O and printf). To me, this link looks more like a directory with multiple sub-directories.

    I'm just not sure how to treat this link. It has so many components to it that I'm possibly just feeling overwhelmed by it. It may very well be as simple as a file. I wish I had the time to do a bunch of parsing exercises. This is for work. No, my boss isn't breathing over my neck about it, but I can't throw my hands up and say I give up. I'd lose too much. I'd rather tell him it's going to take some time, find an approach that works for me and then spend a few hours, days, late nights doing that and get it done. I just don't know what 'that' is.

    *edit* Do I use WGET on the beginning part of that link? On all of it? I need to change those parts out and that's the problem.
    Hang on. Have you not visited the link yourself? It's just a text file with about ten lines in it. (And yes, the whole thing is the link. That's kind of the point of a URL, isn't it?)

  14. #14
    Registered User
    Join Date
    May 2009
    Posts
    27
    Quote Originally Posted by tabstop View Post
    Hang on. Have you not visited the link yourself? It's just a text file with about ten lines in it. (And yes, the whole thing is the link. That's kind of the point of a URL, isn't it?)

    Yes I've visited the link. I need different days, months, years, variables, etc.
    "Do. Or do not. There is no try."
    - Yoda

  15. #15
    and the Hat of Guessing tabstop's Avatar
    Join Date
    Nov 2007
    Posts
    14,185
    Quote Originally Posted by Liz320 View Post
    Yes I've visited the link. I need different days, months, years, variables, etc.
    So, I guess that would require building a bunch of different URLs (NOTE: not parsing; building).

    I guess something like:
    Code:
    char URL[512];
    sprintf(URL, "http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY/%4d%02d/%4d%02d%02d/narr-a_221_%4d%02d%02d_0000_000.ascii?tmpprs[0][3][94][237]",
        year, month, year, month, day, year, month, day);
    and if you need to change the numbers in the brackets you make them variables too. Then you can hand that off to wget and deal with what you get back. (Those data files, those you can parse, but those text files look like they were designed to make that easy.)

Page 1 of 2 12 LastLast
Popular pages Recent additions subscribe to a feed

Similar Threads

  1. URL escape issue
    By George2 in forum C# Programming
    Replies: 2
    Last Post: 08-12-2008, 12:45 PM
  2. Interpreter.c
    By moussa in forum C Programming
    Replies: 4
    Last Post: 05-28-2008, 06:59 PM
  3. frustration on URL encode-decode
    By Lince in forum C Programming
    Replies: 22
    Last Post: 08-22-2007, 06:36 PM
  4. Replies: 1
    Last Post: 07-02-2007, 10:22 AM
  5. Url query encoding/decoding
    By Niara in forum Networking/Device Communication
    Replies: 6
    Last Post: 04-25-2007, 04:30 PM

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21