Using Wininet functions to sign in on Yahoo

Hello everyone. I am making a program that needs to download the HTML source from a Yahoo page. Well, my program is all set up with InternetOpen and InternetOpenUrl, and it does read the HTML with InternetReadFile. The only problem is that it reads a page saying that I am not logged in and cannot perform this action. So my question - is there a way to send Yahoo my username and password so that when I try to get to the protected page, it will recognize that I am logged in and I will get the page I want? Thanks in advance, Joe.

It is possible to do - it might be hard though, especially with Yahoo!. I've done this type of thing with less complicated sites.
Your basic aim is to mimic the data a normal browser with a normal user would send. Looking at the Y! login page (http://login.yahoo.com/config/login), it's a POST form, and it posts to itself, except using HTTPS. So, create a connection with "POST" as the verb (second argument to HttpOpenRequest()). Then when you execute the request with HttpSendRequest(), add in the POST data, which is basically in the form of "field1_name=field1_value&field2_name=field2_data& etc,etc". (I believe this is passed under the lpOptional argument.) I'd also pass the page that normal would submit the data as a referer, some sites check this as a basic security.

Y! uses HTTPS and I've never dealt with such a login. WinINet appears to support it, it's a different server port to pass to InternetConnect(). Once you retrieve the login page (which is a basic redirect to whatever you were trying to access), it'll probably have some cookies: Pass these to whatever you need in Yahoo to show that you have a valid login.

Edit: You'll want to look at the source of the login page to see the variables that the form sends. You'll need to understand HTML, obviously... Y! sends quite a lot of stuff back, 25 post fields by my count, all but 3 are hidden. There is one named challenge in there, so you'll probably have to first get the normal login page (that prompts for user/pass) to get these values, extract them, and them pass them on.
I have no idea what form WinINet leaves received data in, but I use a function to extract data that searches for a start text and an end text and returns what it finds between the two. Works wonderfully. It's prototype is something like

Code:

---

std::string::size_type MarkerMatch(std::string &data,
  std::string &result,
  std::string front_text,
  std::string back_text,
  std::string::size_type start_location = 0);

---

Hello again, and thanks for the reply. I have been following your post and have come to a few problems. Just to make sure I had an understanding of your suggestions and the way the Yahoo login works, I tried to log in just by typing in all those form variables in the URL in my regular browser. I successfully logged in. So, I tried to apply that same principle to my program. I started off by calling InternetOpen:

Code:

---

    if((hSession = InternetOpen("MFFS", INTERNET_OPEN_TYPE_DIRECT, NULL, NULL, 0)) == NULL){
        printf("Could not connect.\n");
        system("PAUSE");
        return 0;
    }

---

This seemed to work fine. I then called InternetConnect:

Code:

---

    if((hDownload = InternetConnect(hSession, "login.yahoo.com", INTERNET_DEFAULT_HTTPS_PORT, NULL, NULL, INTERNET_SERVICE_HTTP, 0, 0)) == NULL){
        printf("Couldn't reach site.\n");
        system("PAUSE");
        return 0;
    }

---

I used the HTTPS port as you recommended. This also seemed to work fine. After that, I called HttpOpenRequest:

Code:

---

    if((hRequest = HttpOpenRequest(hDownload, "POST", "https://login.yahoo.com/config/login?", NULL, NULL, NULL, INTERNET_FLAG_SECURE, 0)) == NULL){
        printf("Couldn't open request.\n");
        system("PAUSE");
        return 0;
    }

---

I was a little unsure about this one. I used "https://login.yahoo.com/config/login?" because that's what it said on the form. I also used INTERNET_FLAG_SECURE because I know that Yahoo uses some sort of SSL certificate. Despite my uncertainties, this section also seemed to work fine. Finally, I tried to call HttpSendRequest:

Code:

---

    if(!(HttpSendRequest(hRequest, NULL, 0, ".tries=1&.src=fpctx&.md5=%22%22&.hash=%22%22&.js=%22%22&.last=%22%22&promo=%22%22&.intl=%22us%22&.bypass=%22%22
&.partner=%22%22&.u=es8as592dk0vm&.v=0&.challenge=Cw8clMhTn1ljY2IIyNbBm3qXDP.X&.yplus=%22%22&.emailCode=%22%22
&pkg=%22%22&stepid=%22%22&.ev=%22%22&hasMsgr=0&.chkP=Y&.done=http://www.yahoo.com&.pd=fpctx_ver%3d0&login=(myusername)
&passwd=(mypassword)&.persistent=y", 401))){
        DWORD errormsg = GetLastError();
        printf("Couldn't send request.\n%i\n", errormsg);
        system("PAUSE");
        return 0;
    }

---

This didn't work. First of all, I was unsure about the headers. I'm not sure what the headers are or if I really need them or not. So, I just used NULL for the headers and 0 for their size. Then I included the options, like you recommended. I used the ones that worked successfully in my browser, and for the size, just typed 401, which was the amount of characters it contained (Assuming that 1 char = 1 byte). However, this didn't work, so I tried to use GetLastError. The code it returned was 12031. I looked this up and found:

Code:

---

ERROR_INTERNET_CONNECTION_RESET
    12031
    The connection with the server has been reset.

---

Hmm... Not quite sure what that means, or why it happened. So anyway, my questions are:
- What are the headers in HttpSendRequest, and do I need them?
- What is ERROR_INTERNET_CONNECTION_RESET trying to tell me?
- And I haven't quite gotten to this point yet, but I believe that once I log in, Yahoo sends my browser a few cookies so that other pages recognize that I am logged in. My question is, how do I get my program to retrieve these cookies and present them to the page I am trying to access so that it recognizes that I am logged on?

Thanks a lot, Joe.

I'd recommend moving the post data (the really long string) into, at least, a char *, and get the size of it by calling strlen() or the equivalent of whatever type you put it in. That'll reduce errors, let you change it in the future, and allow you to generate it dynamically if need be.

Also, you see to have things like &pkg=%22%22 - for clarification, if the HTML code has:

Code:

---

<input type="hidden" name="pkg" value="">

---

, then it has no value. The quotes merely surround the value, so the respective post data would be &pkg=&name_of_next_variable... the quotes could be throwing Y! off.

One of the variables is named "challenge". You might want to check if this perhaps changes, or if there are any cookies that perhaps the initial login page is sending. The value may also be randomized so that you have to get the initial login page, parse through it, and send the values that Y! sends you.

As for the connection reset, I believe it just means that the connection was closed. It's nonetheless odd - I would've expected at least a HTTP response that had an error enclosed.

Best of luck.

EDIT: Scratch what I said about stepid - I was logged in.

Hey Hugger, I just cleaned up my code a little like you recommended, taking out the %22s and putting everything in one string. The issue is still there though. I noticed what you wrote about the challenge variable, and I don't think that I'd have to find the value that Yahoo! gives me because when I go to the login page on my browser and see the new value in the HTML, I can type in my old post data in the URL with the old challenge variable and it logs me in fine. Yahoo does in fact send a cookie though when I first visit the login page. Only I'm still not quite sure how it works with the login or how to handle cookies in my program. I wish GetLastError would actually give me something useful! Anyway, if you could give me a little more of a nudge in the right direction, I'd really appreciate it. In the meantime, I'll just try to mess around with the parameters and see if I can get anywhere. Thanks again.

I'm still trying to figure this out, and I was wondering if anyone knew about these headers that I am supposed to be sending with HttpSendRequest. For some reason I think that there should be something there, I just can't figure out what it is... Also, does anyone know how to handle the cookies? I've been looking at examples all over the Internet, it's just that I can't relate them to my program... thanks again