Thread: Web Automation using C++.

  1. #1
    Registered User
    Join Date
    Sep 2014
    Posts
    51

    Web Automation using C++.

    Hi, I wanted to write a C++ code that visits a particular URL and then enters a pre-defined value in it and presses "Enter" so that it can navigate to the next page and then get the data present in that page by parsing that page. This is similar to something that can be done by selenium, but that's mostly testing and I want to specifically use C++ so that I can gain some fluency in C++ too. Could you suggest some libraries /Framework that can help us get this task done?

  2. #2
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Try Casablanca (aka C++ Rest).
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  3. #3
    Registered User MutantJohn's Avatar
    Join Date
    Feb 2013
    Posts
    2,665
    Oh God, is that a Microsoft product?

    *vomits*

    OP, would libcurl do what you want?

    Edit : Yay, I finally figured out how to do links!!!! Turns out, it's not [a href="..."][/a] but [url="...
    Last edited by MutantJohn; 06-28-2015 at 12:08 PM.

  4. #4
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Oh God, is that a Linux C API?

    *vomits*

    OP, would you ignore this idiot?
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  5. #5
    Registered User MutantJohn's Avatar
    Join Date
    Feb 2013
    Posts
    2,665
    Hey, they have some Windows stuff on their download page.

    >_>

    <_<

  6. #6
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Don't care, linux lover. My current library (guess what it is - it's actually Casablanca!) suits my purposes just fine.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  7. #7
    Registered User MutantJohn's Avatar
    Join Date
    Feb 2013
    Posts
    2,665
    You say Linux lover like it's a bad thing XD

    Out of curiosity, how would one go about this without using an external library?

    If you want to request the HTML content of a webpage, you obviously have to use sockets, right? How do you make a page request?

  8. #8
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    Quote Originally Posted by MutantJohn View Post
    YHow do you make a page request?
    You connect to the website using your OS APIs and follow the HTTP protocol.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  9. #9
    Unregistered User Yarin's Avatar
    Join Date
    Jul 2007
    Posts
    2,158
    Quote Originally Posted by Elysia View Post
    Oh God, is that a Linux C API?

    *vomits*
    No, it's a cross-platform library/API, you dingbat.

  10. #10
    Registered User
    Join Date
    Dec 2013
    Posts
    241
    Just out of curiousity, even with Casablanca or libcurl - how would automate the whole thing?
    this is not just about creating a simple http request, it's actually parse the HTML, set some details in some DOM element , send the request again and parse the result again.
    I just don't get how can you recomend an HTTP library for a Web automation task, which is not the same thing.
    Unfortunatly, something like Selenium does not exist (from what I know) for C++, and honestly? I don't see why doing it when you can do it in Selenium with simple JavaScript.

  11. #11
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    You go to your page, boot up your local http debugger which is probably built in to your browser, then look at the http requests sent from the page as you push different buttons and fill in different values. Then you just mimic those requests from your http library. Not so hard.

    For parsing, you can use htmlcxx. Fetch the page, then parse it using htmlcxx to get your data. It doesn't need to be complex parsing. Just get to whatever element the data is contained within and extract the relevant data.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  12. #12
    Registered User MutantJohn's Avatar
    Join Date
    Feb 2013
    Posts
    2,665
    I would say, just switch to Java and use jsoup lol.

  13. #13
    Registered User
    Join Date
    Dec 2013
    Posts
    241
    or Node.js , which is a bit more suitable.

  14. #14
    C++まいる!Cをこわせ!
    Join Date
    Oct 2007
    Location
    Inside my computer
    Posts
    24,654
    I doubt anything javascript-based would be better for anything, but regardless, the OP asked specifically for C++.
    Quote Originally Posted by Adak View Post
    io.h certainly IS included in some modern compilers. It is no longer part of the standard for C, but it is nevertheless, included in the very latest Pelles C versions.
    Quote Originally Posted by Salem View Post
    You mean it's included as a crutch to help ancient programmers limp along without them having to relearn too much.

    Outside of your DOS world, your header file is meaningless.

  15. #15
    Registered User
    Join Date
    Dec 2013
    Posts
    241
    this is very language-beagot saying.
    and I don't agree, node.js has excelent html parsing libraries where you can basically excess the DOM as it meant to be excessed (by JS).

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. C code for automation
    By Jacobite in forum C Programming
    Replies: 4
    Last Post: 01-22-2013, 06:00 AM
  2. OLE automation and C++
    By tkennedy in forum C++ Programming
    Replies: 3
    Last Post: 11-17-2006, 08:48 AM
  3. Automation Anywhere
    By danno.c in forum C# Programming
    Replies: 1
    Last Post: 08-07-2005, 10:10 AM
  4. Automation Doubt
    By saravanan_ts in forum Windows Programming
    Replies: 0
    Last Post: 09-25-2003, 01:49 AM
  5. Automation (or similar) without MFC
    By Robert602 in forum Windows Programming
    Replies: 0
    Last Post: 11-14-2002, 10:29 AM