Thread: How to recognize an image on the screen

  1. #1
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91

    How to recognize an image on the screen

    Hi guys, I would like you to point me to the direction on which api's, functions, etc should I look for in order to recognize an image on the screen and retrieve its coordinates.

    I hope that was clear enough hehe.

    thanks in advance!

  2. #2
    Registered User
    Join Date
    Sep 2001
    Posts
    4,912
    Are you talking about having a program know what's on the screen, looking for a specific image, and know what it's location is on the screen? I've never done that before, but you probably want to first figure out how to get an image of the screen, then how to process that image. If a graphics library like SDL doesn't do screen capture (and I don't think it does) it'll be dependent on your OS/environment - but the technical term is screen capture - so try searching with that. What are you trying to accomplish, however? If you're wanting to know where on the screen part of your program is, there are much easier ways to pass info between processes.

  3. #3
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Quote Originally Posted by lala123 View Post
    Hi guys, I would like you to point me to the direction on which api's, functions, etc should I look for in order to recognize an image on the screen and retrieve its coordinates.

    I hope that was clear enough hehe.

    thanks in advance!
    Well your question was vague enough to warrant such an answer. One way of accomplishing what you want is called the Kohonnen Self-organizing Map:
    Self-organizing map - Wikipedia, the free encyclopedia

    If you want more specifics, be a little clearer in your message. This will help do image-recognition.
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

  4. #4
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    Ok I will be more specific: I plan on doing an auto-clicker program. So there will be parts of the screen I need to click on. I must find where these images are and then click on their position.

    I'll check your link now jeff

  5. #5
    Registered User
    Join Date
    Sep 2001
    Posts
    4,912
    Say, this wouldn't be about cheating in a game or writing a bot, now would it?

  6. #6
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    Well I read the forum guidelines and I don't beleive my intentions break any of the rules, but if you do think so...

    It's about preventing me from getting a Carpal tunnel syndrome due to excessive clicking hhehehe

  7. #7
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    hmmm... anyone?

  8. #8
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    hmmm...WRT to API's and such this will be a platform specific project. You haven't mentioned your OS.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  9. #9
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    Os is windows

  10. #10
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    Okay, and by "image" you mean exactly what? Give us a very very specific, actual for real example here.

    Do you mean like "icon on my desktop", or maybe "picture in my web browser" or maybe "an object in a larger image", such as an enemy vehicle in Far Cry?
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  11. #11
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    Well imagine there's a little car in a flash game that has many other objects and i need to click on that car. So I would have to recognize where is the car and know it's position and then click on that position.

  12. #12
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Well then it would seem like the following would work:
    1. Offline, train the net (neural or other) to recognize the car. Then with good training data...
    2. Programmatically isolate the window with the image
    3. Based on image size, iterate through all possible positions in the window where the image might be.
    4. If found, send the click event to the proper X/Y coords.

    6. Profit!
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

  13. #13
    Registered User jeffcobb's Avatar
    Join Date
    Dec 2009
    Location
    Henderson, NV
    Posts
    875
    Now if you are using this to cheat a fast-moving game this approach will NOT be sufficient. There is a better way to beat that but I think it would lie outside the purview of this forum...
    C/C++ Environment: GNU CC/Emacs
    Make system: CMake
    Debuggers: Valgrind/GDB

  14. #14
    spurious conceit MK27's Avatar
    Join Date
    Jul 2008
    Location
    segmentation fault
    Posts
    8,300
    I think that approach will be sufficient to keep most people busy for a few weeks anyway.
    C programming resources:
    GNU C Function and Macro Index -- glibc reference manual
    The C Book -- nice online learner guide
    Current ISO draft standard
    CCAN -- new CPAN like open source library repository
    3 (different) GNU debugger tutorials: #1 -- #2 -- #3
    cpwiki -- our wiki on sourceforge

  15. #15
    the magic penguim
    Join Date
    Jul 2005
    Posts
    91
    hum.... so maybe getting a print from the screen, then try placing the image I'm looking for against every "block" of pixels to see if they match?

    Is that what you're saying?

Popular pages Recent additions subscribe to a feed

Similar Threads

  1. Problem reading tiff image files?
    By compz in forum C++ Programming
    Replies: 9
    Last Post: 10-30-2009, 04:17 AM
  2. Render text
    By Livijn in forum C++ Programming
    Replies: 6
    Last Post: 07-06-2007, 03:32 PM
  3. Feedback: Functional Specification Wording
    By Ragsdale85 in forum C++ Programming
    Replies: 0
    Last Post: 01-18-2006, 04:56 PM
  4. char copy
    By variable in forum C Programming
    Replies: 8
    Last Post: 02-06-2005, 10:18 PM
  5. i am not able to figure ot the starting point of this
    By youngashish in forum C++ Programming
    Replies: 7
    Last Post: 10-07-2004, 02:41 AM