Comparing pixels

Hi!

As a small hobby project of mine, i would like to create a small library for image recognition (not photos, but graphics).
However, I'm not certain on how to do this. I suppose I should either get pixel per pixel by using some api like imlib_image_query_pixel() (I'm doing this under linux) or create a screenshot of a picture and turn in to an 8bit bitmap and compare pixels by there hue, saturation or alpha.

I've googled, but have yet not found any references about this. Would someone be nice to kick me in the right direction?