which data type in image program?

Hi!

I am on my way now to start making a simple image processing program which shall have function such as blurring, and ... well, other image manipulating functions. But I am not sure whether I shal use an array with unsigned ints to store the whole RGB value, or one array with signed ints for R, one for G and one for B. I believe that using one array for the whole RGB value will be faster, especially since if I use SDL I can write directly into the buffer and display it directly, but using one array for each layer will be much more precise. Do you think there will be visible losses in quality after a few manipulations if I choose to use the less precise type? Which of these methods do you think I should use? What does other raster graphics editors use, like Photoshop and GIMP?

I also intended to build a simple computer vision library with a few functions, maybe these are not as picky as a human.

It depends on what you want to do. If you're frequently dealing with just one of the three colours, then you might want separate arrays. Most of the time, though, I think you'd want to represent one pixel as one element or one structure. It's more convenient that way.

Note that you don't have to represent the red, green and blue channels of an image in one integer if you don't want to. You could always just use a 3D array.

Code:

Uint32 image[width][height][3];

I also intended to build a simple computer vision library with a few functions, maybe these are not as picky as a human.

Do you mean something like image recognition? It's not as easy as it sounds . . . .

Thanks for the answer. I'm not sure yet, if I still am going to use 3 int32 per pixel I will probably want to have them in separate arrays. It could become handy sometime I believe.

Originally Posted by dwks

Do you mean something like image recognition? It's not as easy as it sounds . . . .

Ok, have you had any bad experience? No, I really think image recognition would be very hard, at least I haven't got a clue about how to do it. I thought more in the style of merging two images, maybe one wants to make a panoramic image from a few images. I'm also thinking if there is any way to make a 3d-model out of a bunch of images taken at the same object from different angles, that on the other hand, might be a little bit more difficult. :P

Originally Posted by TriKri

Thanks for the answer. I'm not sure yet, if I still am going to use 3 int32 per pixel I will probably want to have them in separate arrays. It could become handy sometime I believe.

Well, maybe, but I can't really see that happening. For one thing, SDL_Surfaces have the pixel channels in one place, and I'm sure the designers of the SDL know more about the topic than either you or I.

For another, if you ever needed to go red[x] it would be just as easy to go pixel[x][RED], wouldn't it? . . . .

Ok, have you had any bad experience? No, I really think image recognition would be very hard, at least I haven't got a clue about how to do it.

Yes, pretty hard indeed. I didn't think that was what you were referring to.

I thought more in the style of merging two images, maybe one wants to make a panoramic image from a few images.

Hmm, that might be difficult too. Images taken with a camera never match exactly, so you'd have to use some algorithms to detect similarities in the images . . . which I have no idea about whatsoever.

I'm also thinking if there is any way to make a 3d-model out of a bunch of images taken at the same object from different angles, that on the other hand, might be a little bit more difficult. :P

This would be "a little bit more difficult", for sure.

There was actually something about this online somewhere, on TED I think. I'll see if I can find it.

The most common formats are 8/8/8 RGB organized with 3 bytes per pixel (unsigned char), and 8/8/8 RGB packed into 32-bit unsigned integers, with 8 bits of each integer being unused. Which format you choose depends on the application.

The 33% extra memory usage of the 32-bit representation can be important in memory-intensive applications. On the other hand, it is faster to address pixels because you simply shift the index left two bits. You also have to deal with masking in order to extract the components, whereas with the 3-byte representation you just dereference a byte.

It really depends on your application.

it could be faster also, to store values of the same pixel consecutively in memory, considering locality of reference (CPU cache).

Why not just have a POD struct for each pixel.

This is better than an array, because multidimensional arrays on the heap require either extra memmory or fancy casting. Also It is possible to pass the pixel with a single pointer instead of three to masks that apply to all three colors and possibly alpha.

Originally Posted by King Mir

Why not just have a POD struct for each pixel.

It's nice in theory, but you have no control over the compiler's padding and alignment. In most cases things work out okay, but in the general case of portable code it's a pain.

For passing single pixels around it is often convenient to pack/unpack them into a pixel structure. But as far as holding large images in memory, most code I've written or worked with just uses arrays of unsigned char or unsigned int.

Using masks is hard when you first start out. After getting through the first couple thousand lines of it, though, you start to internalize it and it's not so bad anymore. It's becoming less common for people to hammer their own image code, instead relying on libraries, but among those of us who do, we all pretty much speak the same language.

Originally Posted by brewbuck

The most common formats are 8/8/8 RGB organized with 3 bytes per pixel (unsigned char), and 8/8/8 RGB packed into 32-bit unsigned integers, with 8 bits of each integer being unused. Which format you choose depends on the application.

So you don't think I will notice any loss in quality of the picture, after I have modified it a bit with different functions, if I choose the 8/8/8 model over the 32/32/32 model? Just a question, how much slower do you think it will be if I choose to use 3*32 bits/pixel instead of 8/8/8 and 32 bits per pixel?

http://pippin.gimp.org/image_processing/chap_dir.html
floats for R/G/B/A, if you're going to be doing a lot of transformations.

With 8/8/8, every intermediate step is going to get clipped at 255 (or worse, lost altogether through modulo truncation), and any fractional part will be lost as well.

I think that float sounds like a good idea, better that unsigned int too. Thanks for the link by the way, a good article. I realized int32 may be a little hard to use, since whenever you want to do a simple multiplication with another integer > 1, you can't guarantee there won’t be an overflow.

Maybe I will make a basic image template class, so that I can create more image classes later if I would ever need to have an image of another type.

In the case of using floats, I think it would probably be better to have the R, G and B values in separate arrays but having the arrays stay next to each other in the same allocation. Or are there any advantages with having a big array, with each third R, each third G and each third B?

> Or are there any advantages with having a big array, with each third R, each third G and each third B?
If you have separate arrays, you'll need 3 cache lines because each array will be larger than your cache.
Consecutive RGB values will always end up adjacent in the cache.

Whether either of these is good or bad remains to be seen, but it's something to think about.

But optimising programs for locality of reference seems to be a good idea on the whole.

Originally Posted by Salem

If you have separate arrays, you'll need 3 cache lines because each array will be larger than your cache.
Consecutive RGB values will always end up adjacent in the cache.

Why do I need 3 cache lines? Can you please explain? I'm not that well studied in CPU caches...

You're probably right about the locality of reference. Thanks, I shall keep that in mind.

By the way, since I'm using floats, is there any library that can read a jpg image into floats, and skip the later typecasting of the values into 8/8/8-bit format? It would just be unnecessary in my case.