bugGNU Octave - Bugs: bug #40664, Inconsistent handling of...

 
 

bug #40664: Inconsistent handling of N-dimensional RGB images

Submitter:  Adam H Aitkenhead <adama>
Submitted:  Thu 21 Nov 2013 09:14:22 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Other
Status:  Fixed Assigned to:  None
Originator Name:  Adam Aitkenhead Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 24 Nov 2013 04:49:43 AM UTC, comment #7: 

If it works in Matlab then we have to but if it doesn't we can think twice about it. Just because we don't see a problem now doesn't mean we won't have one in the future. Comes to my mind now, the fact that until recently, imread was returning an RGB image with alpha channel as a MxNx4 matrix which I had to remove now since that's for a CMYK image. Hopefully there isn't many people using it.

I think this functions, to convert between a colorspace and a indexed image, only makes sense before or after reading, writing or displaying an image (I can't think of any operations that make sense in indexed images). Those functions are also unable to deal with matrices with more than 4 dimensions so the user will have to reshape it anyway after calling rgb2ind. The convention on those types of functions is

  • 2D matrix => grayscale image


  • 2D matrix with colormap => indexed image


  • 3D matrix => CMYK or RGB depending on size of dimension 3


  • 4D matrix => multiple images of any of the above type, all concatenated in the 4th dimension


Your bug report is about consistency (take a look at the title) but at the same time you're asking to change this convention for just this 2 functions. Such user may be happy now if we make this work with matrices of size > 4, but will be much more frustrated if one day the behavior changes and because there was never an error, he will not know that his code is giving incorrect results.

For now, I have pushed a cset that at least makes rgb2ind reverse ind2rgb and thus handling N-dimensional images (matrices with at least 4 dimensions)

http://hg.savannah.gnu.org/hgweb/octave/rev/b8cd17571a1d

Carnë Draug <carandraug>
Group Member
Fri 22 Nov 2013 04:51:14 PM UTC, comment #6: 

Matlab's implementation for gray2ind works for ND arrays, we can't say if that's through accident or intent.  Whether other functions work properly for ND or not would have to be checked individually.  But either way, if equivalent Octave functions can be written which do handle ND arrays correctly, then there is no downside to allowing it.  If that ability is documented and clear to anyone wishing to use it, then all is good.

Input checks which block valid (although perhaps not common) uses of a function seem like a recipe for frustration with little benefit in return.  If a user unwittingly inputs an ND array to rgb2ind or ind2rgb then the lack of an error message is unlikely to be a problem for them.  They might even think 'That's cool, Octave still meaningfully processes that unusual data.'

The use of ind2rgb and rgb2ind needn't be limited to the capabilities of other read/write image functions.  There are other more general uses - which is how I came across this issue in the first place.  (Also, the current implementation of ind2rgb blocks the use of 4D images.  It can only handle 2D or 3D RGB images.)


Adam H Aitkenhead <adama>
Fri 22 Nov 2013 01:18:46 PM UTC, comment #5: 

My experience is that Matlabs's "support" for this type of images/matrices is often an accident that arises due to the lack of proper input checking. For some cases it works fine but sometimes it just returns garbage back. For example, experiment with im2col and other block processing functions. Or im2bw which should use rgb2gray before the conversion to bw when input is rgb (but does not if it has 4 dimensions - my guess is because it must be using isrgb internally which does not work for ND). So, Matlab does not support anything, it simply forgets to do input check and sometimes it works. The problem with this approach is that when it doesn't work, you won't know.

The read and write functions will read an image into a 4D matrix only, no matter how many dimensions it has. I think it makes sense to keep this type of functions consistent with the image IO functions. Some image processing functions such as mathematical morphology or other filters make sense for matrices of any dimension, they are just moving window filters, but this is not the case.

Anyway, rgb2ind should be fixed to reverse ind2rgb so this bug is still valid. I'll open another bug for the use of gm convert.

Carnë Draug <carandraug>
Group Member
Fri 22 Nov 2013 08:55:42 AM UTC, comment #4: 

Thanks for looking at this again Carne.

I still don't agree that these functions should be blocked for N>3 dimensions.  Matlab supports ND images in greyscale.  Eg. in R2010a:


temp = rand(3,3,3,3,3,3);
[x,m] = gray2ind(temp);
size(x)
ans =
     3     3     3     3     3     3


Matlab happens not to support ND images in RGB.  I don't know why not, other than it's a little more tricky to code and the demand for it is small.

More generally, functions don't need to be limited to work for the 2D/3D case only - even for images.  After all, there is no difference between an image and a matrix in Matlab or Octave.  It's only when you display it that it becomes an image - during processing it's just a matrix of numbers.   There are valid reasons why others may want to process 3+ dimension matrices through rgb2ind or ind2rgb, and I don't think that functionality should be blocked.  It doesn't have any impact at all on the handling of single 2D images or stacks of 2D images.

As for there being no convention on which dimension the RGB channels should belong to - I agree.  Originally I had defined the RGB channels as being in the final dimension, as that seemed to be a sensible way of handling it.  The change to ind2rgb then defined the RGB channel to be in the 3rd dimension - not a problem as long as we remain consistent.  Since there is no standard convention, either definition is fine as long as it is clearly documented - the user can always use permute to arrange the dimensions accordingly.

Adam H Aitkenhead <adama>
Fri 22 Nov 2013 02:00:16 AM UTC, comment #3: 

Also, I just noticed that the dither option is implemented by:

- saving image as a tiff file
- making a system call to "gm convert"
- read back the converted file

The dither option will need to be written as an oct file (I don't think it can be vectorized).

If we want to use GraphicsMagick to do it, then there's a map method http://www.graphicsmagick.org/Magick++/Image.html#map though if I remember correctly it was limited to 8bit images. But the algorithm is pretty simple so that should not be necessary.

Carnë Draug <carandraug>
Group Member
Fri 22 Nov 2013 01:36:30 AM UTC, comment #2: 

I agree that rgb2ind is wrong and needs to be updated to be consistent with ind2rgb. However, I disagree with the changes that would make it work with ndims > 4.

I'm well familiar with images that have that many dimensions (mine are microscopy images with multiple channels, time and z-slices) but there is no definition about which dimension each belongs to. At the end, they're all 2D images. The third dimension is for color, and the 4th just stacks the images together. The 4th dimension of the matrix is not meant to represent a 3D image, it's to just group many 2D images into one matrix.

Carnë Draug <carandraug>
Group Member
Thu 21 Nov 2013 09:24:27 PM UTC, comment #1: 

(The example code seems to have gone missing below.  I think I messed up using verbatim...  Here it is again.)

This patch makes ind2rgb and rgb2ind reversible for ND arrays - see the example below for a 4D test RGB image of size 4x5x6x7, with the RGB channels in the third dimension:


test = rand(4,5,3,6,7);
[x,m] = rgb2ind(test);
rgb = ind2rgb(x,m);
isequal(test,rgb)

ans =  1

whos

Variables in the current scope:

   Attr Name        Size                     Bytes  Class
   ==== ====        ====                     =====  =====
        ans         1x1                          1  logical
        m         840x3                      20160  double
        rgb         4x5x3x6x7                20160  double
        test        4x5x3x6x7                20160  double
        x           4x5x6x7                   1680  uint16


Also, there's an old discussion of this on the mailing list at:
http://octave.1599824.n4.nabble.com/ind2rgb-problem-td4653534.html#a4659386

Adam H Aitkenhead <adama>
Thu 21 Nov 2013 09:14:22 PM UTC, original submission:  

N-dimensional RGB images are currently handled differently by ind2rgb and rgb2ind.

- In rgb2ind, the final dimension of an N-dimensional RGB image is assumed to hold the RGB values.
- In ind2rgb, the third dimension of an N-dimensional RGB image is assumed to hold the RGB values.

This arose because a change was made to ind2rgb a while back which changed from using the final dimension to the third dimension.  For consistency, rgb2ind also ought to be changed to use the same convention.  The attached patch makes this change.

Also, ind2rgb currently blocks processing of images with >3 dimensions.  Matlab only allows 2D images, but I don't think there's a need to limit Octave's functionality here.  The ability to process ND images, where N>3, is worthwhile.  The reason I've been looking into this is that I've been using these functions on CT images, which have 3 dimensions (x,y,z).  I use the functions to prepare displays showing other data (eg. radiotherapy dose) overlaid as an RGB colorwash on the CT data - hence the need for handling 3-dimensional RGB images.  In future I'd also like to be able to process 4D CT images (x,y,z,time), where the patient is imaged over a breathing cycle.  In principle, these images could be taken each time a patient is treated - potentially giving 5D images (x,y,z,time,treatment).

This patch makes ind2rgb and rgb2ind reversible for ND arrays - see the example below for a 4D test RGB image of size 4x5x6x7, with the RGB channels in the third dimension:


test = rand(4,5,3,6,7);
[x,m] = rgb2ind(test);
rgb = ind2rgb(x,m);
isequal(test,rgb)

ans =  1

whos

Variables in the current scope:

   Attr Name        Size                     Bytes  Class
   ==== ====        ====                     =====  =====
        ans         1x1                          1  logical
        m         840x3                      20160  double
        rgb         4x5x3x6x7                20160  double
        test        4x5x3x6x7                20160  double
        x           4x5x6x7                   1680  uint16
+verbatim+

(For background, an early discussion of this (from May/June 2013) is on the mailing list at the link below.)
http://octave.1599824.n4.nabble.com/ind2rgb-problem-td4653534.html

Adam H Aitkenhead <adama>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #29688:  ND_RGB_images.diff added by adama (5KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by adama (Submitted the item)
  • -email is unavailable- added by adama
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-11-24 carandraug StatusNone Fixed
        Open/ClosedOpen Closed
    2013-11-21 adama Attached File- Added ND_RGB_images.diff, #29688
        Carbon-Copy- Added adama

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code