bugGNU Octave - Bugs: bug #51059, statistics: hist3 function doesn't...

 
 

bug #51059: statistics: hist3 function doesn't handle NaN inputs properly

Submitter:  None
Submitted:  Thu 18 May 2017 02:11:47 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Alastair Harrison Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 4.2.1
Operating System:  * Any Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Fri 19 May 2017 11:16:26 AM UTC, comment #5: 


> Actually there's still an issue if after nan-removal the matrix X becomes empty. I think there needs to be another empty check just after line 97.


Looking at this better, there's actually a bunch of other special cases. The NaN needs to be removed after computing the edges only.  Also we need to center the values in the histogram if all values are the same

http://hg.code.sf.net/p/octave/statistics/rev/498e4ee08706

Carnë Draug <carandraug>
Group Member
Thu 18 May 2017 03:33:42 PM UTC, comment #4: 

Actually there's still an issue if after nan-removal the matrix X becomes empty. I think there needs to be another empty check just after line 97.

Anonymous
Thu 18 May 2017 03:30:26 PM UTC, comment #3: 

Thank you for the fix. I have pushed your proposed fix and test http://hg.code.sf.net/p/octave/statistics/rev/9cab018f36a5

Carnë Draug <carandraug>
Group Member
Thu 18 May 2017 02:27:18 PM UTC, comment #2: 

So sorry, it seems my markup was malformed. One last try:

In statistics 1.3.0, hist3 doesn't appear to do the right thing for inputs containing NaN values. The code appears to try to ignore input rows containing NaN values, but instead it ends up turning the input matrix into a column vector with NaNs removed.

Reproduce like this:

>> X = [1 1; nan 2; 3 1; 3 3; 1 nan; 3 1]
X =

     1     1
   NaN     2
     3     1
     3     3
     1   NaN
     3     1

>> result = hist3(X)
error: inits(2): out of bound 1
error: called from
    hist3>edges_from_nbins at line 190 column 11
    hist3 at line 121 column 24


I think the problem is on line 97:

94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2)) = [];


Should be:

94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2), :) = [];


Here's a unit test that exercises the NaN-suppression code:

%!test
%! D = [1 1; nan 2; 3 1; 3 3; 1 nan; 3 1];
%! [c, nn] = hist3 (D, {0:4, 0:4});
%! exp_c = zeros (5);
%! exp_c([7 9 19]) = [1 2 1];
%! assert (c, exp_c);
%! assert (nn, {0:4, 0:4});


Anonymous
Thu 18 May 2017 02:24:50 PM UTC, comment #1: 

Sorry, my original submission seems to have been truncated. Hopefully the remainder won't be.

I think the problem is on line 97:

94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2)) = [];
-verbatim+

Should be:
+verbatim+
94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2), :) = [];
-verbatim+

Here's a unit test that exercises the NaN-suppression code:
+verbatim+
%!test
%! D = [1 1; nan 2; 3 1; 3 3; 1 nan; 3 1];
%! [c, nn] = hist3 (D, {0:4, 0:4});
%! exp_c = zeros (5);
%! exp_c([7 9 19]) = [1 2 1];
%! assert (c, exp_c);
%! assert (nn, {0:4, 0:4});


Anonymous
Thu 18 May 2017 02:11:47 PM UTC, original submission:  

In statistics 1.3.0, hist3 doesn't appear to do the right thing for inputs containing NaN values. The code appears to try to ignore input rows containing NaN values, but instead it ends up turning the input matrix into a column vector with NaNs removed.

Reproduce like this:

>> X = [1 1; nan 2; 3 1; 3 3; 1 nan; 3 1]
X =

     1     1
   NaN     2
     3     1
     3     3
     1   NaN
     3     1

>> result = hist3(X)
error: inits(2): out of bound 1
error: called from
    hist3>edges_from_nbins at line 190 column 11
    hist3 at line 121 column 24


I think the problem is on line 97:

94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2)) = [];
-verbatim+

Should be:
+verbatim+
94:  if (! ismatrix (X) || columns (X) != 2)
95:    error ("hist3: X must be a 2 columns matrix");
96:  endif
97:  X(any (isnan (X), 2), :) = [];
-verbatim+

Here's a unit test that exercises the NaN-suppression code:
+verbatim+
%!test
%! D = [1 1; nan 2; 3 1; 3 3; 1 nan; 3 1];
%! [c, nn] = hist3 (D, {0:4, 0:4});
%! exp_c = zeros (5);
%! exp_c([7 9 19]) = [1 2 1];
%! assert (c, exp_c);
%! assert (nn, {0:4, 0:4});


Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-05-18 carandraug Summaryhist3 function doesn't handle NaN inputs properly statistics: hist3 function doesn't handle NaN inputs properly
    2017-05-18 carandraug StatusNone Fixed
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code