bugGNU Octave - Bugs: bug #55100, [octave forge] (image) Unit test...

 
 

bug #55100: [octave forge] (image) Unit test in normxcorr2 fails randomly

Submitter:  Rafael Laboissière <rlaboiss>
Submitted:  Mon 26 Nov 2018 05:44:02 AM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 04 Dec 2018 06:15:25 AM UTC, comment #6: 
Avinoam Kalma <avinoam>
Group Member
Thu 29 Nov 2018 07:49:51 PM UTC, comment #5: 

OK, thanks, I will set it to 100*eps.
I'll push it to the the stable branch.

Avinoam Kalma <avinoam>
Group Member
Thu 29 Nov 2018 08:33:15 AM UTC, comment #4: 

This test seems to be about finding the POSITION of one msall image in a bigger one. And in this very test the position change is at most 42*eps = 1e-14. So the output is "noisy" by an accuracy of 1/1e14 of a single pixel width. This is still negligible for any real application. I would to suggest to just increase the test tolarance accordingly, maybe to something like 100*eps?

(Leaving the test as is, and therefore failing once in a while would probably also be fine. But now that we've done all this dicsuccion I would like to finally settle this and never hear of it again...)

Hartmut <hardy>
Thu 29 Nov 2018 04:49:22 AM UTC, comment #3: 

I thought also that factor of 4 will suffice, but using the following simple script


Nm = 0;
cnt = 0;
for i=1:10000000
a = rand (10, 10);
c = normxcorr2 (a(5:7, 6:9), a);
f = abs(c(7,9)-1)/eps;
if (f > 20)
    cnt++;
    fprintf (" i = %d cnt = %d f = %g\n", i, cnt, f);
end
if (f > Nm)
  Nm = f;
  fprintf (" i = %d Nm = %g\n", i, Nm);
end
end


shows that if I make 10,000,000 tests, the factor should be 42 (the answer to...), and in 21 cases out of 10,000,000,  a factor of 20 will not suffice.
The question is if we can allow a few random failures (21 out of 10,000,000 is realy negligible), or we should use a factor that the test will never fail.

Avinoam Kalma <avinoam>
Group Member
Wed 28 Nov 2018 08:18:24 PM UTC, comment #2: 

I could also observe this test failing on my linux system. (I have tried 100 times in a for loop, it failed once.)

But there is a lot of numeric calculation involved in normxcorr2. So it seems reasonable to me that we just increase the tolerance (from now 2*eps) to 3 (or better 4) times eps for this test on RANDOM input images.

Hartmut <hardy>
Wed 28 Nov 2018 05:24:07 AM UTC, comment #1: 

Change summary signal->image

Avinoam Kalma <avinoam>
Group Member
Mon 26 Nov 2018 05:44:02 AM UTC, original submission:  

I was hit by a failure in a non-determinstic unit test in file normxcorr2.m:


octave:1> pkg load image
octave:2> test normxcorr2 verbose
>>>>> /usr/share/octave/packages/image-2.8.1/normxcorr2.m
***** function offsets = get_max_offsets (c)
  l = find (c == max (c(:)));
  offsets = nthargout (1:ndims (c), @ind2sub, size (c), l);
***** endfunction
***** test
 row_shift = 18;
 col_shift = 20;
 a = randi (255, 30, 30);
 b = a(row_shift-10:row_shift, col_shift-7:col_shift);
 c = normxcorr2 (b, a);
 ## should return exact coordinates
 assert (get_max_offsets (c), {row_shift col_shift});

 ## Even with some small noise, should return exact coordinates
 b = imnoise (b, "gaussian");
 c = normxcorr2 (b, a);
 assert (get_max_offsets (c), {row_shift col_shift});
***** test
 a = rand (10, 10);
 c = normxcorr2 (a(5:7, 6:9), a);
 assert (c(7, 9), 1, eps*2);
!!!!! test failed
ASSERT errors for:  assert (c (7, 9),1,eps * 2)

  Location  |  Observed  |  Expected  |  Reason
     ()           1            1         Abs err 6.6613e-16 exceeds tol 4.4409e-16 by 2e-16


Even though the failure seems to happen very rarely, it would be better to avoid it, as in the patch attached to this bug report.

Rafael Laboissière <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #45527:  xtest-in-normxcorr2.patch added by rlaboiss (491B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hardy (Posted a comment)
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by avinoam
  • -email is unavailable- added by avinoam
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-04 avinoam StatusNone Fixed
        Open/ClosedOpen Closed
    2018-11-28 avinoam Operating SystemOther Any
        Summary[octave forge] (signal) Unit test in normxcorr2 fails randomly [octave forge] (image) Unit test in normxcorr2 fails randomly
        Carbon-Copy- Added hardy
        Carbon-Copy- Added carandraug
    2018-11-26 rlaboiss Attached File- Added xtest-in-normxcorr2.patch, #45527

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code