bugGNU Octave - Bugs: bug #62817, Regression in interpn, likely...

 
 

bug #62817: Regression in interpn, likely harmless

Submitter:  Arun Giridhar <arungiridhar>
Submitted:  Wed 27 Jul 2022 12:36:07 AM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Inaccurate Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 06 Aug 2022 03:53:17 AM UTC, comment #9: 

No complaints for a week. Closing as fixed.

Arun Giridhar <arungiridhar>
Group Member
Thu 28 Jul 2022 11:28:06 PM UTC, comment #8: 

Pushed this change to set a random state and increase the tolerance. https://hg.savannah.gnu.org/hgweb/octave/rev/8f3885b4d7ae

Marking as Ready for test.

Arun Giridhar <arungiridhar>
Group Member
Wed 27 Jul 2022 05:58:45 PM UTC, comment #7: 

So! I ran this test a statistically large number of times and found it's not 100% repeatable!
Test:

for i in `seq 1 100`; do octave --eval "test interpn"; done >/tmp/tmp


Results: it fails about half the time, and within the failures about 1/3 is for the imag part and 2/3 for the real part.

$ grep PASSES tmp | sort | uniq -c
     48 PASSES 18 out of 19 tests
     52 PASSES 19 out of 19 tests

$ grep ASSERT tmp | sort | uniq -c
     15 ASSERT errors for:  assert (imag (vi_complex),vi_imag)
     33 ASSERT errors for:  assert (real (vi_complex),vi_real)


The numerical errors are all of the order of eps or less:

$ cat tmp | grep exceeds | awk '{print $(NF-5)}' | sort -n | uniq -c
     34 1.1102e-16
      2 2.2204e-16
      1 2.7756e-17
     14 5.5511e-17


As you said, it's better solved by setting the seed (or not using random numbers) than only by increasing the tolerance.

Arun Giridhar <arungiridhar>
Group Member
Wed 27 Jul 2022 04:47:03 PM UTC, comment #6: 

Thanks for trying to track this down.
It might be that they are using slightly different implementations for some of their basic runtime functions for real and for complex input.
Which of the interpolation methods is (or are) failing the test?

This could be a bug in Octave or in the compiler's runtime (if this small deviation can even be considered a bug).

Slightly increasing the tolerance to `eps ()` is probably ok in this case.
Is the deviation always exactly the same even if you run the test multiple times? I grew a bit mistrusting when a test uses random numbers without setting a seed for the pRNG...

Markus Mützel <mmuetzel>
Group administrator
Wed 27 Jul 2022 03:34:09 PM UTC, comment #5: 

I ran "objdump -p $i | grep NEEDED" on the various binaries. The only difference between Clang and GCC is that Clang links to libomp and GCC links to libgomp.

End of Clang binaries:

    50          NEEDED               libgfortran.so.5
    51          NEEDED               libquadmath.so.0
    52          NEEDED               libm.so.6
    53          NEEDED               libstdc++.so.6
    54          NEEDED               libomp.so
    55          NEEDED               libgcc_s.so.1
    56          NEEDED               libc.so.6


End of GCC binaries:

    50          NEEDED               libgfortran.so.5
    51          NEEDED               libquadmath.so.0
    52          NEEDED               libstdc++.so.6
    53          NEEDED               libm.so.6
    54          NEEDED               libgomp.so.1
    55          NEEDED               libgcc_s.so.1
    56          NEEDED               libc.so.6


Difference:

$ diff clang.lt-octave-gui gcc.lt-octave-gui
52d51
<   NEEDED               libm.so.6
54c53,54
<   NEEDED               libomp.so
---
>   NEEDED               libm.so.6
>   NEEDED               libgomp.so.1

$ diff clang.octave-gui gcc.octave-gui
(same as above)

$ diff clang.octave-cli gcc.octave-cli
(same as above with different line numbers)


Everything else is the same.

Arun Giridhar <arungiridhar>
Group Member
Wed 27 Jul 2022 03:11:29 PM UTC, comment #4: 

I don't have libc++, only libstdc++. They both point to it. Output of "ldd $i | grep "c\+\+" in the respective build directories:

Clang:
lt-octave-gui:  libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f1ea8794000)
octave-cli:     libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f8d89cc6000)
octave-gui:     libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f5762aba000)

GCC:
lt-octave-gui:  libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f910d656000)
octave-cli:     libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f2740af0000)
octave-gui:     libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f9618725000)


It's probably a slightly different order of operations giving the discrepancy of 0.25 eps.

Arun Giridhar <arungiridhar>
Group Member
Wed 27 Jul 2022 07:54:47 AM UTC, comment #3: 

Do you link with libc++ when using clang? Or do you still use libstdc++?

Markus Mützel <mmuetzel>
Group administrator
Wed 27 Jul 2022 07:44:48 AM UTC, comment #2: 

Yes, it is repeatable. It also fails with "make check". I tested it with different compilers and multiple modes, and found that it shows up when compiled by Clang 14 (Clang on my distro is 14.0.6) but does not show up with GCC 12.1. Both compilers use the same libraries for me, verified with ldd. The distro is Manjaro, so I expect Arch to have the same behavior.

Arun Giridhar <arungiridhar>
Group Member
Wed 27 Jul 2022 07:17:31 AM UTC, comment #1: 

Do you see this test fail every time you run it?
It might also help to set a seed for the random number generator in case that isn't done yet...

Out of interest: Which distribution/version is that? Do you have an idea which library specifically could be the culprit?

Markus Mützel <mmuetzel>
Group administrator
Wed 27 Jul 2022 12:36:07 AM UTC, original submission:  

Getting a new regression after a recent distro upgrade:

>> test interpn
***** test <*61907>
 yi = [0.5, 1.5]';
 xi = [2.5, 3.5];
 zi = [2.25, 4.75];
 v = rand (4, 3, 5) + 1i * rand (4, 3, 5);
 for method = {"nearest", "linear", "spline"}
   vi_complex = interpn (v, yi, xi, zi, method{1});
   vi_real = interpn (real (v), yi, xi, zi, method{1});
   vi_imag = interpn (imag (v), yi, xi, zi, method{1});
   assert (real (vi_complex), vi_real)
   assert (imag (vi_complex), vi_imag)
 endfor
!!!!! regression: https://octave.org/testfailure/?61907
ASSERT errors for:  assert (real (vi_complex),vi_real)

  Location  |  Observed  |  Expected  |  Reason
  (2,1,2)      0.46978      0.46978      Abs err 5.5511e-17 exceeds tol 0 by 6e-17
PASSES 18 out of 19 tests


It's probably nothing serious. Is it OK to increase the tolerance on that test to eps instead of 0?

Arun Giridhar <arungiridhar>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by arungiridhar (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-08-06 arungiridhar StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-07-28 arungiridhar StatusNeed Info Ready For Test
    2022-07-27 mmuetzel CategoryNone Test Suite
        Item GroupNone Inaccurate Result
        StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code