bugGNU Octave - Bugs: bug #48365, __run-test_suite_ FAIL for...

 
 

bug #48365: __run-test_suite_ FAIL for assert.m (tolerances)

Submitter:  Philip Nienhuis <philipnienhuis>
Submitted:  Fri 01 Jul 2016 01:34:33 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Inaccurate Result
Status:  Fixed Assigned to:  None
Originator Name:  Philip Nienhuis Open/Closed:  * Closed
Release:  * 4.2.1 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 03 Sep 2017 02:04:13 AM UTC, comment #22: 

Marking as fixed.  Closing report.

Rik <rik5>
Group administrator
Sun 27 Aug 2017 05:50:20 PM UTC, comment #21: 

Avinoam, thank you for testing. I pushed the patch here: http://hg.savannah.gnu.org/hgweb/octave/rev/2c3a69791ca2

Markus Mützel <mmuetzel>
Group administrator
Sun 27 Aug 2017 05:45:13 PM UTC, comment #20: 

The attached patch fixes the failure.

Avinoam Kalma <avinoam>
Group Member
Sat 26 Aug 2017 07:44:45 AM UTC, comment #19: 

The attached patch increases the tolerance in the BIST from eps (y1) to 4*eps (y1) because it fails due to an upstream bug in mingw-w64.

(file #41664)

Markus Mützel <mmuetzel>
Group administrator
Sun 13 Aug 2017 12:49:32 PM UTC, comment #18: 

This might be related to a known bug in mingw-w64 [1]: The mantissa is off by 1 bit when calling std::pow with a variable. Results with literals are correct.
I checked with mingw-w64 5.0.2 and gcc 7.1 (patch #9269) and the bug is still present.
A proposed workaround is to use std::powl which is not affected.

For the time being it might be easier to increase the tolerance for the affected tests (this and bug #48364) or to mark them as xtest.

[1] https://sourceforge.net/p/mingw-w64/bugs/466/


Markus Mützel <mmuetzel>
Group administrator
Fri 04 Aug 2017 06:07:46 PM UTC, comment #17: 

I tried to code the test in plain C++ as suggested in comment #6 (see attached file). Compiled with the g++ that comes with the Windows installer for Octave 4.2.1 64bit, the deviations are the same as in the BIST test:

x: -40, y1-y2: 1.87267e-096
x: -39, y1-y2: 1.19851e-094
x: -38, y1-y2: -1.53409e-092
x: -37, y1-y2: 0
x: -36, y1-y2: 1.25673e-088
x: -35, y1-y2: 1.60861e-086
x: -34, y1-y2: 2.05902e-084
x: -33, y1-y2: -1.31777e-082
x: -32, y1-y2: 0
x: -31, y1-y2: 2.15904e-078
x: -30, y1-y2: -4.14536e-076
x: -29, y1-y2: 1.76869e-074
x: -28, y1-y2: 3.39588e-072
x: -27, y1-y2: -4.34673e-070
x: -26, y1-y2: -1.8546e-068
x: -25, y1-y2: -1.18695e-066
x: -24, y1-y2: -1.51929e-064
x: -23, y1-y2: 0
x: -22, y1-y2: -3.73381e-060
x: -21, y1-y2: 1.59309e-058
x: -20, y1-y2: 0
x: -19, y1-y2: -2.61012e-054
x: -18, y1-y2: -3.34096e-052
x: -17, y1-y2: -2.13821e-050
x: -16, y1-y2: 0
x: -15, y1-y2: 1.75162e-046
x: -14, y1-y2: -2.24208e-044
x: -13, y1-y2: 1.43493e-042
x: -12, y1-y2: 3.67342e-040
x: -11, y1-y2: -1.17549e-038
x: -10, y1-y2: 0
x: -9, y1-y2: 0
x: -8, y1-y2: 0
x: -7, y1-y2: 1.57772e-030
x: -6, y1-y2: -2.01948e-028
x: -5, y1-y2: 1.29247e-026
x: -4, y1-y2: 0
x: -3, y1-y2: -2.11758e-022
x: -2, y1-y2: 0
x: -1, y1-y2: 0



(file #41414)

Markus Mützel <mmuetzel>
Group administrator
Sun 26 Feb 2017 04:12:56 PM UTC, comment #16: 

I can confirm this test still fails on Octave 4.2.1 64bit on Windows 10.

Markus Mützel <mmuetzel>
Group administrator
Wed 02 Nov 2016 09:45:40 PM UTC, comment #15: 

Confirmed on 4.2.0-rc3 w64.

Avinoam Kalma <avinoam>
Group Member
Wed 21 Sep 2016 06:23:56 PM UTC, comment #14: 

With a recent 4.2.0-rc1 (w64)I get this:

>> x = [-40:0];
>>  y1 = (10.^x).*(10.^x);
>>  y2 = 10.^(2*x);
>> assert (y1, y2, eps (y1))
error: ASSERT errors for:  assert (y1,y2,eps (y1))

  Location  |  Observed  |  Expected  |  Reason
    (11)        1e-060       1e-060      Abs err 4.1454e-076 exceeds tol 1.3818e-076
    (13)        1e-056       1e-056      Abs err 3.3959e-072 exceeds tol 1.132e-072
    (14)        1e-054       1e-054      Abs err 4.3467e-070 exceeds tol 1.4489e-070
    (19)        1e-044       1e-044      Abs err 3.7338e-060 exceeds tol 1.2446e-060
    (22)        1e-038       1e-038      Abs err 2.6101e-054 exceeds tol 1.3051e-054
    (23)        1e-036       1e-036      Abs err 3.341e-052 exceeds tol 1.6705e-052
    (27)        1e-028       1e-028      Abs err 2.2421e-044 exceeds tol 1.121e-044
    (29)        1e-024       1e-024      Abs err 3.6734e-040 exceeds tol 1.8367e-040


I'll have a try in he next days with -rc2

Philip Nienhuis <philipnienhuis>
Group Member
Wed 21 Sep 2016 03:53:30 AM UTC, comment #13: 

The test code below is now working for me.


 x = [-40:0];
 y1 = (10.^x).*(10.^x);
 y2 = 10.^(2*x);
 assert (y1, y2, eps (y1));


This still fails on 4.0.2 in a Windows VM, but passes in the same VM with a recent self-build MXE version 4.2.0-rc2.

Rik <rik5>
Group administrator
Mon 11 Jul 2016 10:05:49 PM UTC, comment #12: 

Yes, thank you.

This actually agrees with what I see on Debian x86_64, that the value of the FPU control register has no effect on the floating point rounding effects in these tests. I see all tests pass whether I set the rounding mode to double or long double, you see these tests fail on Windows regardless of whether the rounding mode is double or long double.

Mike Miller <mtmiller>
Group Member
Mon 11 Jul 2016 09:49:29 PM UTC, comment #11: 

@Mike:
I made a new 64-bit Windows build (not 64-bit indexing), but forgot what to test exactly. One of the below?

As to bug #48364, immediately after starting Octave, fpucw = 0x037f.

Invoking some Java call (javaObject) does not change it.

axis.m and assert.m still FAIL with the usual tolerance errors.

BTW this is all with OpenBLAS. I have no more time to swap to ref BLAS; FYI I run Octave in Windows directly from the mxe-octave/dist/octave-<date_time> subdir (the Linux partition has been mounted in Windows with ext2fsd but I do that read-only).
Installing in Windows can be done tomorrow.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 10 Jul 2016 08:19:31 PM UTC, comment #10: 

Sure, hopefully Monday night.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 10 Jul 2016 12:15:18 AM UTC, comment #9: 

Yeah, that was my recollection as well. The fix was originally made for 32-bit GNU/Linux systems, where loading the JVM reset the FPUCW. I just pushed a fix to set the FPUCW when Octave starts up, which appears to be necessary on 32-bit Windows environments. No idea why this is different on 64-bit Windows environments.

Philip, would you mind retesting a 64-bit build from revision a5a99a830c8c or later, just to be sure this change has no effect?

Mike Miller <mtmiller>
Group Member
Sat 09 Jul 2016 12:58:13 PM UTC, comment #8: 

I seem to remember that the extra precision was more of an issue with older, 32-bit processors.  Modern processors with SSE intstructions were always using 64-bit doubles.  This, anyways, was some of the background for the --enable-float-truncate option that is turned on by default now for all platforms (only affects sorting).

So, if FPUCW isn't going to fix it, then I think a direct test written in C++ to see if it is really a difference between libc implementations is in order.

Rik <rik5>
Group administrator
Sat 09 Jul 2016 02:33:57 AM UTC, comment #7: 

No I think we got our answer on bug #48364 (maybe we should just mark one of these as a duplicate of the other because we seem confident that they are the same error, and we now have split discussion between the two of them).

It looks like on 32-bit Windows, setting the FPUCW to 0x37f is necessary when Octave starts up and does fix the test failures (see bug #48418, patch attached there). But in 64-bit mode on Windows, while the FPUCW did come up initially at 0x27f, setting it to 0x37f had no effect, running "test axis" still failed with the same error.

Mike Miller <mtmiller>
Group Member
Sat 09 Jul 2016 01:14:38 AM UTC, comment #6: 

@Mike: I was thinking of a completely stand-alone test, not an oct-file.  But, I'd like to wait and see how the results of the FPUCW test turns out.  Maybe it's just that we are truncating internal results and that is what is causing the accumulation of a few eps error.

Anonymous
Sat 09 Jul 2016 12:49:45 AM UTC, comment #5: 

Rik: a C++ .oct file? That uses liboctave calls to construct the arrays and multiply them? Or a completely non-liboctave non-BLAS equivalant using standard library calls only?

Mike Miller <mtmiller>
Group Member
Tue 05 Jul 2016 04:42:19 PM UTC, comment #4: 

Is it possible to write a pure C++ function to test this Octave code?


x = [-40:0];
 y1 = (10.^x).*(10.^x);
 y2 = 10.^(2*x);
 assert (y1, y2, eps (y1));


It seems possible that we are also running in to issues with BLAS for the multiplication.  Looking at y1


 y1 = (10.^x).*(10.^x);


I could imagine that the first '^' operator introduces 1 eps, the second power operator introduces 1 eps, and the multiplication of the two produces a further 1 eps for a total of 3 eps.

Rik <rik5>
Group administrator
Tue 05 Jul 2016 04:27:17 PM UTC, comment #3: 

It is possible that this issue and the failure of axis are really the same bug.  Both this code, and logspace in bug #48364 make use of the power operator '^'.  In the C library this would be the function pow, or powf for floats.  It seems Octave has been using std::pow for years.  Look in libinterp/corefcn/xpow.cc.  So, is this back to being a question about glibc?

Rik <rik5>
Group administrator
Fri 01 Jul 2016 05:07:05 PM UTC, comment #2: 

I have been "moving on" for years - this fail has been there for so long, just like the one in bug #48364. IMO it is merely cosmetic.
It is just that you suggested me to file a bug report for it, or rather, for the remaining FAILs in bug #48312.

Intriguingly, IIRC Tatsuro once reported 0 FAILs for Windows builds (a year ago?).

Philip Nienhuis <philipnienhuis>
Group Member
Fri 01 Jul 2016 04:40:16 PM UTC, comment #1: 

The values are all off by 3*eps(y1) instead of eps(y1) as the test is written. This is similar to bug #48364. Why are these values coming out different by 3 eps instead of eps on Windows? Is it worth investigating or do we just add tolerance and move on?

Mike Miller <mtmiller>
Group Member
Fri 01 Jul 2016 01:34:33 PM UTC, original submission:  

Like bug #48364, assert.m has FAILed for a long time on Windows due to tolerances but works fine in daily use:

>>>>> processing L:\philip\devel\octdev\mxe\mxe_64b_20160623\dist\octave-2016-07-01-12-48\share\octave\4.1.0+\m\testfun\assert.m
***** test
 x = [-40:0];
 y1 = (10.^x).*(10.^x);
 y2 = 10.^(2*x);
 assert (y1, y2, eps (y1));
 fail ("assert (y1, y2 + eps*1e-70, eps (y1))");
!!!!! test failed
ASSERT errors for:  assert (y1,y2,eps (y1))

  Location  |  Observed  |  Expected  |  Reason
    (11)        1e-060       1e-060      Abs err 4.1454e-076 exceeds tol 1.3818e-076
    (13)        1e-056       1e-056      Abs err 3.3959e-072 exceeds tol 1.132e-072
    (14)        1e-054       1e-054      Abs err 4.3467e-070 exceeds tol 1.4489e-070
    (19)        1e-044       1e-044      Abs err 3.7338e-060 exceeds tol 1.2446e-060
    (22)        1e-038       1e-038      Abs err 2.6101e-054 exceeds tol 1.3051e-054
    (23)        1e-036       1e-036      Abs err 3.341e-052 exceeds tol 1.6705e-052
    (27)        1e-028       1e-028      Abs err 2.2421e-044 exceeds tol 1.121e-044
    (29)        1e-024       1e-024      Abs err 3.6734e-040 exceeds tol 1.8367e-040


Philip Nienhuis <philipnienhuis>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #41664:  bug48365_assert_mingw_pow.patch added by mmuetzel (871B - application/octet-stream)
file #41414:  test_pow.cc added by mmuetzel (1008B - application/octet-stream)

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by philipnienhuis (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 14 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-09-03 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2017-08-27 mmuetzel StatusPatch Submitted Ready For Test
    2017-08-26 mmuetzel Attached File- Added bug48365_assert_mingw_pow.patch, #41664
        StatusConfirmed Patch Submitted
    2017-08-04 mmuetzel Attached File- Added test_pow.cc, #41414
    2017-02-26 mmuetzel Release4.2.0 4.2.1
    2017-02-26 mmuetzel Dependencies- bugs #50402 is dependent
    2016-11-17 mtmiller Release4.2.0-rc3 4.2.0
    2016-11-02 avinoam StatusWorks For Me Confirmed
        Releasedev 4.2.0-rc3
    2016-11-02 mtmiller Dependencies- bugs #49517 is dependent
    2016-09-21 rik5 StatusNeed Info Works For Me
    2016-07-09 mtmiller StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code