bugGNU Octave - Bugs: bug #48365, __run-test_suite_ FAIL for...

 
 

bug #48365: __run-test_suite_ FAIL for assert.m (tolerances)

Submitted by:  Philip Nienhuis <philipnienhuis>
Submitted on:  Fri 01 Jul 2016 01:34:33 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Inaccurate Result
Status: FixedAssigned to: None
Originator Name: Philip NienhuisOpen/Closed: Closed
Release: 4.2.1Operating System: Microsoft Windows

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 03 Sep 2017 02:04:13 AM UTC, comment #22:

Marking as fixed. Closing report.

Rik <rik5>
Project Administrator
Sun 27 Aug 2017 05:50:20 PM UTC, comment #21:

Avinoam, thank you for testing. I pushed the patch here: http://hg.savannah.gnu.org/hgweb/octave/rev/2c3a69791ca2

Markus Mützel <mmuetzel>
Project Member
Sun 27 Aug 2017 05:45:13 PM UTC, comment #20:

The attached patch fixes the failure.

Avinoam Kalma <avinoam>
Project Member
Sat 26 Aug 2017 07:44:45 AM UTC, comment #19:

The attached patch increases the tolerance in the BIST from eps (y1) to 4*eps (y1) because it fails due to an upstream bug in mingw-w64.

(file #41664)

Markus Mützel <mmuetzel>
Project Member
Sun 13 Aug 2017 12:49:32 PM UTC, comment #18:

This might be related to a known bug in mingw-w64 [1]: The mantissa is off by 1 bit when calling std::pow with a variable. Results with literals are correct.
I checked with mingw-w64 5.0.2 and gcc 7.1 (patch #9269) and the bug is still present.
A proposed workaround is to use std::powl which is not affected.

For the time being it might be easier to increase the tolerance for the affected tests (this and bug #48364) or to mark them as xtest.

[1] https://sourceforge.net/p/mingw-w64/bugs/466/

Markus Mützel <mmuetzel>
Project Member
Fri 04 Aug 2017 06:07:46 PM UTC, comment #17:

I tried to code the test in plain C++ as suggested in comment #6 (see attached file). Compiled with the g++ that comes with the Windows installer for Octave 4.2.1 64bit, the deviations are the same as in the BIST test:

(file #41414)

Markus Mützel <mmuetzel>
Project Member
Sun 26 Feb 2017 04:12:56 PM UTC, comment #16:

I can confirm this test still fails on Octave 4.2.1 64bit on Windows 10.

Markus Mützel <mmuetzel>
Project Member
Wed 02 Nov 2016 09:45:40 PM UTC, comment #15:

Confirmed on 4.2.0-rc3 w64.

Avinoam Kalma <avinoam>
Project Member
Wed 21 Sep 2016 06:23:56 PM UTC, comment #14:

With a recent 4.2.0-rc1 (w64)I get this:

I'll have a try in he next days with -rc2

Philip Nienhuis <philipnienhuis>
Project Member
Wed 21 Sep 2016 03:53:30 AM UTC, comment #13:

The test code below is now working for me.

This still fails on 4.0.2 in a Windows VM, but passes in the same VM with a recent self-build MXE version 4.2.0-rc2.

Rik <rik5>
Project Administrator
Mon 11 Jul 2016 10:05:49 PM UTC, comment #12:

Yes, thank you.

This actually agrees with what I see on Debian x86_64, that the value of the FPU control register has no effect on the floating point rounding effects in these tests. I see all tests pass whether I set the rounding mode to double or long double, you see these tests fail on Windows regardless of whether the rounding mode is double or long double.

Mike Miller <mtmiller>
Project Administrator
Mon 11 Jul 2016 09:49:29 PM UTC, comment #11:

@Mike:
I made a new 64-bit Windows build (not 64-bit indexing), but forgot what to test exactly. One of the below?

As to bug #48364, immediately after starting Octave, fpucw = 0x037f.

Invoking some Java call (javaObject) does not change it.

axis.m and assert.m still FAIL with the usual tolerance errors.

BTW this is all with OpenBLAS. I have no more time to swap to ref BLAS; FYI I run Octave in Windows directly from the mxe-octave/dist/octave-<date_time> subdir (the Linux partition has been mounted in Windows with ext2fsd but I do that read-only).
Installing in Windows can be done tomorrow.

Philip Nienhuis <philipnienhuis>
Project Member
Sun 10 Jul 2016 08:19:31 PM UTC, comment #10:

Sure, hopefully Monday night.

Philip Nienhuis <philipnienhuis>
Project Member
Sun 10 Jul 2016 12:15:18 AM UTC, comment #9:

Yeah, that was my recollection as well. The fix was originally made for 32-bit GNU/Linux systems, where loading the JVM reset the FPUCW. I just pushed a fix to set the FPUCW when Octave starts up, which appears to be necessary on 32-bit Windows environments. No idea why this is different on 64-bit Windows environments.

Philip, would you mind retesting a 64-bit build from revision a5a99a830c8c or later, just to be sure this change has no effect?

Mike Miller <mtmiller>
Project Administrator
Sat 09 Jul 2016 12:58:13 PM UTC, comment #8:

I seem to remember that the extra precision was more of an issue with older, 32-bit processors. Modern processors with SSE intstructions were always using 64-bit doubles. This, anyways, was some of the background for the --enable-float-truncate option that is turned on by default now for all platforms (only affects sorting).

So, if FPUCW isn't going to fix it, then I think a direct test written in C++ to see if it is really a difference between libc implementations is in order.

Rik <rik5>
Project Administrator
Sat 09 Jul 2016 02:33:57 AM UTC, comment #7:

No I think we got our answer on bug #48364 (maybe we should just mark one of these as a duplicate of the other because we seem confident that they are the same error, and we now have split discussion between the two of them).

It looks like on 32-bit Windows, setting the FPUCW to 0x37f is necessary when Octave starts up and does fix the test failures (see bug #48418, patch attached there). But in 64-bit mode on Windows, while the FPUCW did come up initially at 0x27f, setting it to 0x37f had no effect, running "test axis" still failed with the same error.

Mike Miller <mtmiller>
Project Administrator
Sat 09 Jul 2016 01:14:38 AM UTC, comment #6:

@Mike: I was thinking of a completely stand-alone test, not an oct-file. But, I'd like to wait and see how the results of the FPUCW test turns out. Maybe it's just that we are truncating internal results and that is what is causing the accumulation of a few eps error.

Anonymous
Sat 09 Jul 2016 12:49:45 AM UTC, comment #5:

Rik: a C++ .oct file? That uses liboctave calls to construct the arrays and multiply them? Or a completely non-liboctave non-BLAS equivalant using standard library calls only?

Mike Miller <mtmiller>
Project Administrator
Tue 05 Jul 2016 04:42:19 PM UTC, comment #4:

Is it possible to write a pure C++ function to test this Octave code?

It seems possible that we are also running in to issues with BLAS for the multiplication. Looking at y1

I could imagine that the first '^' operator introduces 1 eps, the second power operator introduces 1 eps, and the multiplication of the two produces a further 1 eps for a total of 3 eps.

Rik <rik5>
Project Administrator
Tue 05 Jul 2016 04:27:17 PM UTC, comment #3:

It is possible that this issue and the failure of axis are really the same bug. Both this code, and logspace in bug #48364 make use of the power operator '^'. In the C library this would be the function pow, or powf for floats. It seems Octave has been using std::pow for years. Look in libinterp/corefcn/xpow.cc. So, is this back to being a question about glibc?

Rik <rik5>
Project Administrator
Fri 01 Jul 2016 05:07:05 PM UTC, comment #2:

I have been "moving on" for years - this fail has been there for so long, just like the one in bug #48364. IMO it is merely cosmetic.
It is just that you suggested me to file a bug report for it, or rather, for the remaining FAILs in bug #48312.

Intriguingly, IIRC Tatsuro once reported 0 FAILs for Windows builds (a year ago?).

Philip Nienhuis <philipnienhuis>
Project Member
Fri 01 Jul 2016 04:40:16 PM UTC, comment #1:

The values are all off by 3*eps(y1) instead of eps(y1) as the test is written. This is similar to bug #48364. Why are these values coming out different by 3 eps instead of eps on Windows? Is it worth investigating or do we just add tolerance and move on?

Mike Miller <mtmiller>
Project Administrator
Fri 01 Jul 2016 01:34:33 PM UTC, original submission:

Like bug #48364, assert.m has FAILed for a long time on Windows due to tolerances but works fine in daily use:

Philip Nienhuis <philipnienhuis>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #41664:  bug48365_assert_mingw_pow.patch added by mmuetzel (871B - application/octet-stream)
file #41414:  test_pow.cc added by mmuetzel (1008B - application/octet-stream)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -unavailable- added by mmuetzel (Posted a comment)
  • -unavailable- added by avinoam (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by philipnienhuis (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 14 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 03 Sep 2017 02:04:13 AM UTCrik5StatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
    Sun 27 Aug 2017 05:50:20 PM UTCmmuetzelStatusPatch Submitted=>Ready For Test
    Sat 26 Aug 2017 07:44:45 AM UTCmmuetzelAttached File-=>Added bug48365_assert_mingw_pow.patch, #41664
      StatusConfirmed=>Patch Submitted
    Fri 04 Aug 2017 06:07:46 PM UTCmmuetzelAttached File-=>Added test_pow.cc, #41414
    Sun 26 Feb 2017 04:12:56 PM UTCmmuetzelRelease4.2.0=>4.2.1
    Sun 26 Feb 2017 04:10:27 PM UTCmmuetzelDependencies-=>bugs #50402 is dependent
    Thu 17 Nov 2016 04:08:08 AM UTCmtmillerRelease4.2.0-rc3=>4.2.0
    Wed 02 Nov 2016 09:45:40 PM UTCavinoamStatusWorks For Me=>Confirmed
      Releasedev=>4.2.0-rc3
    Wed 02 Nov 2016 07:10:45 PM UTCmtmillerDependencies-=>bugs #49517 is dependent
    Wed 21 Sep 2016 03:53:30 AM UTCrik5StatusNeed Info=>Works For Me
    Sat 09 Jul 2016 12:49:45 AM UTCmtmillerStatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup1