bugGNU Octave - Bugs: bug #49166, "system.tst" fails under...

 
 

bug #49166: "system.tst" fails under Windows

Submitted by:  Hartmut <hardy>
Submitted on:  Fri 23 Sep 2016 11:47:59 AM UTC  
 
Category: Test SuiteSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Inaccurate Result
Status: FixedAssigned to: None
Originator Name: Open/Closed: Closed
Release: 4.2.0-rc2Operating System: Microsoft Windows

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 28 Sep 2016 03:59:01 PM UTC, comment #6:

It's my opinion as well that we don't want to do open heart surgery on Octave to find all these instances. It's a shame there isn't some compiler option to just guarantee conformance to the IEEE specification for doubles.

The failures reported in this bug report are all

To be safe, I added a margin of 2 eps (t) in this cset http://hg.savannah.gnu.org/hgweb/octave/rev/29dab5b8332a.

Rik <rik5>
Project Administrator
Wed 28 Sep 2016 03:00:12 PM UTC, comment #5:

That's a funny test.

I don't know what we should do about extended precision. We probably don't want to declare every floating point variable with volatile. Or track down every place where it might matter. Is there any place where these results could vary by more than eps or 2*eps (relative to the value of the variables)?

John W. Eaton <jwe>
Project Administrator
Wed 28 Sep 2016 05:17:31 AM UTC, comment #4:

Well, it looks like I know what is happening; But, I don't like the results. I re-compiled a version of Octave via MXE where I truncated the intermediate results in Fcputime in data.cc.

I changed

to

The script which used to fail on the first or second iteration now passes 200 iterations without problem.

I'm adding jwe to the CC list for the bug. This is a fairly large architectural question. Do we want to find all places in the code where there is a potential for extra precision to be maintained and use OCTAVE_FLOAT_TRUNCATE? Or do we want to just add a tolerance to the BIST tests whenever we encounter these issues? I'm tempted to mark this as Postponed since I don't think the answer will be clear before the 4.2.0 release is made.

Rik <rik5>
Project Administrator
Tue 27 Sep 2016 11:06:02 PM UTC, comment #3:

This is repeatable on a Windows XP virtual machine that I keep around for testing. There are some weird correlations. If I remove the 'cd ...' line from the sample script in comment #2--which means Octave is running in the same directory as the script itself--then the failure does not happen so quickly. If I cd to a directory that is not quite so far away from the test script then the script will fail, but at perhaps the 140th iteration. If I keep the original 'cd ...' line then the script fails on the first or second iteration.

This may be due to extended precision, or lack thereof, when running on Windows platforms. If you look at the C++ code for Fcputime in data.cc it is quite simple.

From this, you can see that the m-file code

really should pass.

In the first case, it is C++ double variables and the C++ '+' operator.

In the second case, it is Octave double variables and the Octave '+' operator.

After a failure, using format long, I can see that the variables look correct.

And if I do some arbitrary arithmetic, I can get the assert statement to pass.

I'm going to try declaring the double variables in data.cc Fcputime using the OCTAVE_FLOAT_TRUNCATE option which should truncate intermediate results and see if that makes a difference. That test will take several hours as I have to rebuild a Windows installer with MXE.

Rik <rik5>
Project Administrator
Sat 24 Sep 2016 08:37:56 PM UTC, comment #2:

Here is an easier way to maybe reproduce this behavior.

The following script basically does "test system.tst" in a loop. It NEVER makes it to the end on my Win-7 machine with Octave-4.2.0-rc2. But this very same script ALWAYS finishes just fine on the same PC, if I use the Octave-4.0.3 release instead.

I have observed the following discrepancies on my PC so far (observed time, absolute error):

  • 13.1105 , 1.7764e-15
  • 25.4753 , 3.5527e-15
  • (some others as well)
Hartmut <hardy>
Fri 23 Sep 2016 04:14:10 PM UTC, comment #1:

To test just system.tst you need to copy the file in to a directory that is in your PATH variable. Use the File Explorer in Windows to locate system.tst which should be at

unless you chose a different installation directory than the default.

Copy that file somewhere convenient, like your present working directory in Octave.

Then you can run

Rik <rik5>
Project Administrator
Fri 23 Sep 2016 11:47:59 AM UTC, original submission:

When I do "__run_test_suite__" within Octave, installed via the official Windows installer from the alpha.gnu.org website (version 4.2.0-rc2) on a Win7 machine, then I get 1 FAIL-ing test result from "system.tst".

Here is the corresponding text from fntests.log:

Additional notes:

  • When I try "test system.tst" this does not seem to call the test at all. How do I call this single test?
  • When I paste only the relevant test code

into an m-file, then this runs totally fine. No errors in this case.

  • I have not tried to run the full test suite for a second time, because it takes so horrendously long.
Hartmut <hardy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by rik5
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by hardy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 28 Sep 2016 03:59:01 PM UTCrik5StatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Wed 28 Sep 2016 05:17:31 AM UTCrik5Carbon-Copy-=>Added jwe
    Tue 27 Sep 2016 11:06:02 PM UTCrik5StatusNone=>Confirmed

    Back to the top


    Powered by Savane 3.1-cleanup1