bugGNU Octave - Bugs: bug #45008, [octave forge] (control) unit...

 
 

bug #45008: [octave forge] (control) unit tests for ltimodels and @lit/minreal fail with OpenBLAS

Submitter:  Rafael Laboissière <rlaboiss>
Submitted:  Mon 04 May 2015 04:57:49 PM UTC
   
 
Category:  Octave Package Severity:  2 - Minor
Priority:  3 - Low Item Group:  Inaccurate Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 22 Mar 2018 03:42:58 PM UTC, comment #5: 

This can be closed as fixed.


Doug Stewart <dastew>
Tue 20 Mar 2018 03:00:15 PM UTC, comment #4: 

I have decided to mark these as known fails, and added 12 more test that are testing the same thing but they pass with the values given by the library that is used today. This way if anything changes it will be immediately obvious.
change set is:
https://sourceforge.net/p/octave/control/ci/e08d7da1b56cb8697c128280d111811f06a358f9/

Doug Stewart <dastew>
Tue 09 Jun 2015 08:44:18 PM UTC, comment #3: 

Thanks for the explanation, Lukas. I do not know whether it is feasible or not, but you could change the "test failed" message to a "known failure" message, in case the reference BLAS/LAPACK is not used.

Rafael Laboissière <rlaboiss>
Tue 09 Jun 2015 07:10:44 PM UTC, comment #2: 

Yes, this is a known "error". The examples used in these tests are from the SLICOT library. The SLICOT guys strongly recommend the use of reference BLAS/LAPACK and so do I. One could argue that these tests should be removed, but I prefer to keep them so users are aware of the fact that they are getting different results which may be wrong.


octave:5> help test_control
'test_control' is a script from the file /Users/lukas/control/inst/test_control.m

 -- Script File: test_control
     Execute all available tests at once.  The Octave control package is
     based on the SLICOT (http://www.slicot.org) library.  SLICOT needs
     BLAS and LAPACK libraries which are also prerequisites for Octave
     itself.  In case of failing tests, it is highly recommended to use
     Netlib's reference BLAS (http://www.netlib.org/blas/) and LAPACK
     (http://www.netlib.org/lapack/) for building Octave.  Using ATLAS
     may lead to sign changes in some entries of the state-space
     matrices.  In general, these sign changes are not 'wrong' and can
     be regarded as the result of state transformations.  Such state
     transformations (but not input/output transformations) have no
     influence on the input-output behaviour of the system.  For better
     numerics, the control package uses such transformations by default
     when calculating the frequency responses and a few other things.
     However, arguments like the Hankel singular Values (HSV) must not
     change.  Differing HSVs and failing algorithms are known for using
     Framework Accelerate from Mac OS X 10.7.


Lukas Reichlin <paramaniac>
Sun 31 May 2015 06:43:48 PM UTC, comment #1: 

I think this is a known "error" and Lukas has responded to this before saying that the control package tests should be run with reference BLAS and LAPACK only. If the tests are run with ATLAS or OpenBLAS there may be tolerance errors.

If I update-alternatives to use reference BLAS/LAPACK I get no errors, but with OpenBLAS I see the same as reported here.

Marking as "works for me", I'll let Lukas reply definitively whether this should be closed as working or not.

Mike Miller <mtmiller>
Group Member
Mon 04 May 2015 04:57:49 PM UTC, original submission:  

I am getting this with the control package, version 2.8.1:


octave:1> test ltimodels
!!!!! test failed
ASSERT errors for:  assert (ac,ac_e,1e-4)

  Location  |  Observed  |  Expected  |  Reason
   (1,5)       -1.4113      -1.2627      Abs err 0.14862 exceeds tol 0.0001
   (2,5)        -0.706       -0.852      Abs err 0.146 exceeds tol 0.0001
   (3,5)       0.052188     -0.2651      Abs err 0.31729 exceeds tol 0.0001
   (4,5)        0.706        0.852       Abs err 0.146 exceeds tol 0.0001
   (5,5)        1.9898       -1.554      Abs err 3.5438 exceeds tol 0.0001
   (6,5)       0.011224     -0.6533      Abs err 0.66452 exceeds tol 0.0001
   (7,5)       0.15568      -0.5892      Abs err 0.74488 exceeds tol 0.0001
   (1,6)      -0.0050932     0.4334      Abs err 0.43849 exceeds tol 0.0001
   (2,6)       -0.70821      0.2924      Abs err 1.0006 exceeds tol 0.0001
   (3,6)      -0.052025     -0.7723      Abs err 0.72028 exceeds tol 0.0001
   (4,6)       0.70821      -0.2924      Abs err 1.0006 exceeds tol 0.0001
   (5,6)      0.0071807      0.5334      Abs err 0.52622 exceeds tol 0.0001
   (6,6)      4.0506e-05     0.2242      Abs err 0.22416 exceeds tol 0.0001
   (7,6)      0.00056182     0.2022      Abs err 0.20164 exceeds tol 0.0001
   (1,7)      -0.090251      0.4666      Abs err 0.55685 exceeds tol 0.0001
   (2,7)      3.8858e-16    -0.4342      Abs err 0.4342 exceeds tol 0.0001
   (3,7)       -0.81316        0         Abs err 0.81316 exceeds tol 0.0001
   (4,7)     -1.1725e-15     0.4342      Abs err 0.4342 exceeds tol 0.0001
   (5,7)       0.12724       0.5742      Abs err 0.44696 exceeds tol 0.0001
   (6,7)      0.00071777     0.2414      Abs err 0.24068 exceeds tol 0.0001
   (7,7)      0.0099555      0.2177      Abs err 0.20774 exceeds tol 0.0001

octave:2> test @lti/minreal
!!!!! test failed
ASSERT errors for:  assert (Ar,Ae,1e-4)

  Location  |  Observed  |  Expected  |  Reason
   (1,3)       0.098016      -0.098      Abs err 0.19602 exceeds tol 0.0001
   (2,3)       -0.27169      0.2717      Abs err 0.54339 exceeds tol 0.0001
   (3,3)       -1.3887       1.3887      Abs err 2.7774 exceeds tol 0.0001
   (1,4)       0.10656      -0.1066      Abs err 0.21316 exceeds tol 0.0001
   (2,4)       -0.26093      0.2609      Abs err 0.52183 exceeds tol 0.0001
   (3,4)       -0.66991      0.6699      Abs err 1.3398 exceeds tol 0.0001
   (4,4)        1.2147      -1.2147      Abs err 2.4294 exceeds tol 0.0001
   (5,5)        1.0545      -1.0545      Abs err 2.109 exceeds tol 0.0001
   (1,6)       0.23302       -0.233      Abs err 0.46602 exceeds tol 0.0001
   (2,6)       -0.67578      0.6758      Abs err 1.3516 exceeds tol 0.0001
   (3,6)       -1.6389       1.6389      Abs err 3.2778 exceeds tol 0.0001
   (4,6)       0.97919      -0.9792      Abs err 1.9584 exceeds tol 0.0001
   (6,6)       -1.6355       1.6355      Abs err 3.271 exceeds tol 0.0001
   (1,7)      -0.077733      0.0777      Abs err 0.15543 exceeds tol 0.0001
   (2,7)       0.35529      -0.3553      Abs err 0.71059 exceeds tol 0.0001
   (3,7)       0.76153      -0.7615      Abs err 1.523 exceeds tol 0.0001
   (4,7)       -0.47884      0.4788      Abs err 0.95764 exceeds tol 0.0001
   (6,7)       0.43234      -0.4323      Abs err 0.86464 exceeds tol 0.0001


Rafael Laboissière <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dastew (Posted a comment)
  • -email is unavailable- added by jwe (Updated the item)
  • -email is unavailable- added by paramaniac (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by mtmiller
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-03-22 mtmiller StatusWorks For Me Fixed
        Open/ClosedOpen Closed
    2017-08-13 jwe Summarycontrol package: unit tests for ltimodels and @lit/minreal fail with OpenBLAS [octave forge] (control) unit tests for ltimodels and @lit/minreal fail with OpenBLAS
    2015-12-22 mtmiller Severity3 - Normal 2 - Minor
        Priority5 - Normal 3 - Low
        Summarycontrol: Unit tests for ltimodels and @lit/minreal fail control package: unit tests for ltimodels and @lit/minreal fail with OpenBLAS
    2015-05-31 mtmiller Item GroupIncorrect Result Inaccurate Result
        StatusNone Works For Me
        Carbon-Copy- Added paramaniac

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code