bugGNU Octave - Bugs: bug #60858, [octave forge] (control) Wrong...

 
 

bug #60858: [octave forge] (control) Wrong signs in unit test for bstmodred

Submitter:  Rafael Laboissière <rlaboiss>
Submitted:  Wed 30 Jun 2021 07:33:44 AM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 14 Jan 2024 09:54:25 AM UTC, comment #10: 

These issues should be fixed with version 4.0.0 of the control package.

Torsten Lilge <ttl>
Group Member
Sun 05 Mar 2023 05:44:19 PM UTC, comment #9: 

Thanks for testing the files are updated in the stable branch of the control package.

Torsten Lilge <ttl>
Group Member
Sun 05 Mar 2023 04:33:21 PM UTC, comment #8: 

Oops. I meant the test suite for the control package of course...

Markus Mützel <mmuetzel>
Group administrator
Sun 05 Mar 2023 04:24:48 PM UTC, comment #7: 

Thank you for the quick update.

The test suite of the statistics package passes for me with the updated files:

Failure Summary:

  ..6-w64\mingw64\share\octave\packages\control-3.5.0\ltimodels.m  pass   23/33
                                                    (reported bug) XFAIL  10

Summary:

  PASS                              340
  FAIL                                0
  XFAIL (reported bug)               10


(And sorry for having highjacked this report if this is unrelated to the original issue.)

Markus Mützel <mmuetzel>
Group administrator
Sun 05 Mar 2023 02:02:14 PM UTC, comment #6: 

@Markus, all the cases you have posted affect functions that identify or approximate the input/output behavior of a system. Since the resulting state space representations are not unique (e.g. some states have different signs like in your cases). I have therefore changed the tests in order to just test the input/output behavior of the results, not the state space representation.

Could you please test again with the attached files?

(file #54442, file #54443, file #54444, file #54445)

Torsten Lilge <ttl>
Group Member
Sun 05 Mar 2023 10:31:57 AM UTC, comment #5: 

I'm seeing similarly opposite signs in some of the BISTs with control 3.5.0 and Octave 8.0.90 (hg id 436f771403bc):

Failure Summary:

  ..6-w64\mingw64\share\octave\packages\control-3.5.0\btaconred.m  pass    1/2
                                                                   FAIL    1
  ..6-w64\mingw64\share\octave\packages\control-3.5.0\btamodred.m  pass    1/2
                                                                   FAIL    1
  ..6-w64\mingw64\share\octave\packages\control-3.5.0\hnamodred.m  pass    1/2
                                                                   FAIL    1
  ..6-w64\mingw64\share\octave\packages\control-3.5.0\ltimodels.m  pass   23/33
                                                    (reported bug) XFAIL  10
  ..17-16-w64\mingw64\share\octave\packages\control-3.5.0\moen4.m  pass   14/17
                                                                   FAIL    3


I don't know if this is related to the original issue.

I'm attaching the fntests.log with the more verbose test results.


(file #54441)

Markus Mützel <mmuetzel>
Group administrator
Thu 01 Jul 2021 07:16:52 PM UTC, comment #4: 

Maybe this is related to bug #45008 where the use of OpenBLAS leads to failing tests (also the ones from SLICOT) whereas the tests pass when using BLAS.

Torsten Lilge <ttl>
Group Member
Thu 01 Jul 2021 12:41:17 PM UTC, comment #3: 

comment #1:

> Do the tests in bug #51931, which are different from this one, still fail?


No, they do not fail.

Rafael Laboissière <rlaboiss>
Thu 01 Jul 2021 12:39:40 PM UTC, comment #2: 

Please, put this bug on hold. It manifests itself on a Debian unstable system, with Octave 6.2.0. However, I could not replicate it on a Debian experimental system. I will investigate this further.

Rafael Laboissière <rlaboiss>
Wed 30 Jun 2021 06:37:05 PM UTC, comment #1: 

Thank you for the report and the patch.

The implemented test is the original one from the SLICOT documentation for the routine AB09HD [1], which is used in this case. Therefore, and since the test passes on other systems (including mine with Debian 11) I am not convinced that changing the test is the optimal measure here.

Do the tests in bug #51931, which are different from this one, still fail?

[1] http://slicot.org/objects/software/shared/doc/AB09HD.html

Torsten Lilge <ttl>
Group Member
Wed 30 Jun 2021 07:33:44 AM UTC, original submission:  

The unit test for the bstmodred function in the control package (version 3.3.0) fails with the following error message:


octave:1> pkg load control
octave:2> test bstmodred
***** assert (Mo, Me, 1e-4);
!!!!! test failed
ASSERT errors for:  assert (Mo,Me,1e-4)

  Location  |  Observed  |  Expected  |  Reason
   (3,1)        2.9889      -2.9889      Abs err 5.9778 exceeds tol 0.0001 by 6
   (6,1)       0.69073      -0.6907      Abs err 1.3814 exceeds tol 0.0001 by 1
   (7,1)      -0.067554      0.0676      Abs err 0.13515 exceeds tol 0.0001 by 0.1
   (8,1)       -0.69073      0.6907      Abs err 1.3814 exceeds tol 0.0001 by 1
   (6,2)       0.68823      -0.6882      Abs err 1.3764 exceeds tol 0.0001 by 1
   (8,2)       0.68823      -0.6882      Abs err 1.3764 exceeds tol 0.0001 by 1
   (1,3)       -6.5947       6.5947      Abs err 13.189 exceeds tol 0.0001 by 1e+01
   (5,3)        2.0339      -2.0339      Abs err 4.0678 exceeds tol 0.0001 by 4
   (6,4)      -0.095762      0.0958      Abs err 0.19156 exceeds tol 0.0001 by 0.2
   (8,4)      -0.095762      0.0958      Abs err 0.19156 exceeds tol 0.0001 by 0.2
   (3,5)       0.36919      -0.3692      Abs err 0.73839 exceeds tol 0.0001 by 0.7
   (6,5)      0.0037553     -0.0038      Abs err 0.0075553 exceeds tol 0.0001 by 0.007
   (7,5)       0.75223      -0.7522      Abs err 1.5044 exceeds tol 0.0001 by 2
   (8,5)      -0.0037553     0.0038      Abs err 0.0075553 exceeds tol 0.0001 by 0.007
   (1,6)       -0.13307      0.1331      Abs err 0.26617 exceeds tol 0.0001 by 0.3
   (2,6)       0.086198     -0.0862      Abs err 0.1724 exceeds tol 0.0001 by 0.2
   (4,6)        3.5767      -3.5767      Abs err 7.1534 exceeds tol 0.0001 by 7
   (5,6)        2.3033      -2.3033      Abs err 4.6066 exceeds tol 0.0001 by 5
   (1,7)       0.13307      -0.1331      Abs err 0.26617 exceeds tol 0.0001 by 0.3
   (2,7)       0.086198     -0.0862      Abs err 0.1724 exceeds tol 0.0001 by 0.2
   (4,7)        3.5767      -3.5767      Abs err 7.1534 exceeds tol 0.0001 by 7
   (5,7)       -2.3033       2.3033      Abs err 4.6066 exceeds tol 0.0001 by 5
shared variables   scalar structure containing the fields:

[…]


This issue has been already reported in the past. The patch attached to this bug report fixes the problem. It simply flips the signs of some expected values.

Rafael Laboissière <rlaboiss>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54442:  btaconred.m added by ttl (9KiB - text/x-objcsrc)
file #54443:  btamodred.m added by ttl (10KiB - text/x-objcsrc)
file #54444:  hnamodred.m added by ttl (14KiB - text/x-objcsrc)
file #54445:  moen4.m added by ttl (148KiB - text/x-objcsrc)
file #54441:  fntests.log added by mmuetzel (119KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Updated the item)
  • -email is unavailable- added by ttl (Posted a comment)
  • -email is unavailable- added by rlaboiss (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-01-14 ttl StatusNone Fixed
        Open/ClosedOpen Closed
    2023-03-05 ttl Attached File- Added btaconred.m, #54442
        Attached File- Added btamodred.m, #54443
        Attached File- Added hnamodred.m, #54444
        Attached File- Added moen4.m, #54445
    2023-03-05 mmuetzel Attached File- Added fntests.log, #54441
    2021-06-30 rlaboiss Attached File- Added fix-unit-test-bstmodred.patch, #51628

    Back to the top

    Powered by Savane 3.13-5884.
    Corresponding source code