bugGNU Octave - Bugs: bug #65296, Test fails because of variables...

 
 

bug #65296: Test fails because of variables shared and modified by multiple tests

Submitter:  Fernando <tutissanalio>
Submitted:  Mon 12 Feb 2024 07:06:40 PM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 9.0.90 Operating System:  * Any
Fixed Release:  9.1.0 Planned Release:  9.1.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 13 Mar 2024 05:08:55 AM UTC, comment #11: 

I made a few more adjustments to the wording and checked in your change here https://hg.savannah.gnu.org/hgweb/octave/rev/0f9fe0289e9b.

Marking as Fixed and closing report.

Rik <rik5>
Group administrator
Sat 09 Mar 2024 10:09:43 AM UTC, comment #10: 

Maybe this alternative patch is better. I think it's clearer and it recommends not modifying shared variables.

(file #55812)

Fernando <tutissanalio>
Sat 02 Mar 2024 05:46:31 PM UTC, comment #9: 

Here is a candidate patch.

(file #55768)

Fernando <tutissanalio>
Mon 19 Feb 2024 11:01:47 PM UTC, comment #8: 

I'd say go ahead and work up a changeset to add extra warning to testfun.txi.  Because it is a documentation change, it can go right on to the stable branch in time for the 9.1 release.

Rik <rik5>
Group administrator
Thu 15 Feb 2024 06:40:18 PM UTC, comment #7: 


> Perhaps Markus' comment should be added to doc/interpreter/testfun.txi


I agree.

Fernando <tutissanalio>
Thu 15 Feb 2024 09:06:21 AM UTC, comment #6: 

Perhaps Markus' comment should be added to
doc/interpreter/testfun.txi

Modifications to shared variables might notpersist if tests fail.
This can cause hard to discover failing tests. Avoid that issue
by not modifying any of the shared variables in the BISTs.


Thomas Dean <tomdean>
Wed 14 Feb 2024 02:24:03 PM UTC, comment #5: 

Thank you for testing.

I pushed the change to the stable branch:
https://hg.savannah.gnu.org/hgweb/octave/rev/4239095e41c1

Markus Mützel <mmuetzel>
Group administrator
Wed 14 Feb 2024 02:01:53 PM UTC, comment #4: 

Yes, it does! Thanks.

Fernando <tutissanalio>
Wed 14 Feb 2024 01:47:45 PM UTC, comment #3: 

The attached patch should be avoiding this potential issue.
I'd like to push it to the stable branch.

Does it avoid this issue for you?

(file #55695)

Markus Mützel <mmuetzel>
Group administrator
Tue 13 Feb 2024 08:39:07 AM UTC, comment #2: 

Thomas Dean analyzed the issue further on the mailing list:

> In gsvd.cc:
> I inserted assert (size (A), [3, 5]) after line 333 and at the start of
> the next test at line 340.
>
> The shape of A is 3x5 at line 333.
> https://hg.savannah.gnu.org/hgweb/octave/file/19973af3ac42/libinterp/corefcn/gsvd.cc#l334
> This is after the call to gsvd (A, B).
> The display of shared variables in this test shows A to have shape 5x3.
>
> The shape of A is 5x3 at line 340.
> https://hg.savannah.gnu.org/hgweb/octave/file/19973af3ac42/libinterp/corefcn/gsvd.cc#l340



Imho, the tests shouldn't depend on each other like they currently do. Potential errors in previous tests shouldn't have these (unexpected) side effects on subsequent tests.

Markus Mützel <mmuetzel>
Group administrator
Mon 12 Feb 2024 07:14:07 PM UTC, comment #1: 

This is probably is due to openblas problems.
What CPU do you have (cat /proc/cpuinfo)?

Try to repeat the test with reference (netlib) blas.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 12 Feb 2024 07:06:40 PM UTC, original submission:  

I downladed octave-9.0.90.tar.lz from alpha.gnu.org and compiled it on a Linux Mint 21.3 (based on ubuntu 22.04). I am seeing a failed test for gsvd when running make check.


libinterp/corefcn/gsvd.cc-tst .................................. pass   37/46   [ 0.981s /  0.540s]
                                                                   FAIL    1
                                                    (reported bug) XFAIL   8


I looked at the log file fntests.log and I saw it was complaining of incorrect dimensions of the matrices:


***** test
 s = gsvd (single (eye (5)), B);
 assert (class (s), "single");
 [U,V,X,C,S] = gsvd (single (eye(5)), B);
 assert (class (U), "single");
 assert (class (V), "single");
 assert (class (X), "single");
 assert (class (C), "single");
 assert (class (S), "single");

 s = gsvd (A, single (eye (5)));
 assert (class (s), "single");
 [U,V,X,C,S] = gsvd (A, single (eye (5)));
 assert (class (U), "single");
 assert (class (V), "single");
 assert (class (X), "single");
 assert (class (C), "single");
 assert (class (S), "single");
!!!!! test failed
gsvd: A and B must have the same number of columns


The problem is that A and B are shared with previous tests and are modified in some of those tests. They even change dimensions. If a previous test fails (with known error), the modifications on A and B of the test seem to be discarded, and that triggers the error, because later tests rely on A, B having been modified.

Fernando <tutissanalio>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #55812:  bug65296-doc-2.patch added by tutissanalio (978B - text/x-patch)
file #55768:  bug65296-doc.patch added by tutissanalio (844B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by tomdean (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by tutissanalio (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-03-13 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2024-03-09 tutissanalio Attached File- Added bug65296-doc-2.patch, #55812
    2024-03-02 tutissanalio Attached File- Added bug65296-doc.patch, #55768
    2024-02-14 mmuetzel StatusConfirmed Ready For Test
        Fixed ReleaseNone 9.1.0
    2024-02-14 mmuetzel Attached File- Added bug65296-gsvd-tests-shared-variables.patch, #55695
    2024-02-13 mmuetzel StatusNone Confirmed
        Planned ReleaseNone 9.1.0
        Carbon-Copy- Added tomdean

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code