bugGNU Octave - Bugs: bug #64324, Changes to Octave core causing...

 
 

bug #64324: Changes to Octave core causing BIST failures in Statistics pkg

Submitter:  Arun Giridhar <arungiridhar>
Submitted:  Tue 20 Jun 2023 03:58:48 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Regression
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 15 Apr 2024 01:06:37 AM UTC, comment #13: 

yes they were all addressed on the statistics package github repository with the 1.6.5 release.  closing report.

Nicholas Jankowski <nrjank>
Group Member
Sun 14 Apr 2024 10:07:02 PM UTC, comment #12: 

Those are all fixed in the current statistics package:

  PASS                            10342
  FAIL                                0
  XFAIL (reported bug)                2
  XFAIL (expected failure)            1


GNU Octave Version: 9.1.1 (hg id: 7bad34ee4021)
statistics-1.6.5

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 14 Apr 2024 09:39:13 PM UTC, comment #11: 

Last reported action on this was in July, 2023.  Has this been fixed with the 9.1 release?  Can report be closed?

Rik <rik5>
Group administrator
Mon 10 Jul 2023 12:13:23 AM UTC, comment #10: 

cc statistics package maintainer.  a few new items have crept up after fixing a subsequent corr issue in bug #64395.

Nicholas Jankowski <nrjank>
Group Member
Mon 10 Jul 2023 12:12:43 AM UTC, comment #9: 

Since hgid 37e184a83cf4 I got few additional failures

Failure Summary:

  ..re/octave/api-v58/packages/statistics-1.6.0/dist_fit/gevfit.m  pass    4/6
                                                                   FAIL    2
  ..share/octave/api-v58/packages/statistics-1.6.0/shadow9/mean.m  pass   79/80
                                                    (reported bug) XFAIL   1
  ..e/octave/api-v58/packages/statistics-1.6.0/correlation_test.m  pass   18/19
                                                                   FAIL    1
  ../share/octave/api-v58/packages/statistics-1.6.0/fillmissing.m  pass  377/378
                                                    (reported bug) XFAIL   1
  ..a/.local/share/octave/api-v58/packages/statistics-1.6.0/pca.m  pass   31/32
                                                (expected failure) XFAIL   1
  ../share/octave/api-v58/packages/statistics-1.6.0/stepwisefit.m  pass    0/1
                                                                   FAIL    1


Also noticed that running "pkg test statistics" twice causes octave to hang at dendrogram test (on the second run).

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Wed 21 Jun 2023 06:57:08 PM UTC, comment #8: 

Bug #64305 and bug #64318 have been fixed.  I can now run the test code in comment #7 without problem and "test violin" also passes.  I think the problems with inputParser have been fixed.

I'll leave the bug report open for the other issuses with the changes to cov.m.


Rik <rik5>
Group administrator
Wed 21 Jun 2023 06:09:17 PM UTC, comment #7: 

After installing the statistics package, I can confirm that the following minimal example produces an error in inputParser.


data = exprnd (0.1, 500,4);
violin (data, "color", jet(4))


I am investigating.

Rik <rik5>
Group administrator
Wed 21 Jun 2023 08:49:05 AM UTC, comment #6: 

@username doesn't work to add someone to the CC list of a bug report on savannah. Instead, they need to be added to the field in the "Mail Notification Carbon-Copy List" section below.

CC'ing Rik now.

Markus Mützel <mmuetzel>
Group administrator
Tue 20 Jun 2023 08:24:28 PM UTC, comment #5: 

Retitled the bug as a core regression.

Arun Giridhar <arungiridhar>
Group Member
Tue 20 Jun 2023 07:01:51 PM UTC, comment #4: 

There are seem to be 2 regressions.
The first on is 31918:e67b7b85670b

changeset:   31918:e67b7b85670b
user:        Nicholas R. Jankowski <jankowski.nicholas@gmail.com>
date:        Wed Mar 22 13:33:46 2023 -0400
summary:     cov.m: Overhaul function for matlab compatibility (bug #50583).

changeset:   31917:655e757c7522
user:        Rik <rik@octave.org>
date:        Wed Mar 22 12:00:05 2023 -0700
summary:     isuniform.m: New function.
-verbatim
The good:
+verbatim+
Summary:

  PASS                             6648
  FAIL                                0
  XFAIL (reported bug)                2
  XFAIL (expected failure)            1


octave:3> version -hgid
ans = 655e757c7522


The bad (adds 3 failures: 2 in gevfit.m and one in orrelation_test.m)

Failure Summary:

  ..re/octave/api-v58/packages/statistics-1.6.0/dist_fit/gevfit.m  pass    4/6
                                                                   FAIL    2
  ..share/octave/api-v58/packages/statistics-1.6.0/shadow9/mean.m  pass   79/80
                                                    (reported bug) XFAIL   1
  ..e/octave/api-v58/packages/statistics-1.6.0/correlation_test.m  pass   18/19
                                                                   FAIL    1
  ../share/octave/api-v58/packages/statistics-1.6.0/fillmissing.m  pass  377/378
                                                    (reported bug) XFAIL   1
  ..a/.local/share/octave/api-v58/packages/statistics-1.6.0/pca.m  pass   31/32
                                                (expected failure) XFAIL   1

Summary:

  PASS                             6645
  FAIL                                3
  XFAIL (reported bug)                2
  XFAIL (expected failure)            1

<...deleted...>

octave:4> version -hgid
ans = e67b7b85670b


The second one is 32047:e3de59065cf1
(Adds 5 failures to violin.m)

The good:

changeset:   32047:e3de59065cf1
user:        Rik <rik@octave.org>
date:        Fri Apr 21 11:01:40 2023 -0700
summary:     inputParser.m: Re-architect internal data structures for 60% speed improvement.

changeset:   32046:39700c1ea93e
parent:      32044:31f7f5359ba2
parent:      32045:f18da620ab4d
user:        Rik <rik@octave.org>
date:        Thu Apr 20 18:59:26 2023 -0700
summary:     maint: merge stable to default



Failure Summary:

  ..re/octave/api-v58/packages/statistics-1.6.0/dist_fit/gevfit.m  pass    4/6
                                                                   FAIL    2
  ..share/octave/api-v58/packages/statistics-1.6.0/shadow9/mean.m  pass   79/80
                                                    (reported bug) XFAIL   1
  ..e/octave/api-v58/packages/statistics-1.6.0/correlation_test.m  pass   18/19
                                                                   FAIL    1
  ../share/octave/api-v58/packages/statistics-1.6.0/fillmissing.m  pass  377/378
                                                    (reported bug) XFAIL   1
  ..a/.local/share/octave/api-v58/packages/statistics-1.6.0/pca.m  pass   31/32
                                                (expected failure) XFAIL   1

Summary:

  PASS                             6645
  FAIL                                3
  XFAIL (reported bug)                2
  XFAIL (expected failure)            1

<...deleted...>

octave:3> version -hgid
ans = 39700c1ea93e


The bad (adds 5 failures in violin.m):

Failure Summary:

  ..re/octave/api-v58/packages/statistics-1.6.0/dist_fit/gevfit.m  pass    4/6
                                                                   FAIL    2
  ..share/octave/api-v58/packages/statistics-1.6.0/shadow9/mean.m  pass   79/80
                                                    (reported bug) XFAIL   1
  ..e/octave/api-v58/packages/statistics-1.6.0/correlation_test.m  pass   18/19
                                                                   FAIL    1
  ../share/octave/api-v58/packages/statistics-1.6.0/fillmissing.m  pass  377/378
                                                    (reported bug) XFAIL   1
  ..a/.local/share/octave/api-v58/packages/statistics-1.6.0/pca.m  pass   31/32
                                                (expected failure) XFAIL   1
  ..local/share/octave/api-v58/packages/statistics-1.6.0/violin.m  pass    0/5
                                                                   FAIL    5

Summary:

  PASS                             6640
  FAIL                                8
  XFAIL (reported bug)                2
  XFAIL (expected failure)            1

<...deleted...>

octave:3> version -hgid
ans = e3de59065cf1


E.g.:


ctave:9> test violin verbose
>>>>> /home/dima/.local/share/octave/api-v58/packages/statistics-1.6.0/violin.m
***** demo
 clf
 x = zeros (9e2, 10);
 for i=1:10
   x(:,i) = (0.1 * randn (3e2, 3) * (randn (3,1) + 1) + 2 * randn (1,3))(:);
 endfor
 h = violin (x, "color", "c");
 axis tight
 set (h.violin, "linewidth", 2);
 set (gca, "xgrid", "on");
 xlabel ("Variables")
 ylabel ("Values")
!!!!! demo failed
horizontal dimensions mismatch (4x1 vs 1x1)


The other FAILures seem to be the same; I  attached output of
"test("violin", "verbose", "violin_err.txt")" just in case.
Cc @rik.

We should probably change bug classification to octave/regression.

Dmitri.
--


(file #54849)

Dmitri A. Sergatskov <dasergatskov>
Tue 20 Jun 2023 05:48:44 PM UTC, comment #3: 

It seems this changeset: https://hg.savannah.gnu.org/hgweb/octave/rev/e67b7b85670b

changed the internals of corr.m from this:

  cov (x, y);

to this:

  cov ([x, y]);

which causes most of the errors seen.

Copying nrjank.

Arun Giridhar <arungiridhar>
Group Member
Tue 20 Jun 2023 05:44:09 PM UTC, comment #2: 


changeset:   32047:e3de59065cf1
user:        Rik <rik@octave.org>
date:        Fri Apr 21 11:01:40 2023 -0700
summary:     inputParser.m: Re-architect internal data structures for 60% speed improvement.


seems to be responsible in 5 failures in "test violin"

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 20 Jun 2023 04:33:27 PM UTC, comment #1: 

I confirm the failures with octave 9.0.0, but the test passes with octave 8.2.1. So I am not sure if this is statistics bug or octave regression.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 20 Jun 2023 03:58:48 PM UTC, original submission:  

With statistics 1.6.0, I am getting the following test failures.

Whether VM is enabled or not, these 8 errors show up about row / column mismatch, and one index out of bounds:


>>>>> processing ..../statistics-1.6.0/dist_fit/gevfit.m
***** test
 x = 1:50;
 [pfit, pci] = gevfit (x);
 pfit_out = [-0.4407, 15.1923, 21.5309];
 pci_out = [-0.7532, 11.5878, 16.5686; -0.1282, 19.9183, 26.4926];
 assert (pfit, pfit_out, 1e-3);
 assert (pci, pci_out, 1e-3);
!!!!! test failed
horizontal dimensions mismatch (1x50 vs 50x1)
***** test
 x = 1:2:50;
 [pfit, pci] = gevfit (x);
 pfit_out = [-0.4434, 15.2024, 21.0532];
 pci_out = [-0.8904, 10.3439, 14.0168; 0.0035, 22.3429, 28.0896];
 assert (pfit, pfit_out, 1e-3);
 assert (pci, pci_out, 1e-3);
!!!!! test failed
horizontal dimensions mismatch (1x25 vs 25x1)


>>>>> processing ..../statistics-1.6.0/correlation_test.m
***** test
 x = [6 7 7 9 10 12 13 14 15 17];
 y = [19 22 27 25 30 28 30 29 25 32];
 [h, pval, stats] = correlation_test (x, y);
 assert (stats.corrcoef, corr (x, y), 1e-14);
 assert (pval, 0.0223, 1e-4);
!!!!! test failed
c(10,_): out of bound 1 (dimensions are 1x1)


>>>>> processing ..../statistics-1.6.0/violin.m
***** test
 set (0, "DefaultFigureVisible", "off");
 data = exprnd (0.1, 500,4);
 violin (data, "color", jet(4));
 axis ([0 5 0 max(data(:))])
 set (0, "DefaultFigureVisible", visibility_setting);
!!!!! test failed
horizontal dimensions mismatch (4x1 vs 1x1)
shared variables     visibility_setting = on
***** test
 set (0, "DefaultFigureVisible", "off");
 data = {randn(100,1)*5+140, randn(130,1)*8+135};
 subplot (1,2,1)
 title ("Grade 3 heights - vertical");
 set (gca, "xtick", 1:2, "xticklabel", {"girls"; "boys"});
 violin (data, "Nbins", 10);
 axis tight
 set (0, "DefaultFigureVisible", visibility_setting);
!!!!! test failed
horizontal dimensions mismatch (4x1 vs 1x1)
shared variables     visibility_setting = on
***** test
 set (0, "DefaultFigureVisible", "off");
 data = {randn(100,1)*5+140, randn(130,1)*8+135};
 subplot (1,2,1)
 title ("Grade 3 heights - vertical");
 set (gca, "xtick", 1:2, "xticklabel", {"girls"; "boys"});
 violin (data, "Nbins", 10);
 axis tight
 subplot(1,2,2)
 title ("Grade 3 heights - horizontal");
 set (gca, "ytick", 1:2, "yticklabel", {"girls"; "boys"});
 violin (data, "horizontal", "Nbins", 10);
 axis tight
 set (0, "DefaultFigureVisible", visibility_setting);
!!!!! test failed
horizontal dimensions mismatch (4x1 vs 1x1)
shared variables     visibility_setting = on
***** test
 set (0, "DefaultFigureVisible", "off");
 data = repmat(exprnd (0.1, 500,1), 1, 4);
 violin (data, "nbins", [5,10,50,100], "smoothfactor", [4 4 8 10]);
 axis ([0 5 0 max(data(:))])
 set (0, "DefaultFigureVisible", visibility_setting);
!!!!! test failed
horizontal dimensions mismatch (3x1 vs 1x1)
shared variables     visibility_setting = on
***** test
 set (0, "DefaultFigureVisible", "off");
 data = repmat(exprnd (0.1, 500,1), 1, 4);
 violin (data, "width", linspace (0.1,0.5,4));
 axis ([0 5 0 max(data(:))])
 set (0, "DefaultFigureVisible", visibility_setting);
!!!!! test failed
horizontal dimensions mismatch (4x1 vs 1x1)
shared variables     visibility_setting = on


When VM is enabled, there are some 26 additional errors where the expected error is "functionname: function called with too many inputs" but the actual error omits the function name and just gives "function called with too many inputs". Looking through the packages and the Octave code base, it seems that only the Statistics package adds the function name to the expected error, causing a BIST failure when the VM-compiled version is used instead. A patch ("git diff") is attached to remove those function names from the error messages.

Arun Giridhar <arungiridhar>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #54849:  violin_err.txt.gz added by dasergatskov (655B - application/gzip)
file #54848:  statistics.patch added by arungiridhar (18KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by nrjank
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by arungiridhar
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by arungiridhar (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-04-15 nrjank StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2023-07-10 nrjank Carbon-Copy- Added pr0m1th3as
    2023-06-21 rik5 StatusConfirmed In Progress
    2023-06-21 mmuetzel StatusNone Confirmed
        Carbon-Copy- Added rik5
    2023-06-20 arungiridhar CategoryOctave Package Octave Function
        Item GroupUnexpected Error or Warning Regression
        Summary[Octave Packages] (statistics) BIST failures Changes to Octave core causing BIST failures in Statistics pkg
    2023-06-20 dasergatskov Attached File- Added violin_err.txt.gz, #54849
    2023-06-20 arungiridhar Carbon-Copy- Added nrjank
    2023-06-20 arungiridhar Attached File- Added statistics.patch, #54848

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code