bugGNU Octave - Bugs: bug #49040, lscov test failure

 
 

bug #49040: lscov test failure

Submitter:  Orion Poplawski <opoplawski>
Submitted:  Fri 09 Sep 2016 06:18:59 PM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Inaccurate Result
Status:  Wont Fix Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 4.2.0-rc1 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 12 Nov 2016 01:16:29 PM UTC, comment #6: 

I think I've just had some remainders of old mxe-builts in my dist directory with MXE-Octave-4.2.0-rc3 (see comment #5). When building a (definitly clean) MXE-Octave-4.2.0-rc4, then all tests pass on WinXP. There is no failure on lscov.m now.

Hartmut <hardy>
Fri 04 Nov 2016 04:50:12 PM UTC, comment #5: 

I have just compiled

  • octave-4.2.0-rc3
  • via an up-to-date mxe-octave
  • and compilation done on an Ubuntu 14.04. system.


Letting this Octave run the test suite on WindowsXP gives me a failure of "test lscov.m" to my very surprise.

Running the official Windows installer of Octave 4.2.0-rc3 on the very same WinXP machine passes this test. (But this official Windows installer was obviously compiled on a different computer.)

Here is the relevant part of the fntest.log :


>>>>> processing E:\octave-2016-11-04-12-41_rc3-ohne-65001patch\share\octave\4.2.0-rc3\m\statistics\base\lscov.m
***** test
 ## Adapted from example in Matlab documentation
 x1 = [.2 .5 .6 .8 1.0 1.1]';
 x2 = [.1 .3 .4 .9 1.1 1.4]';
 X = [ones(size(x1)) x1 x2];
 y = [.17 .26 .28 .23 .27 .34]';
 [b, se_b, mse, S] = lscov(X, y);
 assert(b, [0.1203 0.3284 -0.1312]', 1E-4);
 assert(se_b, [0.0643 0.2267 0.1488]', 1E-4);
 assert(mse, 0.0015, 1E-4);
 assert(S, [0.0041 -0.0130 0.0075; -0.0130 0.0514 -0.0328; 0.0075 -0.0328 0.0221], 1E-4);
 w = [1 1 1 1 1 .1]';
 [bw, sew_b, msew] = lscov (X, y, w);
 assert(bw, [0.1046 0.4614 -0.2621]', 1E-4);
 assert(sew_b, [0.0309 0.1152 0.0814]', 1E-4);
 assert(msew, 3.4741e-004, -1E-4);
 V = .2*ones(length(x1)) + .8*diag(ones(size(x1)));
 [bg, sew_b, mseg] = lscov (X, y, V);
 assert(bg, [0.1203 0.3284 -0.1312]', 1E-4);
 assert(sew_b, [0.0672 0.2267 0.1488]', 1E-4);
 assert(mseg, 0.0019, 1E-4);
 y2 = [y 2*y];
 [b2, se_b2, mse2, S2] = lscov (X, y2);
 assert(b2, [b 2*b], 2*eps);
 assert(se_b2, [se_b 2*se_b], eps);
 assert(mse2, [mse 4*mse], eps);
 assert(S2(:, :, 1), S, eps);
 assert(S2(:, :, 2), 4*S, eps);
!!!!! test failed
ASSERT errors for:  assert (se_b2,[se_b, 2 * se_b],eps)

  Location  |  Observed  |  Expected  |  Reason
   (2,2)       0.45338      0.45338      Abs err 4.4409e-016 exceeds tol 2.2204e-016
   (3,2)       0.29757      0.29757      Abs err 2.7756e-016 exceeds tol 2.2204e-016


Hartmut <hardy>
Mon 12 Sep 2016 04:36:01 AM UTC, comment #4: 

So it looks like this test failure is due to using ATLAS vs OpenBLAS. Assuming there is nothing that Octave can do about this, it's a tolerance problem from within ATLAS.

I've added this bug number to the test case so when it fails it will be reported as a known bug and refer to this bug number.

Closing as won't fix.

Mike Miller <mtmiller>
Group Member
Fri 09 Sep 2016 08:38:02 PM UTC, comment #3: 

Yes. I can reproduce it with atlas.
Should Fedora switch to openblas?

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 09 Sep 2016 08:34:40 PM UTC, comment #2: 

It's likely due to the default Fedora compile flags (-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic).  Or perhaps my use of --enable-float-truncate (which I can't recall exactly why it was added).  Or perhaps the use of atlas.

Orion Poplawski <opoplawski>
Fri 09 Sep 2016 08:24:03 PM UTC, comment #1: 

Orion,

I built dev/-rc1 on few different computers (Xeon/AMD/i7) and
never saw this problem (running Fedora 24/25).

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Fri 09 Sep 2016 06:18:59 PM UTC, original submission:  


>>>>> processing /builddir/build/BUILD/octave-4.2.0-rc1/scripts/statistics/base/lscov.m

*** test
 ## Longley data from the NIST Statistical Reference Dataset
 Z = [  60323    83.0   234289   2356     1590    107608  1947
        61122    88.5   259426   2325     1456    108632  1948
        60171    88.2   258054   3682     1616    109773  1949
        61187    89.5   284599   3351     1650    110929  1950
        63221    96.2   328975   2099     3099    112075  1951
        63639    98.1   346999   1932     3594    113270  1952
        64989    99.0   365385   1870     3547    115094  1953
        63761   100.0   363112   3578     3350    116219  1954
        66019   101.2   397469   2904     3048    117388  1955
        67857   104.6   419180   2822     2857    118734  1956
        68169   108.4   442769   2936     2798    120445  1957
        66513   110.8   444546   4681     2637    121950  1958
        68655   112.6   482704   3813     2552    123366  1959
        69564   114.2   502601   3931     2514    125368  1960
        69331   115.7   518173   4806     2572    127852  1961
        70551   116.9   554894   4007     2827    130081  1962 ];
 ## Results certified by NIST using 500 digit arithmetic
 ## b and standard error in b
 V = [  -3482258.63459582         890420.383607373
         15.0618722713733         84.9149257747669
        -0.358191792925910E-01    0.334910077722432E-01
        -2.02022980381683         0.488399681651699
        -1.03322686717359         0.214274163161675
        -0.511041056535807E-01    0.226073200069370
         1829.15146461355         455.478499142212 ];
 rsd =  304.854073561965;
 y = Z(:,1); X = [ones(rows(Z),1), Z(:,2:end)];
 alpha = 0.05;
 [b, stdb, mse] = lscov (X, y);
 assert(b, V(:,1), 3e-6);
 assert(stdb, V(:,2), -1.e-5);
 assert(sqrt (mse), rsd, -1E-6);
!!!!! test failed
ASSERT errors for:  assert (b,V (:, 1),3e-6)

  Location  |  Observed  |  Expected  |  Reason
    (1)      -3482258.6346 -3482258.6346   Abs err 3.3514e-06 exceeds tol 3e-06

This has been present in Fedora rpm builds for a long time.

Orion Poplawski <opoplawski>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hardy (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by opoplawski (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-09-12 mtmiller StatusNone Wont Fix
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code