bugGNU Octave - Bugs: bug #34461, Error in svds test with 3.4.3-rc0

 
 

bug #34461: Error in svds test with 3.4.3-rc0

Submitter:  Susi Lehtola <jussilehtola>
Submitted:  Sun 02 Oct 2011 08:45:24 PM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 3.4.3 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 03 Apr 2013 08:03:08 PM UTC, comment #16: 

Ed, I pushed your patch with two minor modifications:

  * We generally use ## for comments, not %
  * No space in the (bug #34461) commit message.

Result is here:

http://hg.savannah.gnu.org/hgweb/octave/rev/610e02bf9579

Thanks, closing this report.

Jordi GutiƩrrez Hermoso <jordigh>
Group Member
Thu 09 Aug 2012 04:23:41 AM UTC, comment #15: 

I'm submitting a patch which I believe fixes this problem in addition to a few others that have been on the maintainers list. The main problem is defaulting the convergence tolerance to 1e-10 which is much too big. It's false economy to not let ARPACK iterate to max precision, which then makes it much more likely that the computed singular values will be close enough to those computed by the dense svd.
I put together a more realistic test with random sizes, shapes, and values; with this patch it passes hundreds of these, without it 10 percent or so fail.  I could add this to the svds() test.

(file #26339)

Edward Meyer <eem2314>
Mon 10 Oct 2011 03:25:24 PM UTC, comment #14: 

John: yes, that was what I was trying to say.

As to the tolerance, IMHO even 10*eps is pretty small; in double precision it's 15 significant digits.

I'm just saying that since different implementations give different results, it'd be better to make the error limit large enough to fit all of them.

Susi Lehtola <jussilehtola>
Mon 10 Oct 2011 03:05:34 PM UTC, comment #13: 

The tolerance for this test is currently 2*eps.  I hesitate to make it something larger without understanding why the test is failing, especially since it seems to be dependent on the linear algebra library that is used, and because it seems like it is a simple enough problem that the tolerance should not need to be very large.

John W. Eaton <jwe>
Group administrator
Mon 10 Oct 2011 02:55:01 PM UTC, comment #12: 

on cygwin using lapack 3.2.2
    assert (svds(speye(10)), ones (6, 1), eps)
pass

while Atlas 3.9.52 needs
    assert (svds(speye(10)), ones (6, 1), 2*eps)

otherwise
"maximum absolute error 4.44089e-16 exceeds tolerance 2.22045e-16"

Tested just using/removing the dlls and not building at all
libatlas-devel


marco atzeri <matzeri>
Mon 10 Oct 2011 02:28:19 PM UTC, comment #11: 

I'm not sure that increasing the tolerance is the right thing to do here because it seems to me that svds (speye (10)) should be a relatively simple problem.  Increasing the tolerance may just be hiding some real problem.  I don't see the failure on my system with ATLAS 3.8.3.

Jussi, I don't understand your last question about runtime library detection or what you mean by "looks only at .so files (and not the libraries themselves)".

On my system, Octave is linked to -llapack and -lblas and it is up to the system to decide which libraries are actually used.  Octave itself does not search for linear algebra libraries or open them with dlopen, if that is what you are asking.

John W. Eaton <jwe>
Group administrator
Wed 05 Oct 2011 10:51:33 PM UTC, comment #10: 

Does Octave do any runtime library detection?

If the runtime library detection only looks at .so files (and not the libraries themselves), then it would explain this phenomenon.

Could the test tolerance be increased before 3.4.3 is released?

Susi Lehtola <jussilehtola>
Tue 04 Oct 2011 05:47:59 AM UTC, comment #9: 

An addition to the unexplained behavior of
atlas-devel:

I just built the current devel system on fc14 64 bit.
make check gave one error in the "gr" tests.

I removed atlas-devel on that system and reran make check.

NO ERRORS.

Any ideas about this?

Michael Godfrey <godfrey>
Group Member
Mon 03 Oct 2011 11:02:40 PM UTC, comment #8: 

No, sorry, was thinking too far back.  Replace
fftfilt with svds.

Sorry for noise.

Michael Godfrey <godfrey>
Group Member
Mon 03 Oct 2011 10:55:41 PM UTC, comment #7: 

What, fftfilt as well...? This issue is only related to svds.

I can see that the svds test is very sensible, however this means that the test needs to be set to be more tolerant as to the accuracy.

But I still don't see how adding or removing a package consisting of symlinks can change how octave works.

Susi Lehtola <jussilehtola>
Mon 03 Oct 2011 09:22:34 PM UTC, comment #6: 

I noticed that the libraries themselves are unchanged,
but somehow atlas-devel has the side-effect of breaking fftfilt.
It would be good to figure this out.  It will surely come
back to cause trouble in the future.



Michael Godfrey <godfrey>
Group Member
Mon 03 Oct 2011 10:33:00 AM UTC, comment #5: 
marco atzeri <matzeri>
Mon 03 Oct 2011 08:13:01 AM UTC, comment #4: 

I see this as well, although I have no idea as to what causes this.

Removing atlas-devel only removes the .so symlinks necessary for development, but doesn't change the resolution of the dynamical linkage (the ATLAS libraries are still available and used(?)).

Susi Lehtola <jussilehtola>
Mon 03 Oct 2011 01:32:12 AM UTC, comment #3: 

Following my previous comment:
On the failing machine I just did:
yum reinstall octave
(I had not done this after the yum remove atlas-devel)
Now this machine runs the svds test
without failure.
============================
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
octave:3> uname
ans =

  scalar structure containing the fields:

    sysname = Linux
    nodename = vzero.stanford.edu
    release = 2.6.40.4-5.fc15.x86_64
    version = #1 SMP Tue Aug 30 14:38:32 UTC 2011
    machine = x86_64

======================================
The question remains: whey does the presence of
atlas-devel have this (or any) effect?

Michael Godfrey <godfrey>
Group Member
Mon 03 Oct 2011 12:23:37 AM UTC, comment #2: 

On one x86_64 fc15 system I get:

octave:2> assert (s, ones (6, 1), 2*eps);
octave:3>
Reducing tolerance to eps:
octave:5> assert (s, ones (6, 1), eps);
error: assert (s,ones (6, 1),eps) expected
   1
   1
   1
   1
   1
   1
but got
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
maximum absolute error 4.44089e-16 exceeds tolerance 2.22045e-16
error: called from:
error:   /usr/share/octave/3.4.2/m/testfun/assert.m at line 237, column 5
============================================
But on another I get, as reported:
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
error: assert (s,ones (6, 1),2 * eps) expected
   1
   1
   1
   1
   1
   1
but got
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16
But then, if I do yum remove atlas-devel
I get:
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
error: assert (s,ones (6, 1),2 * eps) expected
   1
   1
   1
   1
   1
   1
but got
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
   1.0000
maximum absolute error 1.77636e-15 exceeds tolerance 4.44089e-16
error: called from:
error:   /usr/share/octave/3.4.2/m/testfun/assert.m at line 237, column 5
octave:2>
==================================
I removed atls-devel because I noticed that it was not
installed on the machine that works.  But, I did not expect
the change above.  I did notice that atlas-devel installs
libs in /usr/lib64/atlas:
/usr/lib64/atlas/libatlas.so
/usr/lib64/atlas/libcblas.so
/usr/lib64/atlas/libclapack.so
/usr/lib64/atlas/libf77blas.so
/usr/lib64/atlas/liblapack.so
/usr/lib64/atlas/libptcblas.so
/usr/lib64/atlas/libptf77blas.so
==========================================
This does not appear to make a difference, but something
about removing atlas-devel has changed the result.

Michael

Michael Godfrey <godfrey>
Group Member
Sun 02 Oct 2011 08:51:27 PM UTC, comment #1: 

When I run the test with

$ rpm -q octave
octave-3.4.2-2.fc15.x86_64

on the same machine I get a maximum error of 6.66134e-16.

Maybe you could just cranck up the tolerance to something like 5*epsilon or whatnot?

Susi Lehtola <jussilehtola>
Sun 02 Oct 2011 08:45:24 PM UTC, original submission:  


>>>>> processing /builddir/build/BUILD/octave-3.4.3-rc0/scripts/sparse/svds.m

  *** test
 s = svds (speye (10));
 assert (s, ones (6, 1), 2*eps);
!!!!! test failed
assert (s,ones (6, 1),2 * eps) expected
   1
   1
   1
   1
   1
   1
but got
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
   1.00000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16

Susi Lehtola <jussilehtola>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #26339:  hg-export added by eem2314 (2KiB - application/octet-stream)
file #24086:  fntests.log.gz added by jussilehtola (57KiB - application/x-gzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jordigh (Posted a comment)
  • -email is unavailable- added by eem2314 (Updated the item)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by matzeri (Posted a comment)
  • -email is unavailable- added by godfrey (Posted a comment)
  • -email is unavailable- added by jussilehtola (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-04-03 jordigh StatusNone Fixed
        Open/ClosedOpen Closed
    2012-08-09 eem2314 Attached File- Added hg-export, #26339
    2011-10-02 jussilehtola Attached File- Added fntests.log.gz, #24086

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code