bugGNU Octave - Bugs: bug #34461, Error in svds test with 3.4.3-rc0

 
 

bug #34461: Error in svds test with 3.4.3-rc0

Submitted by:  Susi Lehtola <jussilehtola>
Submitted on:  Sun 02 Oct 2011 08:45:24 PM UTC  
 
Category: Configuration and Build SystemSeverity: 3 - Normal
Priority: 5 - NormalItem Group: None
Status: FixedAssigned to: None
Originator Name: Open/Closed: Closed
Release: 3.4.3Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 03 Apr 2013 08:03:08 PM UTC, comment #16:

Ed, I pushed your patch with two minor modifications:

* We generally use ## for comments, not %
* No space in the (bug #34461) commit message.

Result is here:

http://hg.savannah.gnu.org/hgweb/octave/rev/610e02bf9579

Thanks, closing this report.

Jordi GutiƩrrez Hermoso <jordigh>
Project Administrator
Thu 09 Aug 2012 04:23:41 AM UTC, comment #15:

I'm submitting a patch which I believe fixes this problem in addition to a few others that have been on the maintainers list. The main problem is defaulting the convergence tolerance to 1e-10 which is much too big. It's false economy to not let ARPACK iterate to max precision, which then makes it much more likely that the computed singular values will be close enough to those computed by the dense svd.
I put together a more realistic test with random sizes, shapes, and values; with this patch it passes hundreds of these, without it 10 percent or so fail. I could add this to the svds() test.

(file #26339)

Edward Meyer <eem2314>
Mon 10 Oct 2011 03:25:24 PM UTC, comment #14:

John: yes, that was what I was trying to say.

As to the tolerance, IMHO even 10*eps is pretty small; in double precision it's 15 significant digits.

I'm just saying that since different implementations give different results, it'd be better to make the error limit large enough to fit all of them.

Susi Lehtola <jussilehtola>
Mon 10 Oct 2011 03:05:34 PM UTC, comment #13:

The tolerance for this test is currently 2*eps. I hesitate to make it something larger without understanding why the test is failing, especially since it seems to be dependent on the linear algebra library that is used, and because it seems like it is a simple enough problem that the tolerance should not need to be very large.

John W. Eaton <jwe>
Project Administrator
Mon 10 Oct 2011 02:55:01 PM UTC, comment #12:

on cygwin using lapack 3.2.2
assert (svds(speye(10)), ones (6, 1), eps)
pass

while Atlas 3.9.52 needs
assert (svds(speye(10)), ones (6, 1), 2*eps)

otherwise
"maximum absolute error 4.44089e-16 exceeds tolerance 2.22045e-16"

Tested just using/removing the dlls and not building at all
libatlas-devel

marco atzeri <matzeri>
Mon 10 Oct 2011 02:28:19 PM UTC, comment #11:

I'm not sure that increasing the tolerance is the right thing to do here because it seems to me that svds (speye (10)) should be a relatively simple problem. Increasing the tolerance may just be hiding some real problem. I don't see the failure on my system with ATLAS 3.8.3.

Jussi, I don't understand your last question about runtime library detection or what you mean by "looks only at .so files (and not the libraries themselves)".

On my system, Octave is linked to -llapack and -lblas and it is up to the system to decide which libraries are actually used. Octave itself does not search for linear algebra libraries or open them with dlopen, if that is what you are asking.

John W. Eaton <jwe>
Project Administrator
Wed 05 Oct 2011 10:51:33 PM UTC, comment #10:

Does Octave do any runtime library detection?

If the runtime library detection only looks at .so files (and not the libraries themselves), then it would explain this phenomenon.

Could the test tolerance be increased before 3.4.3 is released?

Susi Lehtola <jussilehtola>
Tue 04 Oct 2011 05:47:59 AM UTC, comment #9:

An addition to the unexplained behavior of
atlas-devel:

I just built the current devel system on fc14 64 bit.
make check gave one error in the "gr" tests.

I removed atlas-devel on that system and reran make check.

NO ERRORS.

Any ideas about this?

Michael Godfrey <godfrey>
Project Member
Mon 03 Oct 2011 11:02:40 PM UTC, comment #8:

No, sorry, was thinking too far back. Replace
fftfilt with svds.

Sorry for noise.

Michael Godfrey <godfrey>
Project Member
Mon 03 Oct 2011 10:55:41 PM UTC, comment #7:

What, fftfilt as well...? This issue is only related to svds.

I can see that the svds test is very sensible, however this means that the test needs to be set to be more tolerant as to the accuracy.

But I still don't see how adding or removing a package consisting of symlinks can change how octave works.

Susi Lehtola <jussilehtola>
Mon 03 Oct 2011 09:22:34 PM UTC, comment #6:

I noticed that the libraries themselves are unchanged,
but somehow atlas-devel has the side-effect of breaking fftfilt.
It would be good to figure this out. It will surely come
back to cause trouble in the future.

Michael Godfrey <godfrey>
Project Member
Mon 03 Oct 2011 10:33:00 AM UTC, comment #5:

The svds test is very sensible to start point, library and system

For reference:

https://mailman.cae.wisc.edu/pipermail/octave-maintainers/2011-August/024420.html

https://mailman.cae.wisc.edu/pipermail/octave-maintainers/2011-September/024715.html

marco atzeri <matzeri>
Mon 03 Oct 2011 08:13:01 AM UTC, comment #4:

I see this as well, although I have no idea as to what causes this.

Removing atlas-devel only removes the .so symlinks necessary for development, but doesn't change the resolution of the dynamical linkage (the ATLAS libraries are still available and used(?)).

Susi Lehtola <jussilehtola>
Mon 03 Oct 2011 01:32:12 AM UTC, comment #3:

Following my previous comment:
On the failing machine I just did:
yum reinstall octave
(I had not done this after the yum remove atlas-devel)
Now this machine runs the svds test
without failure.
============================
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
octave:3> uname
ans =

scalar structure containing the fields:

sysname = Linux
nodename = vzero.stanford.edu
release = 2.6.40.4-5.fc15.x86_64
version = #1 SMP Tue Aug 30 14:38:32 UTC 2011
machine = x86_64

======================================
The question remains: whey does the presence of
atlas-devel have this (or any) effect?

Michael Godfrey <godfrey>
Project Member
Mon 03 Oct 2011 12:23:37 AM UTC, comment #2:

On one x86_64 fc15 system I get:

octave:2> assert (s, ones (6, 1), 2*eps);
octave:3>
Reducing tolerance to eps:
octave:5> assert (s, ones (6, 1), eps);
error: assert (s,ones (6, 1),eps) expected
1
1
1
1
1
1
but got
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
maximum absolute error 4.44089e-16 exceeds tolerance 2.22045e-16
error: called from:
error: /usr/share/octave/3.4.2/m/testfun/assert.m at line 237, column 5
============================================
But on another I get, as reported:
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
error: assert (s,ones (6, 1),2 * eps) expected
1
1
1
1
1
1
but got
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16
But then, if I do yum remove atlas-devel
I get:
octave:1> s = svds (speye (10));
octave:2> assert (s, ones (6, 1), 2*eps);
error: assert (s,ones (6, 1),2 * eps) expected
1
1
1
1
1
1
but got
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
maximum absolute error 1.77636e-15 exceeds tolerance 4.44089e-16
error: called from:
error: /usr/share/octave/3.4.2/m/testfun/assert.m at line 237, column 5
octave:2>
==================================
I removed atls-devel because I noticed that it was not
installed on the machine that works. But, I did not expect
the change above. I did notice that atlas-devel installs
libs in /usr/lib64/atlas:
/usr/lib64/atlas/libatlas.so
/usr/lib64/atlas/libcblas.so
/usr/lib64/atlas/libclapack.so
/usr/lib64/atlas/libf77blas.so
/usr/lib64/atlas/liblapack.so
/usr/lib64/atlas/libptcblas.so
/usr/lib64/atlas/libptf77blas.so
==========================================
This does not appear to make a difference, but something
about removing atlas-devel has changed the result.

Michael

Michael Godfrey <godfrey>
Project Member
Sun 02 Oct 2011 08:51:27 PM UTC, comment #1:

When I run the test with

$ rpm -q octave
octave-3.4.2-2.fc15.x86_64

on the same machine I get a maximum error of 6.66134e-16.

Maybe you could just cranck up the tolerance to something like 5*epsilon or whatnot?

Susi Lehtola <jussilehtola>
Sun 02 Oct 2011 08:45:24 PM UTC, original submission:

>>>>> processing /builddir/build/BUILD/octave-3.4.3-rc0/scripts/sparse/svds.m

***** test
s = svds (speye (10));
assert (s, ones (6, 1), 2*eps);
!!!!! test failed
assert (s,ones (6, 1),2 * eps) expected
1
1
1
1
1
1
but got
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16

Susi Lehtola <jussilehtola>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #26339:  hg-export added by eem2314 (2KiB - application/octet-stream)
file #24086:  fntests.log.gz added by jussilehtola (57KiB - application/x-gzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jordigh (Posted a comment)
  • -unavailable- added by eem2314 (Updated the item)
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by matzeri (Posted a comment)
  • -unavailable- added by godfrey (Posted a comment)
  • -unavailable- added by jussilehtola (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 03 Apr 2013 08:03:08 PM UTCjordighStatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Thu 09 Aug 2012 04:23:41 AM UTCeem2314Attached File-=>Added hg-export, #26339
    Sun 02 Oct 2011 08:45:24 PM UTCjussilehtolaAttached File-=>Added fntests.log.gz, #24086

    Back to the top


    Powered by Savane 3.1-cleanup1