bugGNU Octave - Bugs: bug #29553, missing arpack: make check...

 
 

bug #29553: missing arpack: make check segfaults on eigs

Submitter:  Petr Mikulik <mikulik>
Submitted:  Wed 14 Apr 2010 07:34:56 PM UTC
   
 
Category:  Configuration and Build System Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Works For Me Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 08 Jun 2010 03:11:38 PM UTC, comment #20: 

The test code doesn't recognize "%! shared".  There can be no space between the "%!" and the "shared" keyword.

If you write

  %!testif FOO
  %!shared
  %! ...

this is just saying that there is an empty test block that is executed if FOO is "true", followed by a block of statements that are to be shared by any following tests.  So the statements in the shared block are still executed.

The "testif" directive is not just a conditional, but also introduces a test block.  If you lump them all together, you will only see one failure no matter how many asserts are triggered.  It will make it much harder to determine precisely which assert statement failed.

Note that there is a difference between

  %!testif FOO
  %! assert (cond)
  %! assert (cond)

and

  %!assert (cond)
  %!assert (cond)

The first is a single test with two assertions.  The second is a set of two tests.

Yes, the syntax for inline tests is rigid and imperfect, but I don't think it is worth a lot of effort to fix these things.  Is there really any harm in computing a few values even if they are not actually needed?

John W. Eaton <jwe>
Group administrator
Tue 08 Jun 2010 09:20:46 AM UTC, comment #19: 

File eigs.cc still contains testing code block which does nothing unless Octave is compiled with arpack:

%!shared n, k, A, d0, d2
%! n = 20;
%! k = 4;
%! A = sparse([3:n,1:n,1:(n-2)],[1:(n-2),1:n,3:n],[ones(1,n-2),4*ones(1,n),ones(1,n-2)]);
%! d0 = eig (A);
%! d2 = sort (d0);
%! [~, idx] = sort (abs(d0));
%! d0 = d0(idx);

There is no assert() command afterwards, thus these commands are useless.

I think the whole test code should be in %!testif HAVE_ARPACK,

%% Real positive definite tests, n must be even
%!testif HAVE_ARPACK
%! shared n, k, A, d0, d2
%! n = 20;
%! k = 4;
%! d1 = eigs (A, k);
...

instead of tens of
%!testif HAVE_ARPACK
%! d1 = eigs (A, k);
%! assert (d1, d0(end:-1:(end-k+1)), 1e-11);


Petr Mikulik <mikulik>
Mon 07 Jun 2010 08:36:32 PM UTC, comment #18: 

Closed, after 1 month without response to request for clarification.

Rik <rik5>
Group administrator
Thu 06 May 2010 08:56:16 PM UTC, comment #17: 

I checked in the following change for the xgl2ps.c compile problem:

  http://hg.savannah.gnu.org/hgweb/octave/rev/31717cc7eeb3

John W. Eaton <jwe>
Group administrator
Thu 06 May 2010 08:47:47 PM UTC, comment #16: 

You wrote:

"The compile went fine after installing libpcre3-dev, thanks for the hint. Finally, "make check" passes OK on that debian machine..."

Is that with or without ARPACK installed?  If the tests pass even when you don't have ARPACK installed, then why should we add "testif ARPACK" to skip the test block?

John W. Eaton <jwe>
Group administrator
Tue 04 May 2010 01:38:58 PM UTC, comment #15: 

The compile went fine after installing libpcre3-dev, thanks for the hint. Finally, "make check" passes OK on that debian machine but not on my OpenSUSE machine with the same gcc; therefore the problem might be in a lapack library or elsewhere as John has indicated.

In summary, I propose the following two patches to Octave:

1. See comment #13:
   Patch xgl2ps.c in order not to #include "gl2ps.c" without the associated HAVE_*GL*

2. See comment #9:
   The whole test code block in eigs.cc should be prefaced by "testif HAVE_ARPACK":
%!testif HAVE_ARPACK
%! n = 20;
%! k = 4;
   I've enclosed the patch.

Is this fine? Could somebody commit these patches?


(file #20425)

Petr Mikulik <mikulik>
Mon 03 May 2010 09:08:01 PM UTC, comment #14: 

Since you are trying to create a stripped down environment you can dispense with building the documentation.  Pass '--disable-doc' to configure which will silence your warning #2.

For warning #3, I checked and my configure is passing -lpcre in for the regular expression libraries.  The package, on debian, was libpcre3-dev (Perl 5 Compatible Regular Expression Library)

Rik <rik5>
Group administrator
Mon 03 May 2010 07:35:50 PM UTC, comment #13: 

I would like to try to (not) reproduce the error in a debian system with gcc 4.3.2.
There are almost no libraries installed on that system (including GL).
There are following problems during make:


1. xgl2ps.c fails to compile. I patched it temporarily by:

$ ../run-octave
$ diff -u xgl2ps-orig.c xgl2ps.c
--- xgl2ps-orig.c       2010-05-03 20:52:06.000000000 +0200
+++ xgl2ps.c    2010-05-03 20:52:15.000000000 +0200
@@ -27,4 +27,6 @@
 #include <config.h>
 #endif

+#if 0  // SHOULD BE SOME #ifdef HAVE_XXXXXX ???
 #include "gl2ps.c"
+#endif


2. Make fails with this error:

Making all in interpreter
make[3]: Entering directory `/home/mikulik/work/octave-hg/octave/doc/interpreter'
../../run-octave -f -q -H ./mk_doc_cache.m doc-cache ../../scripts/DOCSTRINGS ../../src/DOCSTRINGS || { rm -f doc-cache; exit 1; }
error: regexprep: Invalid content of \{\} in pattern (@seealso {([^}])})
error: called from:
error:   /home/mikulik/work/octave-hg/octave/doc/interpreter/mk_doc_cache.m at line 60, column 6

make[3]: * [doc-cache] Error 1
make[3]: Leaving directory `/home/mikulik/work/octave-hg/octave/doc/interpreter'


File mk_doc_cache.m is:

58 text = [text{:}, doc_delim];
59
60 text = regexprep (text, "@seealso {([^}])}", "See also: $1.");
61 text = regexprep (text, "-\\*- texinfo -\\*-[ \t]*[\r\n]*", "");
62 text = regexprep (text, "^_@", "^_@@");


3. Maybe the following is the same problem as in 2:

test> ../run-octave
octave:1> fntests

Integrated test scripts:

lineanchors: regexp not implemented in this version
octave:2> quit

Do I need to ask the administrator to install some kind of regexp library? Which one? On OpenSUSE I can find rpm packages regexp and gnu-regexp. Could ./configure check for this requirement?


Petr Mikulik <mikulik>
Thu 29 Apr 2010 09:12:05 PM UTC, comment #12: 

So it seems that in the first call to dgeev, the value of wi.rep is changed.  That shouldn't be happening.  Either we are calling dgeev incorrectly (some argument is out of order or some array is not allocated properly) or there is a bug in your lapack library's dgeev function, or the library is compiled with different calling conventions than are used when compiling Octave.

John W. Eaton <jwe>
Group administrator
Thu 29 Apr 2010 09:05:34 PM UTC, comment #11: 

1. I went through EIG::init(). Tracing and printing:

161       Array<double> wi (n, 1);
(gdb)
162       double *pwi = wi.fortran_vec ();
(gdb) p pwi
$24 = (double *) 0xb7fdf880
(gdb) p *pwi
$25 = 1.5251834411709584e-314
(gdb) p wi.rep
$26 = (class Array<double>::ArrayRep *) 0x83f6670
(gdb) next
...
174       F77_XFCN (dgeev, DGEEV, (F77_CONST_CHAR_ARG2 ("N", 1),
(gdb) p wi.rep
$27 = (class Array<double>::ArrayRep *) 0x83f6670
(gdb)
...
185           double *pwork = work.fortran_vec ();
(gdb)
187           F77_XFCN (dgeev, DGEEV, (F77_CONST_CHAR_ARG2 ("N", 1),
(gdb) p wi.rep
$31 = (class Array<double>::ArrayRep *) 0x40580000
...
208           v.resize (nvr, nvr);
(gdb)
210           for (octave_idx_type j = 0; j < n; j++)
(gdb)
212               if (wi.elem (j) == 0.0)
(gdb)

Program received signal SIGSEGV, Segmentation fault.
0xb74de425 in Array<double>::make_unique (this=0xbfffc52c) at ../liboctave/Array.h:111
111           if (rep->count > 1)
(gdb)


2. Blas and Lapack:
$ rpm -qa | grep blas
blas-3.2.0-2.4
libblas3-3.2.0-2.4
blas-man-3.2.0-2.4

$ rpm -qa | grep lapack
liblapack3-3.2.0-2.4
lapack-man-3.2.0-2.4
lapack-3.2.0-2.4


Petr Mikulik <mikulik>
Thu 29 Apr 2010 05:33:08 PM UTC, comment #10: 

I still am unable to reproduce the problem on my system after configuring with the options

  --without-arpack
  --without-umfpack
  --without-cholmod
  --without-ccolamd
  --without-amd
  --without-colamd
  --without-cxsparse
  --without-qrupdate

After building, I verified that Octave and eigs.oct are not linked with any of these libraries.

What lapack and blas libraries do you have installed?

Can you step through the EIG::init function as I asked?  If you can narrow down where wi.rep becomes invalid, maybe I can help debug the problem.  Otherwise, I have to guess that it is a problem with the lapack library on your system, and since I can't reproduce the problem, I can't debug it.

John W. Eaton <jwe>
Group administrator
Thu 29 Apr 2010 07:35:35 AM UTC, comment #9: 

BTW, ./configure reports:

configure: WARNING: UMFPACK not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: qrupdate not found.  The QR & Cholesky updating functions will be slow.
configure: WARNING: AMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: COLAMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CCOLAMD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CHOLMOD library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: CXSparse library not found.  This will result in some lack of functionality for sparse matrices.
configure: WARNING: arpack not found.  The eigs function will be disabled.
configure:
configure: NOTE: libraries may be skipped if a library is not found OR
configure: NOTE: if the library on your system is missing required features.


Petr Mikulik <mikulik>
Thu 29 Apr 2010 07:19:43 AM UTC, comment #8: 

I have found the problem when trying explicitly the test commands in eigs:

n = 20
k = 4
A = sparse ([3:n, 1:n, 1:(n - 2)], [1:(n - 2), 1:n, 3:n], [ones(1, n - 2), 1:n, -ones(1, n - 2)])
d0 = eig (A)

The above code crashes octave in eig(A) if n>=12.
Thus it is a problem of eig() with a larger sparse matrix.

Whatever is the reason for this crash, fntests.m would pass if "testif" commands are added to each test code block in eigs.cc:
%!testif HAVE_ARPACK
%! n = 20;
%! k = 4;

Petr Mikulik <mikulik>
Thu 29 Apr 2010 06:45:38 AM UTC, comment #7: 

OK, if you set a breakpoint in EIG::init at the point where wi is created, what is the value printed by

  p wi.rep

?  It looks like this value is invalid at the point of the call to the Array::make_unique method.  When does it become invalid?  Is it after the first call to dgeev?  The second?  Somewhere else?

If the call to dgeev is clobbering the value of wi.rep, then we are either calling dgeev incorrectly, not supplying sufficient workspace, etc., or there is a bug in your copy of dgeev, or there is some inconsistency in the way your dgeev is compiled vs the way Octave is calling it (for example, the f2c vs gfortran calling convention change).

John W. Eaton <jwe>
Group administrator
Thu 29 Apr 2010 06:23:00 AM UTC, comment #6: 

I'm enclosing the backtrace.


(file #20367)

Petr Mikulik <mikulik>
Wed 28 Apr 2010 09:27:45 PM UTC, comment #5: 

I'm also unable to reproduce this problem after passing --without-arpack to the configure script.

Can you try running the tests with gdb and see where it crashes?  To do that, edit the generated test/Makefile file and change the lines

check: test_sparse.m test_bc_overloads.m
$(top_builddir)/run-octave --norc --silent --no-history $(srcdir)/fntests.m $(srcdir)

to be

check: test_sparse.m test_bc_overloads.m
$(top_builddir)/run-octave -g --norc --silent --no-history $(srcdir)/fntests.m $(srcdir)

(i.e., add -g to the run-octave command).  Then run make check again, and when the "(gdb)" prompt appears, type "run".  When the crash occurs, get a backtrace with bt and post it here.  Maybe that will give some clues.

John W. Eaton <jwe>
Group administrator
Tue 27 Apr 2010 08:23:29 PM UTC, comment #4: 


> I am unable to reproduce this on a development version
> (83de7b060e91) with a gnulib pulled on 4/19/2010.


With or without arpack installed?

> Can you retry using the the latest version of the dev sources
> and gnulib?


The crash is also in sources as of 2010-04-27.

> It might also be useful to run just the eigs tests themselves
> rather than all of 'make check'
> From the compile tree try:
> ./run-octave -f --eval 'test eigs'


This works; here is the output:

octave$ ./run-octave -f --eval 'test eigs'
GNU Octave, version 3.3.51+
Copyright (C) 2010 John W. Eaton and others.
This is free software; see the source code for copying conditions.
There is ABSOLUTELY NO WARRANTY; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  For details, type `warranty'.

Octave was configured for "i686-pc-linux-gnu".

Additional information about Octave is available at http://www.octave.org.

Please contribute if you find this software useful.
For more information, visit http://www.octave.org/help-wanted.html

Read http://www.octave.org/bugs.html to learn how to submit bug reports.

For information about changes from previous versions, type `news'.

  *** testif HAVE_ARPACK
 d1 = eigs (A, k);
 assert (d1, d0(end:-1:(end-k+1)), 1e-11);
----- skipped test


However, "make check" or "octave> fntests" lead to crash as indicated in the bug report. I can reproduce the problem on two computers.

Petr Mikulik <mikulik>
Tue 27 Apr 2010 08:18:19 PM UTC, comment #3: 

I have no arpack anywhere on my system. Thus ./configure is correct, but
  octave> fntests
is doing something nasty ... crash in eigs instead of being ignored.

However, if I cd to "src/DLD-FUNCTIONS", then this works without problem:

octave> test eigs
  *** testif HAVE_ARPACK
 d1 = eigs (A, k);
 assert (d1, d0(end:-1:(end-k+1)), 1e-11);
----- skipped test
octave>

Petr Mikulik <mikulik>
Tue 27 Apr 2010 06:49:28 PM UTC, comment #2: 

I've seen similar crashes in eigs() in OSX. They were caused by an arpack library compiled with incompatible compiler options (with -ff2c), while Fortran code in Octave was compiled without -ff2c.

Maybe, in your case configure correctly finds that there is no compatible arpack library, but there still is an incompatible one that gets linked in? Maybe easiest to check by installing a compatible arpack library...

Anonymous
Fri 23 Apr 2010 03:27:26 AM UTC, comment #1: 

I am unable to reproduce this on a development version (83de7b060e91) with a gnulib pulled on 4/19/2010.

Can you retry using the the latest version of the dev sources and gnulib?

It might also be useful to run just the eigs tests themselves rather than all of 'make check'

From the compile tree try:
./run-octave -f --eval 'test eigs'

Rik <rik5>
Group administrator
Wed 14 Apr 2010 07:34:56 PM UTC, original submission:  

I do not have arpack installed; this is correctly detected by ./configure:

configure: WARNING: arpack not found.  The eigs function will be disabled.

However, "make check" does checking of eigs which leads to crash:

src/DLD-FUNCTIONS/dmperm.cc ............................ PASS    1/1
src/DLD-FUNCTIONS/eig.cc ............................... PASS   20/20
src/DLD-FUNCTIONS/eigs.cc ..............................panic: Segmentation fault -- stopping myself...
make[2]: * [check] Neoprávněný přístup do paměti (SIGSEGV)

When running Octave, it knows eigs is not compiled in:

> eigs

error: eigs: not available in this version of Octave

The crash happens in both 3.2.4 and dev.

Petr Mikulik <mikulik>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #20425:  eigs_testif.diff added by mikulik (1KiB - text/x-patch - patch for eigs.cc)
file #20367:  bt-eigs.txt added by mikulik (16KiB - text/plain - gdb backtrace)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mikulik (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-06-07 rik5 Open/ClosedOpen Closed
    2010-05-04 mikulik Attached File- Added eigs_testif.diff, #20425
    2010-04-29 mikulik Attached File- Added bt-eigs.txt, #20367
    2010-04-23 rik5 StatusNone Works For Me

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code