Sun 26 Nov 2017 10:54:58 PM UTC, comment #43:
Apparently this bug report goes on and on. I fixed bug #52516 where a matrix of all zeros was returned when the matrix norm was NaN. I broke the test on matrix norm in to two parts so that a norm of NaN returns a NaN matrix and a norm of Inf returns a zeros matrix.
It would be best if we had some BIST tests for all of the corner case matrices which have caused problems in the past. Probably a file in the test/ directory.
|
Thu 23 Nov 2017 10:01:01 PM UTC, comment #42:
I found a few more instances in dMatrix.cc and fMatrix.cc where the new function norm1 could be used. I checked in the changes to stable here (http://hg.savannah.gnu.org/hgweb/octave/rev/117107eed917).
This bug can be closed again, or jwe can leave it open until MXE Octave supports installing Octave's own version of XERBLA.
|
Thu 23 Nov 2017 06:27:53 PM UTC, comment #41:
Indeed this patch solves the problem (checked on Win-7 machine).
Running __run_test_suite__ did not find any new error.
|
Wed 22 Nov 2017 05:57:54 AM UTC, comment #40:
I've attached a patch against stable that works for me. I introduced a new local function norm1 which calculates the 1-norm, but preserves any NaN. The 1-norm was actually being calculated in quite a few places so it makes sense to abstract it to its own function. Please try the patch and review it as I would like to commit it before the 4.2.2 release.
(file #42457)
|
Wed 22 Nov 2017 05:19:27 AM UTC, comment #39:
bug #50007 contains a patch that extends max() for arrays to accept a nanflag. Maybe it could be used here.
|
Tue 21 Nov 2017 08:36:52 PM UTC, comment #38:
In each of the the three bug reports the problem has essentially been operands with Inf or NaN that--through mldivide or mrdivide--end up calling particular Fortran routines in liboctave.
There are a number of ways in, but they only lead to a few routines in dMatrix.cc (and friends). So should something like my suggested code in comment #34 be implemented? Should it reside in place (3 instances in dMatrix.cc) or should a new local function in dMatrix.cc like norm1 be created?
More ambitiously, max() for arrays could be extended to accept a nanflag. There is a bug report about that somewhere, and Matlab has had this for a while. If this were done then the 1-norm would be quite simple to calculate without having to handroll any code.
|
Tue 21 Nov 2017 07:11:09 PM UTC, comment #37:
I guess it depends on what kinds of error messages and results you want from calls to BLAS and LAPACK functions for bad inputs.
I see a few different categories.
1. Incorrect parameters that shouldn't happen (we screw up a call, for example). We still don't want Octave to crash. So our version of xerbla should be called and throw an error.
2. Matrix and vector arguments with NaN or Inf values that don't make sense for functions that we call. Should we protect calls that we make and return results? This is a case-by-case thing. If we can identify and return "reasonable" results (NaNs, 0, Inf, whatever) instead of throwing an error, we probably should.
3. Matrix and vector arguments with NaN or Inf values that don't make sense for internal functions. Same as above, if we can prevent the call somehow, we probably should. But these will be harder to detect.
Since it's probably impossible to detect all possible cases, we really have to have our own version of xerbla that doesn't cause Octave to exit.
FWIW, it's clear that the BLAS authors intended for users to replace XERBLA with their own version. Back in the day, that was simple because you were building the library for yourself anyway. Now it seems absurd that there is no standard way for a user of the BLAS to provide a replacement for this function without modifying the library sources.
|
Tue 21 Nov 2017 06:53:56 PM UTC, comment #36:
@jwe: I support that approach. Unfortunately, Windows is difficult to develop for, at least when the code is originally written for a UNIX environment.
Octave can provide a vanilla version of itself with patched BLAS libraries that just works. If others want to experiment with their own systems and libraries, that is fine, but it isn't an Octave responlibility to get all combinations of HW/SW/Octave to work together.
Is there any code that could be removed if we assume the XERBLA handler can always be overwritten? I'm possibly thinking of code for this bug, bug #45577, and bug #46330. We've attempted many workarounds in Octave for bugs which reside in Fortran libraries, and maybe not all of that needs to be kept now.
|
Tue 21 Nov 2017 06:04:19 PM UTC, comment #35:
The real problem is that we don't seem to have a way to override the call to xerbla on windows systems.
Well, we do, at least for our own builds. We can modify any BLAS libraries that we build so that they don't have their own xerbla routine, or so that it can call a function we supply.
This doesn't fix the problem for everyone, or for people who want to replace the blas library with their own. Unless they also do the same.
I thought we just had to worry about bad values passed to the BLAS or LAPACK functions that we call directly, but that's not the case. So I don't see how it's possible to check all the calls that we make so that there is no possible way that a bad BLAS or LAPACK call can happen INTERNALLY to BLAS or LAPACK. We can't possibly know all the code paths that could happen.
I think I can fix our Windows build so that the reference BLAS and OpenBLAS do the right thing and also document the issue. That would be good enough for me.
|
Tue 21 Nov 2017 05:50:38 PM UTC, comment #34:
The problem is that the 1-norm of the matrix is being calculated. The code is
The 1-norm is the largest column sum of the absolute value of A, but NaN always fails in comparisons, so the fact that one column sum yields NaN is not enough to make the norm NaN.
This can be fixed by changing the norm to something else, like 2, but this implies a singular value decomposition and a lot, lot more computation.
Maybe this is worth it, but I also hate having to code to a specific operating system. If Windows has a problem then perhaps this section of code should be protected by #ifdefs so that only Windows users have to experience the slow down.
As a possible intermediate solution one could write a routine to calculate the 1-norm, but respect the presence of NaNs.
Possible code
|
Tue 21 Nov 2017 07:23:14 AM UTC, comment #33:
I confirm behaviour of comment #21 in linux, both 4.2.1 and dev, with blas, openblas, atlas.
|
Tue 21 Nov 2017 06:47:50 AM UTC, comment #32:
On current dev it crashes so fast that I do not even see any error messages or Windows pop-ups. (Remarkable as I know that behavior (= application window simply disappearing w/o a trace) only from Linux, Windows usually catches misbehaved programs and complains.) CLI or GUI makes no difference.
Win 7, 64bit, a01c210e4287
|
Tue 21 Nov 2017 05:19:34 AM UTC, comment #31:
The line
Causes crash in Windows, and the error
In Linux.
The crash is in line 2217 of dMatrix.cc. The check
Does not help.
A solution could something like
But:
- Is there a better way to find if there is a NaN/Inf in the matrix?
- Where should we locate this check?
|
Sun 28 May 2017 06:18:06 PM UTC, comment #30:
I am reopening this bug since there is an example in bug #51092
that makes Octave crash in Windows:
The error message before the crash is:
|
Thu 06 Oct 2016 03:59:02 PM UTC, comment #29:
I pushed this changeset:
http://hg.savannah.gnu.org/hgweb/octave/rev/5aa8f199e328
Closing this rerport as fixed. If more cases pop up, we need to fix them individually. I don't see any generally reliable way to replace xerbla with our own function.
|
Wed 05 Oct 2016 08:20:02 PM UTC, comment #28:
At least for the reference BLAS and LAPACK, I think that the only way that xerbla can be called is if we call a BLAS or LAPACK function incorrectly. If we can avoid doing that, then we shouldn't even need to override the default version of XERBLA.
It is still nice to replace that function on systems where it is possible (and really, it should be possible on all systems) so that we can avoid causing Octave to exit if there is an accidentally incorrect call to a BLAS or LAPACK function.
|
Wed 05 Oct 2016 07:48:18 PM UTC, comment #27:
Sorry, I meant:
Without this patch every one of the examples ends with a crash (on Windows).
Maybe there is a better solution, I do not know....
|
Wed 05 Oct 2016 07:44:28 PM UTC, comment #26:
@Avinoam:
Your answer is a bit ambiguous to me, sorry.
"... ends with a crash" - with or without the patch ?
In my case patch file #38631 fixed the issue AFAICS.
|
Tue 04 Oct 2016 09:06:19 PM UTC, comment #25:
>> "does this patch really help the windows users?"
In Windows, every one of the examples ends with a crash.
|
Tue 04 Oct 2016 09:00:04 PM UTC, comment #24:
Windows seems to be more important, but for Linux there is no regression with file #38631
So does this patch really help the windows users and should this patch go in the 4.2 release? Then it should happen very soon.
|
Sun 02 Oct 2016 03:52:23 PM UTC, comment #23:
That latest patch (file #38631) seems to work fine in a --enable-windows-64 cross build:
I'm no C++ style expert but it looks good enough to me.
Should the "[ inf 2345 ] / 5" case be included as a test to be able to detect regressions?
|
Fri 30 Sep 2016 04:17:39 PM UTC, comment #22:
Please review the attached patch
(file #38631)
|
Tue 30 Aug 2016 06:41:27 AM UTC, comment #21:
@Avinoam: just nasty because I find it tedious to criticize coding style, if the patch itself is good ;-) sorry a bit off topic
|
Mon 29 Aug 2016 07:38:39 PM UTC, comment #20:
@Kai: Thanks for your remarks (Why nasty?)
Actually, this patch does solve some other bug reports, like bug #46330, (test/bug-46330.tst can be tested on Windows too).
About the coding stile - I'll try to improve the patch. Thanks :-)
|
Mon 29 Aug 2016 08:37:52 AM UTC, comment #19:
I have tested the patch of comment #18 on Linux without problems. Calls like in comment #17 result in a nice warning, like Matlab does.
Before:
After:
But as said in comment #17, only one of many potential cases is fixed by this patch.
Another nasty remark is the code formatting. Please do not introduce multiple empty lines and lines with only white space characters. See http://wiki.octave.org/C%2B%2B_style_guide.
|
Mon 29 Aug 2016 04:23:39 AM UTC, comment #18:
Another patch included. Please review.
This patch calculates the matrix norm, and if it is Inf or NaN,
the result is set to zero, rcon is also set to zero, and a singular matrix warning is shown. I have used
octave::warn_singular_matrix (), but maybe another warning should be used.
(file #38375)
|
Tue 23 Aug 2016 09:51:49 AM UTC, comment #17:
Checking the example
The error (and the crash after it) is produced in calling
(line 2130 in liboctave/array/dMatrix.cc).
A workaround can be something like:
Checking the matrix norm (like in http://hg.savannah.gnu.org/hgweb/octave/rev/bf5fbf347aaf), and avoiding calling dgelsd in case of problematic norn.
But this changeset does not set an error or warning, as it should.
Such modification should also apply to other *Matrix.cc files.
|
Fri 29 Jul 2016 03:22:57 PM UTC, comment #16:
Unfortunately, not all BLAS libraries make it easy to override the default error handler even though that was the original intent of having xerbla...
I suppose we could fix this for our mxe-octave builds for openblas and the reference blas. But that doesn't solve the problem generally. To do that requires cooperation from the people who are writing and packaging BLAS libraries.
Also, as I understand it, XERBLA should only be invoked when a BLAS or LAPACK routine is called incorrectly and we really shouldn't be making calls with invalid parameters.
For the example in comment 13, isn't that just a variation of bug #46330? But even though that bug is closed, I don't think it is really fixed yet. I'll comment there as well.
|
Thu 05 Feb 2015 01:25:19 AM UTC, comment #15:
I just got a Windows VM set up, same segfault here.
|
Thu 05 Feb 2015 01:16:43 AM UTC, comment #14:
The simpler test case
produces the error you cited
The Windows OS then catches the error and says that the application tried to read memory at 0x010 (clearly a segfault) and then the program exits.
Re-titling the bug to reflect the real issue of being unable to override the XERBLA handler on Windows.
|
Wed 04 Feb 2015 03:27:08 PM UTC, comment #13:
Yes, this particular test case looks solved.
However, you're not seeing the xerbla error message that's supposed to show up if this test case were still triggering the underlying problem.
I closed bug #36211 as a duplicate of this one. The simpler test case in that report still shows the xerbla message on Linux for me, what does
do for you on Windows? On Linux, it prints "On entry to DLASCL parameter number 4 had an illegal value" and then catches the error and returns to the Octave prompt. On Windows, last time I checked, this would cause Octave to abort, which is what this bug is really about.
|
Wed 04 Feb 2015 04:15:30 AM UTC, comment #12:
@Mike: I used a very recent tip (2/3/15) and I don't experience any problem with the test data. I used the NSIS installer route and tried both reference BLAS and OpenBLAS. I think whatever gremlin was there has been excised. If you're fine with that level of testing you can close the report.
|
Wed 04 Feb 2015 01:53:23 AM UTC, comment #11:
@Mike: I have a Windows XP VM. I'll test tonight. Going through the MXE compile though is going to take 1-2 hours.
|
Fri 30 Jan 2015 04:06:00 PM UTC, comment #10:
Is the BLAS/LAPACK error handler still not being correctly overridden in the current MXE build? I was able to reproduce this a year ago, but I don't have my WinXP VM at the moment to test a current build.
|
Sat 11 Jan 2014 08:27:02 PM UTC, comment #9:
I get the same error as the original report with an MXE build of 3.8.0 in a Windows XP VM.
|
Sat 18 May 2013 08:25:07 AM UTC, comment #8:
The system is a Core i5-2540M @ 2.60 GHz with Windows 7 64-bit.
I have changed MXE to build gcc 4.8.0 as a cross-compiler, which includes gfortran. So that may also make a difference in terms of floating point arithmetic.
|
Fri 17 May 2013 03:17:56 PM UTC, comment #7:
I'm not able to reproduce the problem with the MXE build either.
I do see a message about the matrix being singular, but no error from lapack.
What hardware are you using? I'm running in a virtual box system with Windows 7 installed.
In any case, I guess we should fix the MXE build so that we can overrride XERBLA with our own version so that it doesn't exit.
|
Fri 17 May 2013 01:12:12 PM UTC, comment #6:
I also cannot reproduce on Linux with octave 3.7.3+, lapack 3.4.2.
|
Fri 17 May 2013 01:01:00 PM UTC, comment #5:
I did not modify the lapack version in my MXE build. The version is 3.4.2.
In SRC/dlasd8.f there is this code:
CALL DLASD4( K, J, DSIGMA, Z, WORK( IWK1 ), RHO, D( J ),
$ WORK( IWK2 ), INFO )
*
- If the root finder fails, the computation is terminated.
*
IF( INFO.NE.0 ) THEN
CALL XERBLA( 'DLASD4', -INFO )
RETURN
END IF
And XERBLA seems to print the error message that I am getting. DLASD8 in turn gets called via functions such as zgelsd.
|
Wed 15 May 2013 10:17:23 PM UTC, comment #4:
I'm unable to duplicate the problem. Are you using the reference version fo lapack that is included in the MXE-octave build, or replacing it with some other version?
The version of DLASD4 that is used by my MXE-octave build does not appear to do any argument checking. So I don't see where that message is coming from.
Also, Octave doesn't call DLASD4 directly, so if there is an invalid call, it seems that it would be a bug in LAPACK.
|
Wed 15 May 2013 04:47:01 PM UTC, comment #3:
That's because you can't do it in Win32 (at least not in the way you do it under Linux/UNIX). The functions in the BLAS DLL will always resolve xerbla to the symbol present in the BLAS DLL (I believe this is done at link time, when the DLL is created).
|
Wed 15 May 2013 04:32:17 PM UTC, comment #2:
This sounds like the same symptom as bug #38575. The xerbla error handler in liboctave is not overriding the xerbla error handler in the BLAS/LAPACK libraries.
|
Wed 15 May 2013 04:09:52 PM UTC, comment #1:
Just to be clear. This crashes Octave, closing the command line window or returning back the cmd shell.
|
Wed 15 May 2013 04:07:23 PM UTC, original submission:
With my MXE build of the development branch I get this crash with the attached file.
% crashme.m
xy = load('xy.txt');
x = xy.x;
y = xy.y;
z = x\y;
> crashme
warning: matrix singular to machine precision, rcond = 1.57068e-020
- On entry to DLASD4 parameter number -1 had an illegal value
Where:
> size(xy.x)
ans =
350 350
> size(xy.y)
ans =
350 1
Does not happen with the 3.6.4 mingw build from the wiki:
> crashme
warning: matrix singular to machine precision, rcond = 8.43218e-021
|