bugGNU Octave - Bugs: bug #46330, Fortran exception with matrices...

 
 

bug #46330: Fortran exception with matrices containing Inf & NaN

Submitted by:  Lukas Reichlin <paramaniac>
Submitted on:  Fri 30 Oct 2015 04:30:53 AM UTC  
 
Category: LibrariesSeverity: 4 - Important
Priority: 5 - NormalItem Group: Segfault, Bus Error, etc.
Status: FixedAssigned to: Kai Torben Ohlhus <siko1056>
Originator Name: Open/Closed: Closed
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 16 Nov 2016 08:14:30 PM UTC, comment #38:

Thanks. Closed as fixed.

Avinoam Kalma <avinoam>
Project Member
Wed 16 Nov 2016 07:52:34 PM UTC, comment #37:

Yes. Odds are that if someone is still affected, we will get a new bug report anyway, and this is always searchable.

Mike Miller <mtmiller>
Project Administrator
Wed 16 Nov 2016 07:42:44 PM UTC, comment #36:

This bug seems to be solved by the solution of bug #39000.
Can I close this report?

Avinoam Kalma <avinoam>
Project Member
Thu 15 Sep 2016 05:40:55 PM UTC, comment #35:

sorry, comment 18.

Avinoam Kalma <avinoam>
Project Member
Thu 15 Sep 2016 05:40:22 PM UTC, comment #34:

Maybe comment 19 in bug #39000 is relevant here.

Avinoam Kalma <avinoam>
Project Member
Fri 29 Jul 2016 03:29:35 PM UTC, comment #33:

For the examples

I'm still seeing this:

In comment #29 Mike said that the patch should avoid calls to XERBLA, but that doesn't seem to be the case.

I'm using HG ID 8971508e21c8.

John W. Eaton <jwe>
Project Administrator
Fri 15 Jul 2016 10:13:44 PM UTC, comment #32:

I assume the newest patch from Kai works--it does for me. Closing report.

Rik <rik5>
Project Administrator
Mon 11 Jul 2016 03:02:15 PM UTC, comment #31:

Marking as Ready for Test.

Rik <rik5>
Project Administrator
Mon 11 Jul 2016 07:32:26 AM UTC, comment #30:

I gave it another try: http://hg.savannah.gnu.org/hgweb/octave/rev/bf5fbf347aaf

@Avinoam: as said in comment #20 and comment #29, I was not aware of bug #39000. Therefore my "debugging" attempt of file #37755 is worthless and I already deleted this file. I hope you'll find my regression test "untested" for Windows. Please report any further errors.

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Sat 09 Jul 2016 11:00:09 PM UTC, comment #29:

The crash on Windows is definitely bug #39000.

Any time you get a crash on Windows with the error message "On entry to ... had an illegal value", this is the BLAS library calling exit().

The proposed patch (file #37772) will bypass this test, so that should avoid the error on Windows, this looks good to me, untested.

Mike Miller <mtmiller>
Project Administrator
Sat 09 Jul 2016 01:23:22 PM UTC, comment #28:

Kai, Comment #19:

I get, using your patch (file #37755)

and than Octave crashes.

Avinoam Kalma <avinoam>
Project Member
Fri 08 Jul 2016 03:33:20 PM UTC, comment #27:

My state so far is in file #37772 . I won't change anything further and push that cset, if I hear no complaints, on Monday.

@Lachlan: Yes, I copied Rik's patch for bug #45577 as well.

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Fri 08 Jul 2016 02:44:07 PM UTC, comment #26:

Thanks, Kai. I assume that you also copied Rik's patch for bug #45577 as well, then.

It would be nice for those examples to give answers, but I don't think that should block 4.2.0, whereas the crashes do. How about applying your current patch and fixing the errors reported by Fortran separately?

Lachlan Andrew <lachlan>
Project Member
Fri 08 Jul 2016 02:08:09 PM UTC, comment #25:

Kai: Earlier in the year I started trying to reduce that duplication but got sidetracked by other issues that needed to be fixed first. Still planning to work on it some more though. No matter what, there will have to be some duplication because we have to specialize templates to call the underlying LAPACK and BLAS functions.

John W. Eaton <jwe>
Project Administrator
Fri 08 Jul 2016 01:55:31 PM UTC, comment #24:

@Lachlan: I did actually diffed CMatrix.cc and fCMatrix.cc (evil code duplication) and applied your patch symmetrically. I will upload it later, as I still get hard errors for other cases like

where Matlab does not give up, but displays a warning. This behavior I would like to make consistent.

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Fri 08 Jul 2016 01:12:03 PM UTC, comment #23:

OK, I'll write a patch for fCMatrix.cc. That hasn't yet been patched for bug #45577 either. Should I write a single patch for both (the affected lines are virtually the same) keep them separate?

Lachlan Andrew <lachlan>
Project Member
Thu 07 Jul 2016 10:10:50 PM UTC, comment #22:

Thanks, and sorry, I realized later this was a collaboration of yours and Lachlan's along with your followup test case.

Sounds good, please investigate the single precision case.

I think it should be possible to add the test case back in, with a comment and a check for ispc() to simply avoid passing bad values into LAPACK on Windows systems where we know about this limitation.

In fact since you are using %!warning, maybe something like this excerpt, based on your commit?

Mike Miller <mtmiller>
Project Administrator
Thu 07 Jul 2016 10:00:19 PM UTC, comment #21:

If bug #39000 was the cause of trouble for the segfault, then I have no cure for it. My current idea was to narrow the guilty lines of code, but if XERBLA causes a segfault with Windows in any case, than a simple check for info will never help. I wasn't aware of it. Here I think fixing bug #39000 was much easier, than checking all preconditions before calling any LAPACK routine. Marking bug #39000 as dependency, and tag this one as WON'T FIX in the wiki.

The fact is, for Linux systems Lachlans patch avoids (from a user perspective) strange error messages about illegal parameters from interally called LAPACK routines.

Your first point should be considered for sure. I will check this example for single values soon.

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Thu 07 Jul 2016 02:38:34 PM UTC, comment #20:

Kai, a couple questions.

  • Should the fix for this bug also include a case for FloatComplexMatrix? You only patched ComplexMatrix. Any test cases should also test both double and single values.
  • Since this involves LAPACK functions, does your test case involve calling a LAPACK function and having Octave handle the error? If so, then this may crash on Windows simply because of bug #39000, unable to override XERBLA function on Windows.
Mike Miller <mtmiller>
Project Administrator
Thu 07 Jul 2016 01:43:33 PM UTC, comment #19:

@avinoam: I backed out my Cset. Can you apply the patch (file #37755), run something like

And just post the console output? For me it looks like:

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Thu 07 Jul 2016 01:00:39 PM UTC, comment #18:

Reopen this bug, see bug #48431

Avinoam Kalma <avinoam>
Project Member
Wed 06 Jul 2016 12:57:59 PM UTC, comment #17:

Pushed: http://hg.savannah.gnu.org/hgweb/octave/rev/571b42508e1e

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Wed 06 Jul 2016 07:15:52 AM UTC, comment #16:

Thanks for the reminder to add the regression test, and for adding it. Yes, please apply the patch.

Lachlan Andrew <lachlan>
Project Member
Tue 05 Jul 2016 02:30:04 PM UTC, comment #15:

@Lachlan: I created a successor of your patch. If you agree on creating this extra regression test case, I will submit it.

The test script segfaulted my Octave 4.0.2 and after applying this patch, no new errors are introduced, and I only get a gentle warning.

(file #37718)

Kai Torben Ohlhus <siko1056>
Project MemberIn charge of this item.
Sun 26 Jun 2016 11:42:53 PM UTC, comment #14:

I think this patch (or its successor) should be applied before 4.2 is released.

Perhaps it could issue a warning when inv() is returning a zero due to infinite anorm if no-one cane confirm that that is always valid.

Lachlan Andrew <lachlan>
Project Member
Mon 02 Nov 2015 07:48:12 PM UTC, comment #13:

@Lucas: so Mac OSX is more vulnerable than Linux and Windows, good to know.

@Lachlan:
In my Fortran times (20-30+ years back) the FORTRAN math libs didn't quite bother catching NaNs, Infs, uninitialized memory bit patterns and possibly a couple of other non-numbers (I suppose one reason Fortran could be so fast); and happily produced just garbage instead. Of course there were some limits but esp. for the "OPT=3" cases they were lax.
Maybe Fortran has "improved" since those days of CDC Cyber mainframes but I actually doubt it.
So, testing for +/- Inf besides NaN sounds like a valuable addition :-)

Philip Nienhuis <philipnienhuis>
Project Member
Sun 01 Nov 2015 10:06:11 AM UTC, comment #12:

Philip, I'm using Netlib's reference BLAS and LAPACK.

Lukas Reichlin <paramaniac>
Sun 01 Nov 2015 08:05:22 AM UTC, comment #11:

Oops -- bad markup. I should have said

Lachlan Andrew <lachlan>
Project Member
Sun 01 Nov 2015 08:04:14 AM UTC, comment #10:

I think this is related to bug #45577.

The attached patch works around the problem on my Ubuntu system, and a related problem with inv (iInfe - a).

The solution for inv returns a zero, when it may perhaps be possible that some elements are non-zero but anorm is still infinite -- I don't understand the algorithm or LAPACK well enough to tell.

I hope the patch is a pointer to someone more knowledgeable who can fix it properly.

(file #35352)

Lachlan Andrew <lachlan>
Project Member
Sun 01 Nov 2015 04:26:41 AM UTC, comment #9:

@Philip (comment #6):
Checking in Ubuntu, I got the same results as you get in Windows"

fails, while

works.

Avinoam Kalma <avinoam>
Project Member
Sat 31 Oct 2015 07:58:46 PM UTC, comment #8:

On Mageia-5 (w. standard BLAS) I get:

so it errors at some other FORTRAN subroutine than DLASCL here.

I had a first go at a more descriptive title; better suggestions?

Even if OpenBLAS crashes Octave, the error is still there with BLAS. And we don't know yet what BLAS the OP has used on Mac OSX - Lucas?

Philip Nienhuis <philipnienhuis>
Project Member
Sat 31 Oct 2015 05:14:37 PM UTC, comment #7:

On Debian (and probably Ubuntu also) it crashes when using OpenBLAS but not when using netlib BLAS.

With netlib:

and Octave continues running. Using OpenBLAS:

Mike Miller <mtmiller>
Project Administrator
Sat 31 Oct 2015 04:30:58 PM UTC, comment #6:

On Ubuntu it may not crash, but I think the FORTRAN error shouldn't happen either. So OS should still be "Any".

Could you try breaking down the command like in comment #3 and see if it fails in the same step?

Philip Nienhuis <philipnienhuis>
Project Member
Sat 31 Oct 2015 04:18:28 PM UTC, comment #5:

I have checked this script on Linux and on Windows.

In Windows (Win7, Octave 4.1.0+, 32bit) it crashed.

On Ububtu It did not crash:

>>


So the OS could be non-Unix?

Avinoam Kalma <avinoam>
Project Member
Fri 30 Oct 2015 08:56:01 AM UTC, comment #4:

I stumbled across this bug when I tried the following with the control package:

where argument w is a vector of frequencies. If w contains any Inf's or NaN's, then Octave will crash.

BTW: As a consequence, I will change the functions in _control_helper_functions_.oct such that they only accept values that are considered as finite by Octave's isfinite mapper. Inf's and NaN's don't make sense in transfer functions polynomials and state-space matrices anyway. These changes will be part of control-3.0.1.

Lukas Reichlin <paramaniac>
Fri 30 Oct 2015 08:39:16 AM UTC, comment #3:

(Sigh - so consecutive minus verbatim and plus verbatim cut off postings)
Another try:

Pulling the equation out of arrayfun(), it turns out that Octave-4.1.0+ on Windows also crashes on:

Narrowing further,

works fine,

works fine as well, but

leads to a crash

Philip Nienhuis <philipnienhuis>
Project Member
Fri 30 Oct 2015 08:35:38 AM UTC, comment #2:

(Argh... that bug tracker markup stuff bit me again... sorry)
Again with with "verbatim":

Pulling the equation out of arrayfun(), it turns out that Octave-4.1.0+ on Windows also crashes on:

Narrowing further,

Philip Nienhuis <philipnienhuis>
Project Member
Fri 30 Oct 2015 08:31:54 AM UTC, comment #1:

The crash occurs also on Windows with Octave-4.1.0+, so I provisionally changed OS->Any, Release->dev.

But....
It's the statement "w = Inf" that makes the difference. Are you sure that it Inf is a proper value?
Substituting a large number (e.g., 1e16), make the script run:

so all in all very small output numbers.
These get a little bigger when substituting 1e10 or 1e5 for w, so I suppose you'd expect H{1} to become a zero matrix?

Pulling the equation out of arrayfun(), it turns out that Octave-4.1.0+ on Windows also crashes on:

c/(iInfe - a)*b

Narrowing further,

(i*Inf.e) - a => OK

c / (iInfe) => OK

c / (iInfe - a) => crash

so it doesn't look like an arrayfun() problem per se, but more a problem with manipulation of matrices containing Infs and Nans.

Title adapted a little, I don't know how to put it exactly.

Philip Nienhuis <philipnienhuis>
Project Member
Fri 30 Oct 2015 04:30:53 AM UTC, original submission:

The attached script file will produce a segfault:

Tested on a Mac running Octave 3.8.2. Other platforms and Octave 4.x needs to be tested.

Lukas Reichlin <paramaniac>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #35352:  patch_46330.txt added by lachlan (2KiB - text/plain - Workaround of crash)
file #35330:  bug_arrayfun.m added by paramaniac (958B - application/octet-stream)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by siko1056 (Updated the item)
  • -unavailable- added by lachlan (Updated the item)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by avinoam (Posted a comment)
  • -unavailable- added by paramaniac (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 25 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 16 Nov 2016 08:14:30 PM UTCavinoamStatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Thu 15 Sep 2016 05:03:42 AM UTCrik5StatusFixed=>Confirmed
      Open/ClosedClosed=>Open
      Summarysegfault with matrices containing Inf & NaN=>Fortran exception with matrices containing Inf & NaN
    Fri 15 Jul 2016 10:13:44 PM UTCrik5StatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
    Mon 11 Jul 2016 03:02:15 PM UTCrik5StatusPatch Submitted=>Ready For Test
    Sat 09 Jul 2016 11:00:09 PM UTCmtmillerStatusIn Progress=>Patch Submitted
    Fri 08 Jul 2016 03:30:00 PM UTCsiko1056Attached File-=>Added patch-46330-2016-07-08.txt, #37772
    Fri 08 Jul 2016 03:26:34 PM UTCsiko1056Attached File#37755=>Removed
    Fri 08 Jul 2016 01:12:03 PM UTClachlanStatusPatch Submitted=>In Progress
    Thu 07 Jul 2016 01:40:37 PM UTCsiko1056Attached File-=>Added patch-46330-2016-07-07.txt, #37755
    Thu 07 Jul 2016 01:40:11 PM UTCsiko1056Attached File#37718=>Removed
    Thu 07 Jul 2016 01:00:39 PM UTCavinoamStatusFixed=>Patch Submitted
      Open/ClosedClosed=>Open
    Thu 07 Jul 2016 08:39:54 AM UTCsiko1056Dependencies-=>bugs #48431 is dependent
    Wed 06 Jul 2016 12:57:59 PM UTCsiko1056StatusPatch Submitted=>Fixed
      Assigned toNone=>siko1056
      Open/ClosedOpen=>Closed
    Tue 05 Jul 2016 02:30:04 PM UTCsiko1056Attached File-=>Added patch-46330-2016-07-05.txt, #37718
    Tue 22 Dec 2015 03:26:29 PM UTCmtmillerCategoryNone=>Libraries
      Severity3 - Normal=>4 - Important
      StatusConfirmed=>Patch Submitted
    Sun 01 Nov 2015 08:04:14 AM UTClachlanAttached File-=>Added patch_46330.txt, #35352

    Back to the top


    Powered by Savane 3.1-cleanup1