This is not great, but I don't think we should spend excessive programmer time on what amounts to a very small fraction of Octave running time (How much time to people really spend, in the debugger, and then within graphics system callbacks?).  If there is a relatively simple way to plug up the hole and stop segfaults that would be nice, but creating an entire edifice for instantiating multiple readline instances doesn't sound like a good use of time.

Pantxo, your patch from comment #13 is a change in the right direction but there is a simpler way.  In the catch block for the execution_exception, you can just write

catch (octave::execution_exception& e)
  {
    m_interpreter.handle_exception (e);
  }

There is a lot more going on in this bug.  The reason we see the error

 terminate called after throwing an instance of 'octave::quit_debug_exception'

is that in the debugger::repl function the quitting_debugger function throws that exception outside of any try/catch block that can catch it.  Fixing that by moving the quitting_debugger call inside the try/catch block that immediately follows the call leads to an infinite loop because the execution state remains unchanged.  Fixing that mostly works but leads to a strange error  message about "dbquit may only be used when debugging" when the user presses ENTER.  That's happening because we are making recursive calls to readline without saving and restoring readline's internal state.

I'm working on fixes for all these issues but it may take a little time.  With some luck, maybe I won't break anything else!

@Pantxo: Yes, you get the error message and the full stack trace:

>> testme
>> Unrecognized function or variable 'throw_an_error'.

Error in testme>clbk (line 7)
  throw_an_error ();

Error while evaluating UIControl Callback.

>> lasterr

ans =

    'Error while evaluating UIControl Callback.'

>> lasterror

ans =

  struct with fields:

       message: 'Error while evaluating UIControl Callback.'
    identifier: 'MATLAB:UndefinedFunction'
         stack: [0x1 struct]

Here's another way to demonstrate the problem.  Define the following function (can be at the command line):

function fff ()
  remove_input_event_hook ("fff");
  error ("foobar!");
endfunction

then execute

debug_on_error (true);
add_input_event_hook ("fff");

that should put you at a debug prompt.  Then enter a dbquit command and Octave will crash with

terminate called after throwing an instance of 'octave::quit_debug_exception'

I'm working on a fix so that quit_debug_exception is handled properly when it is thrown from a function that is executed in the readline event loop.

Yes sorry, the bisection in comment #4 was done using code from bug #57354 and pressing <return>. This is closer to bug #57244, which is fixed by the patch.

If run the code from comment #3 I observe the very same as described by Rik, including the strange "pseudo-debug mode" after dbcont or dbnext.

The attached change seems to fix the problem for me.  If it works for you and there are no additional problems I will push a proper changeset.


(file #48020)

As Guillaume said in bug #27544, <return> (instead of dbquit) is enough to trigger the crash.
I bisected and found that the revision that introduced this issue is

changeset:   27504:7a31b25e3252
user:        John W. Eaton <jwe@octave.org>
summary:     use shared_ptr for storing classdef and statement_list objects in parser

Adding jwe to the recepients of this report.

Ah, sorry, there was a typo in the formatting:
https://lists.gnu.org/archive/html/octave-bug-tracker/2019-11/msg00386.html

Debugging errors in callbacks is difficult as the call stack is not displayed
and only this error message appears:

execution error in graphics callback function

as, for example, here:

function testme
  F = figure ();
  uicontrol (F, 'String', 'Button', 'Callback', @clbk);
endfunction

function clbk (h, e)
  throw_an_error ();
endfunction

then if one use 'dbstop if error', click the button and type 'dbquit', there's a segfault.

Debugging errors in callbacks is difficult as the call stack is not displayed and only this error message appears:

execution error in graphics callback function
-verbatim+

as, for example, here:

+verbatim+
function testme
  F = figure ();
  uicontrol (F, 'String', 'Button', 'Callback', @clbk);
endfunction

function clbk (h, e)
  throw_an_error ();
endfunction

then if one use 'dbstop if error', click the button and type 'dbquit', there's a segfault.