bugGNU Octave - Bugs: bug #52146, CTRL+C and cursor up results in...

 
 

bug #52146: CTRL+C and cursor up results in segfault

Submitter:  Andreas Weber <andy1978>
Submitted:  Sat 30 Sep 2017 11:36:51 AM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 19 Apr 2018 04:45:09 PM UTC, comment #5: 

Ctrl+C and signal handling have been massively reworked in Octave now, and I can't reproduce this anymore with 4.4 release candidate. I can still reproduce with Octave 4.2.2. So I think this has been fixed.

Mike Miller <mtmiller>
Group Member
Sun 15 Oct 2017 11:21:49 PM UTC, comment #4: 

This works for me, but I have readline 6.3.8 installed.  I tried using LD_PRELOAD to get readline5 but I still don't see a segfault.

Rik <rik5>
Group administrator
Sat 30 Sep 2017 08:15:54 PM UTC, comment #3: 

I've put some fprintf()'s near all the add_history() and remove_history() functions.  I don't see any of those function calls associated with cntrl-C.  Given that using_history() is called at startup, I don't see what other influence on readline Octave would have.

I don't think that BLAS/OpenBLAS influences this, given no functions from that library are run in this example.  The fail example in https://savannah.gnu.org/bugs/?47400 runs some code that uses the BLAS library prior to the cntrl-C problem.

Point taken though about the multi-thread environment.  I'm thinking of ways that might manifest.  There is now an Octave signal handler intermediate, which if I understand correctly catches the signal, does some GUI related stuff (if such a function is defined), then rethrows the signal, which presumably gets passed on to library readline.  Perhaps something has been overlooked in the proper way of rethrowing a signal.

Another thought is that although there is no mandatory initialization for libreadline, there is an initialization routine:

https://tiswww.case.edu/php/chet/readline/readline.html#SEC39


Function: int rl_initialize (void)
    Initialize or re-initialize Readline's internal state. It's not strictly necessary to call this; readline() calls it before reading any input.


That's interesting in the sense that typing up-arrow is not "reading any input".  Is readline library internally calling rl_initialize() in such a scenario?

Ah, but the crash is not consistent, but intermittent.  That's typical of multi-threaded race conditions.  Is there some conceivable way in which rl_initialization() is run in the GUI thread (where the program launch and library install occurred), while the up-arrow action is run in the worker thread (where Octave core was moved to and is using readline)?  If so, that could be a race between rl_initialization() completing and up-arrow accessing the history list (which is bogus because without the .octave_hist nothing is put into the history list).

I wonder if we were to force rl_initialization() at the start of the program (I don't currently see it happening at startup) that could solve the problem.  Could you try the attached patch?  If it fixes the issue, we could have JWE find a better location for the rl_initialize() call.

(file #42021)

Dan Sebald <sebald>
Sat 30 Sep 2017 05:03:27 PM UTC, comment #2: 

I can't repeat that here, but that doesn't mean this isn't an Octave bug.

The error message you are seeing originates from GNU readline, so I doubt you would have a chance to use the debugger to much effect unless readline were built with debug info.  Since I can't replicate the crash and get the message, I'm not 100% sure which readline is being used on my system (5.2 or 6.3):


sebald@ ~ $ ls /lib/x86_64-linux-gnu/libreadline.so*
/lib/x86_64-linux-gnu/libreadline.so.5
/lib/x86_64-linux-gnu/libreadline.so.5.2
/lib/x86_64-linux-gnu/libreadline.so.6
/lib/x86_64-linux-gnu/libreadline.so.6.3


If Octave ends up using 6.3, perhaps that is more robust to this situation.

My thinking is this could be a simple one or two line change.

First hypothesis
----------------
The readline library needs to be initialized with using_history():

https://tiswww.case.edu/php/chet/readline/history.html#SEC10

and one line of thought is that without an .octave_hist file present, perhaps Octave is inadvertently skipping the call to using_history() such that readline's innards aren't properly initialized when the up-arrow initiates a fetch from readline's symbol list.

However, it's strange that only such an error would occur after a Cntrl-C rather than an up-arrow immediately upon launch.  What is it about Cntrl-C that would affect the history list internal to GNU readline?  Is Cntrl-C removing a command from the list?

OK, I don't think it is an initialization issue.  I've printed out a comment at every location there is a using_history() [why it appears so often, I don't know]:


sebald@ ~ $ rm .octave_hist
sebald@ ~ $ /usr/local/src/octave/octave/build1/run-octave --no-gui
[snip]
using_history #5
octave:1>


So, using_history() does get called.

Second hypothesis
-----------------
Given that the error is occurring in readline, it must be readline's use of history, for which prev-history (up arrow) is described here

http://cnswww.cns.cwru.edu/php/chet/readline/readline.html#SEC15

Could it be that SIGINT handling is doing something strange?

http://cnswww.cns.cwru.edu/php/chet/readline/readline.html#SEC44

"
When a SIGINT is received, the Readline signal handler performs some additional work, which will cause any partially-entered line to be aborted (see the description of rl_free_line_state() below).
"

That would seem like a bug in library readline, wouldn't it?

Could you try the following?


rm .octave_hist
./run-octave --no-gu
typesometext
<cntrl-C>
<up-arrow>


Are you ever seeing a crash in that case?

Dan Sebald <sebald>
Sat 30 Sep 2017 05:00:38 PM UTC, comment #1: 

Confirmed, both with --no-gui and with --force-gui. Does not affect the actual octave-cli. Only happens when using OpenBLAS, so this may be another manifestation of bug #47400. Can you confirm that this goes away if you link with BLAS or run with OMP_NUM_THREADS=1?

Mike Miller <mtmiller>
Group Member
Sat 30 Sep 2017 11:36:51 AM UTC, original submission:  

default branch, hg id 80906ac05e7d

with empty history (rm ~/.octave_hist) start CLI with

./run-octave --no-gui


then press CTTRL+C, then cursor up (which should give the last line of history) results sometimes in a segfault, sometimes it shows


symbol lookup error: /lib/x86_64-linux-gnu/libreadline.so.5: undefined symbol: history_list


I tried to debug this with getpid and attaching gdb but as soon as there is a command executed (getpid in this case) it doesn't crash.

Andreas Weber <andy1978>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by sebald (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by andy1978 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-04-19 mtmiller StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2017-10-15 rik5 CategoryNone Interpreter
    2017-09-30 sebald Attached File- Added octave-readline_initializeation-djs2017sep30.patch, #42021
    2017-09-30 mtmiller StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code