bugGNU Octave - Bugs: bug #51948, GUI fails to start when Address...

 
 

bug #51948: GUI fails to start when Address Sanitizer is compiled in

Submitted by:  Rik <rik5>
Submitted on:  Tue 05 Sep 2017 10:53:14 PM UTC  
 
Category: InterpreterSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Other
Status: FixedAssigned to: None
Originator Name: Open/Closed: Closed
Release: devOperating System: Any

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 14 Sep 2017 04:04:07 PM UTC, comment #3:

I fixed the problem in this cset (http://hg.savannah.gnu.org/hgweb/octave/rev/f50ef29d874e). This was a classic off-by-one error where the amount of memory allocated for the terminal was

but when the associated memory was used the calculation was

Rik <rik5>
Project Administrator
Tue 12 Sep 2017 05:03:14 AM UTC, comment #2:

Thanks for the pointer. I commented out three lines in QUnixTerminalImpl.cpp beginning at 264 and it now produces a stack trace.

The stack trace is

Rik <rik5>
Project Administrator
Wed 06 Sep 2017 02:10:39 AM UTC, comment #1:

I think the STDERR redirection is done in QUnixTerminalImpl.cpp (connectToPty()).

Torsten <ttl>
Project Member
Tue 05 Sep 2017 10:53:14 PM UTC, original submission:

When Octave is compiled with --enable-address-sanitizer-flags the GUI refuses to start. It was unclear why this was, but now I have a strong indication that it is due to a heap-buffer-overflow.

I configured the build environment like so

Using run-octave fails, so I cd'ed to the src directory and used

results were

You can see that ASAN starts to write out on file descriptor 2 (stderr). Later there is

which sure looks like it was writing "heap-buffer-overflow". Obviously stderr must have been redirected somewhere because I don't get these on the screen.

@jwe: Is there a way to temporarily halt the redirection of stderr while debugging this?

Rik <rik5>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by ttl (Posted a comment)
  • -unavailable- added by rik5 (Submitted the item)
  • -unavailable- added by rik5
  • -unavailable- added by rik5
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 14 Sep 2017 06:04:51 PM UTCrik5StatusConfirmed=>Fixed
      Open/ClosedOpen=>Closed
    Tue 12 Sep 2017 05:03:14 AM UTCrik5StatusNone=>Confirmed
    Tue 05 Sep 2017 10:53:14 PM UTCrik5Carbon-Copy-=>Added jwe
      Carbon-Copy-=>Added dasergatskov

    Back to the top


    Powered by Savane 3.1-cleanup1