bugGNU Octave - Bugs: bug #42136, Unicode box drawing chars in GUI...

 
 

bug #42136: Unicode box drawing chars in GUI don't line up very well

Submitter:  None
Submitted:  Thu 17 Apr 2014 03:54:16 PM UTC
   
 
Category:  GUI Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * dev
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 10 Jan 2020 10:15:35 AM UTC, comment #36: 

I have tested this and it works great!  Thank you.

Colin Macdonald <cbm>
Sat 14 Dec 2019 12:40:42 PM UTC, comment #35: 

After checking again using Colin's scripts that the box drawing characters display with the patch, I went ahead and applied it here:
http://hg.savannah.gnu.org/hgweb/octave/rev/ba4271934b1c

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Sat 20 Jul 2019 12:00:55 PM UTC, comment #34: 

With jwe working on a potential new implementation of the terminal widget, I went ahead and took the route of least effort. As Mike suggested, I completely removed the code that was handling the special case for box drawing characters.
With my font settings, the results don't look as good as what Andrew showed in comment #33. But no glyphs are missing at least.

Please, see the attached patch.

(file #47243)

Markus Mützel <mmuetzel>
Group administrator
Sat 23 Mar 2019 08:30:41 PM UTC, comment #33: 

The code might still be there, but the behavior looks a lot better with the current upstream QTermWidget code. Here's what I get when running it in Octave 4.4.1 inside the current development qterminal 0.14.1: . The diagonal lines appear though they look a bit raggedy.

I'll still look at putting together an upstream bug report, but now it looks more like a minor cosmetic issue.

Andrew Janke <apjanke>
Fri 22 Mar 2019 06:23:59 AM UTC, comment #32: 

It looks to me like it's still in qtermwidget. I haven't tested, so please go ahead and confirm independently. But in Octave, it's a variable called LineChars and a function called drawLineChar in TerminalView.cpp. That function is still in qtermwidget master:

https://github.com/lxqt/qtermwidget/blob/master/lib/TerminalDisplay.cpp#L497

Instead of letting the font do the work, it's trying to draw unicode box characters itself, but ignores any characters with angles or curves.

Mike Miller <mtmiller>
Group Member
Fri 22 Mar 2019 04:17:19 AM UTC, comment #31: 

Oh crap, you're right. Those diagonal lines are so completely gone that I didn't even notice them missing.

Here's what the same code looks like under CLI Octave 4.4.1 in iTerm 3.2.7 on the same Mac box: .

That is not good.

I'll run this in a recent upstream standalone qterminal build and see what happens.

Andrew Janke <apjanke>
Fri 22 Mar 2019 04:06:54 AM UTC, comment #30: 

Your screengrab looks almost exactly like mine, good.

The important bug is the diagonal lines missing. If you didn't look at the files in a different terminal or text editor you may not have even noticed, but 's2' is supposed to be "sqrt(sin(x/2))", the line drawing that makes up the V-shaped part of the square root sign is completely invisible in qterminal.

Mike Miller <mtmiller>
Group Member
Fri 22 Mar 2019 02:53:07 AM UTC, comment #29: 

So, here's what I see running Octave.app 5.1.0 with Qt 5.12.2 on macOS 10.14.3.



The horizontal alignment is a little off, but I don't see the weird substitution effects that other posters are talking about.

I don't really know what the Unicode font situation is on Linux and Windows these days. But my gut is that I agree with you, and that this whole feature is left over from the olden days, but now it's not necessary because fonts probably cover it, and it's undesirable in some cases because it makes bad graphics on high-resolution displays.

(It's bugs like these that make me pine for GitHub's in-line image attachment display. I wonder if anyone's working on Savane these days?)

Andrew Janke <apjanke>
Fri 22 Mar 2019 02:16:58 AM UTC, comment #28: 

Basically it's whichever 'python' is first in your PATH by default. But the PYTHON environment variable is supported, so you can set that to a different name or to a full path to the executable if you need to.

If you can add notes to https://github.com/cbm755/octsympy/wiki/Notes-on-macOS-installation, I think that would be most welcome.

The fastest way to see this bug is to just use file #31199, file #31200, and file #31201 Colin attached here, they're just copy-pasted unicode text originally from SymPy.

Mike Miller <mtmiller>
Group Member
Fri 22 Mar 2019 02:00:15 AM UTC, comment #27: 

Looks interesting.

Can someone tell me how to get SymPy set up on Python on macOS so that Octave can see it, so I can get the symbolic package working? I have a bunch of Python environments on my machine, and I don't know which one Octave is using or how Octave and Python interact.

Andrew Janke <apjanke>
Thu 21 Mar 2019 11:39:33 PM UTC, comment #26: 

@apjanke if you're looking at qterminal stuff, take a look at this too.  Basically instead of using fonts, it intercepts certain "box drawing characters" and draw one-pixel lines intead.  This already looks bad but its also missing diagonal lines.

Its easy to disable (see my patch); worth looking at w.r.t. upstream too?

Colin Macdonald <cbm>
Thu 14 Mar 2019 10:21:02 PM UTC, comment #25: 

I agree with all that. I only assume that this special code for box drawing characters was done in the first place because of a lack of widely available font coverage at the time, and maybe is no longer needed. But yeah, only guessing that it can be safely removed, and suggesting we try it, see if it breaks anything, and then contribute that simplification upstream.

Mike Miller <mtmiller>
Group Member
Mon 11 Mar 2019 07:53:57 AM UTC, comment #24: 

My last comment was probably ambiguous. Trying again:
I don't know what might break if we removed that code because I don't have the background to know why it was introduced in the first place.
The respective code upstream is different from what we have in our fork. I don't know if the issue we are seeing is already fixed upstream.
We might want to try merging in the upstream changes. But I don't know how difficult that would be.

Markus Mützel <mmuetzel>
Group administrator
Sun 10 Mar 2019 09:45:38 PM UTC, comment #23: 

Let's fix that in our copy of qterminal, and submit a PR to qtermwidget to remove the feature there too.

I forget where I saw this, but there was some interest in getting Octave to work with the upstream qtermwidget library. We should help improve qtermwidget if we can, and if they can get it compatible with Octave before someone gets around to reimplement the command window, that would be a good outcome too.

Mike Miller <mtmiller>
Group Member
Sun 10 Mar 2019 02:18:49 PM UTC, comment #22: 

I also don't know why the box drawing characters aren't taken from the font directly and what could possibly break if we removed that logic. It looks like we inherited that code from QTermWidget when it was forked in 2012. The code in question is still there in current QTermWidget:
https://github.com/lxqt/qtermwidget/blob/master/lib/TerminalDisplay.cpp

I don't know if it is worth trying to merge changes from there, given that our fork deviated considerably (at least the file names no longer match) and there are plans to re-write the command window widget anyway.

How should we proceed?

Markus Mützel <mmuetzel>
Group administrator
Sun 10 Mar 2019 06:06:11 AM UTC, comment #21: 

I attached a couple screenshots.  At least for me, the fonts are much better.  Surely this is some 1990s cruft?


Colin Macdonald <cbm>
Sun 10 Mar 2019 06:00:51 AM UTC, comment #20: 

Aha!  That explains why when I increase the font size, the lines of box drawing chars alawys stay 1-pixel thin.  This isn't a nice behaviour for high-dpi screens...  Upstream active?

I did this:

diff -r b4a9888bb3c9 libgui/qterminal/libqterminal/unix/TerminalView.cpp
--- a/libgui/qterminal/libqterminal/unix/TerminalView.cpp       Wed Mar 06 12:41:14 2019 -0800
+++ b/libgui/qterminal/libqterminal/unix/TerminalView.cpp       Sat Mar 09 22:00:07 2019 -0800
@@ -142,7 +142,7 @@
    QCodec.
 */

-static inline bool isLineChar(quint16 c) { return ((c & 0xFF80) == 0x2500);}
+static inline bool isLineChar(quint16 c) { return (false);}
 static inline bool isLineCharString(const QString& string)
 {
   return (string.length() > 0) && (isLineChar(string.at(0).unicode()));

And now box drawing looks fine on my system...

Colin Macdonald <cbm>
Fri 08 Mar 2019 06:35:26 PM UTC, comment #19: 

@mmuetzel I don't see any hint of hair line on my system. I've now tried a few different font sizes in the command window settings and I still see nothing.

I just looked into our copy of qterminal, and the unix variant has its own custom code relevant to drawing the Unicode box drawing characters. LineFont.h defines 128 bit-packed values encoding how to draw these characters. And it turns out that U+256D through U+2573 are blank, so they are intentionally not rendered. This code looks like it only handles the box drawing characters made up of horizontal and vertical segments only.

Mike Miller <mtmiller>
Group Member
Fri 08 Mar 2019 07:59:45 AM UTC, comment #18: 

@Mike: That was what I was thinking.
Looking at the console screen shot again, it looks like the slashes are very fine hair lines. Could this be a case of anti-aliasing and/or re-scaling gone wrong?

Markus Mützel <mmuetzel>
Group administrator
Tue 05 Mar 2019 08:13:46 PM UTC, comment #17: 

Yeah the invisible slash characters seems like a bigger problem than alignment. The characters are there, they are just rendered completely invisible.


Mike Miller <mtmiller>
Group Member
Mon 04 Mar 2019 06:45:00 PM UTC, comment #16: 

Perhaps the thing worth worrying about here is why the slash characters are missing...   Should I file a separate issue just about that?

@mmuetzel: thanks for the tip about how to use _mfile_encoding_

Colin Macdonald <cbm>
Mon 04 Mar 2019 04:27:39 PM UTC, comment #15: 

@cbm I see about the same on my Debian system as on your Fedora, slightly worse alignment on the line with the 'e', slightly more space between rows so vertical bars look more like dashed lines, and yes the slashes are completely missing. All of which is much better in a terminal, except for the space between rows.

@mmuetzel If you're referencing the missing slash symbols, it's not the font, it's the same behavior in the Octave command window (on GNU/Linux) with multiple fonts, all of which do show the characters when the same unicode text is displayed in a terminal or another text editor, even in the Octave editor.

Mike Miller <mtmiller>
Group Member
Mon 04 Mar 2019 03:59:07 PM UTC, comment #14: 

The Linux and Windows command window use different fonts.
It looks like the brackets aren't correctly monospaced with the font used in Windows. Hence the misalignment. That is a problem with the font and there is probably little we can do in Octave.
Some symbols might be missing because the glyphs aren't present in the font.

If you want to use UTF-8 encoded files on Windows, you first have to switch the mfile encoding (which is system locale dependent by default):

__mfile_encoding__ utf-8
clear functions


After this, unicode_align2 should be displaying better.

Markus Mützel <mmuetzel>
Group administrator
Mon 04 Mar 2019 03:32:00 PM UTC, comment #13: 

Nice to see the good results usig a terminal app.

I think this will only be truly resolved when Octave replaces the GUI console window code which has been discussed.

Rik <rik5>
Group administrator
Mon 04 Mar 2019 07:11:55 AM UTC, comment #12: 

I also tested on Konsole (KDE Terminal App) and the slashes and alignment look fine.  So it might be our problem not QTs...


Colin Macdonald <cbm>
Mon 04 Mar 2019 07:05:23 AM UTC, comment #11: 

I tested on Fedora 29, see attached screenshot.

  * alignment is much improved.
  * sqrt slash characters are still missing :(

I also tested on Win 10: traditionally unicode stuff is broken there it seems this is improving!

  * the alignment is poor.
  * sqrt slash chars are present.

The missing slash characters is rather serious for Symbolic.  Maybe GNU/Linux users mostly don't use the GUI...



Colin Macdonald <cbm>
Sun 03 Mar 2019 06:29:30 PM UTC, comment #10: 

The last comment on this bug report was 3 years ago.  Could someone try to reproduce this on the new 5.1.0 release?

Rik <rik5>
Group administrator
Tue 01 Dec 2015 08:58:45 AM UTC, comment #9: 

Ok, tested on a clean user account and underscores seem fine.  I re-read the thread: it seems the slashes that make up unicode-art sqrts never worked.

So in summary, this bug is still valid, and to me seems unchanged since 3.8.1.

Colin Macdonald <cbm>
Fri 27 Nov 2015 10:09:14 PM UTC, comment #8: 

Hmm, if anything its worse!  On Octave 4.0.0, sqrt signs (the slashes) are completely missing.  Come to notice it, so are plain ol' underscores (so some low ascii problems).

I've tested with a couple different fonts, but not yet tested on a clean user account.

I've upgraded my OS (Fedora 23 now) since I originally filed this.

Colin Macdonald <cbm>
Wed 25 Nov 2015 09:38:46 PM UTC, comment #7: 

This bug is 1.5 years old and filed against the GUI in 3.8.1.  So much has changed since then, is this report still relevant or can it be closed?

Rik <rik5>
Group administrator
Fri 18 Apr 2014 12:56:43 PM UTC, comment #6: 

Changeset
http://hg.savannah.gnu.org/hgweb/octave/rev/5fb180e37d7c
fixes the issue for some fonts. For other fonts (like Inconsolata) the issue still shows up. However, the incorrect result for those fonts can also be seen in other applications when pasting the text there (e.g. geany or gedit).

The gui terminal does not show the beginning of the square root whereas the result in the gnome terminal is correct.

Torsten Lilge <ttl>
Group Member
Fri 18 Apr 2014 06:12:21 AM UTC, comment #5: 

Hmmm, at least my browser (Firefox 28) also has a small problem with the e of exp(x).

But OTOH, the sqrt looks much better here than in my terminals (or Octave GUI).

So perhaps we don't think we need perfection here, just to do better than in my original screenshots.

I read somewhere (can probably find a reference if needed) but I think perfect alignment is not really possible when using certain types of hinting.  But at least with monospace fonts, I'd expect errors of no large than 1/3 pixel.

My guess is the exp(x) term is a font-substitution issue, given that I see it in Firefox, Emacs and Konsole.  But the other alignment problems are not apparent in those software.

Colin Macdonald <cbm>
Fri 18 Apr 2014 06:03:47 AM UTC, comment #4: 

Yes I tried different fonts, see gui_bad2.png in the original set.

I've attached the samples.  all are utf-8 except unicode_align3.m which is the strings converted to double so is just ascii.

I'll try to paste into a verbatim block too:



⎡            l          x   ⎤
⎢  1         ─         ℯ    ⎥
⎢            2              ⎥
⎢                           ⎥
⎢   ⎛x⎞     ⎛12⋅x⎞     ⎛ x ⎞⎥
⎢sin⎜─⎟  cos⎜────⎟  sin⎜───⎟⎥
⎣   ⎝2⎠     ⎝ 17 ⎠     ⎝121⎠⎦

    ________
   ╱    ⎛x⎞
  ╱  sin⎜─⎟
╲╱      ⎝2⎠



(file #31199, file #31200, file #31201)

Colin Macdonald <cbm>
Thu 17 Apr 2014 10:03:11 PM UTC, comment #3: 

That was my first thought too, if memory serves me right bug #37187 was related to fractional offsets from moving the cursor back over already entered text.

Colin, can you post one of your examples in text form or attach as an m-file in UTF-8 so someone can load it and reproduce in Octave on our end? That would help in determining if this bug is reproducible and whether it's related to this other font spacing bug.

Also have you tried different fonts? I know you say this font aligns perfectly in gnome-terminal and not in konsole, what about other fixed-width fonts?

Mike Miller <mtmiller>
Group Member
Thu 17 Apr 2014 09:47:23 PM UTC, comment #2: 

This issue seems related to bug #37187. I am going to add a patch for it within the next days.

Torsten Lilge <ttl>
Group Member
Thu 17 Apr 2014 03:59:48 PM UTC, comment #1: 

I installed Konsole, and it also looks a bit poor.

Perhaps its just QT in general?  Looks a bit look not our fault, but still, I wonder if there is a workaround?

(I'd quite to enable this unicode output by default the octsympy symbolic package.)



Colin Macdonald <cbm>
Thu 17 Apr 2014 03:54:16 PM UTC, original submission:  

Drawing boxes with unicode that look fine when running in Gnome-Terminal, have minor (several pixel) alignment problems in the new GUI.

I'll attach some screenshots.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #47243:  bug42136_box_drawing_chars.patch added by mmuetzel (10KiB - application/octet-stream)
file #46411:  konsole_2019_fedora29.png added by cbm (47KiB - image/png - 2019-03 updates)
file #46409:  gui_2019_fedora29.png added by cbm (107KiB - image/png - 2019-03 updates)
file #46410:  gui_2019_win10_bad.png added by cbm (52KiB - image/png - 2019-03 updates)
file #31199:  unicode_align1.txt added by cbm (370B - text/plain)
file #31200:  unicode_align3.m added by cbm (1KiB - text/x-objcsrc)
file #31201:  unicode_align2.m added by cbm (627B - text/x-objcsrc)
file #31198:  konsole_also_bad.png added by cbm (72KiB - image/png - konsole (kde terminal) output, also bad)
file #31197:  gui_bad2.png added by None (119KiB - image/png - Not just the font, b/c gui_bad2.png has the same font as my gnome-terminal)
file #31196:  gui_bad1.png added by None (107KiB - image/png - Not just the font, b/c gui_bad2.png has the same font as my gnome-terminal)
file #31195:  gui_bad1_font.png added by None (32KiB - image/png - Not just the font, b/c gui_bad2.png has the same font as my gnome-terminal)
file #31194:  gnometerm-good.png added by None (50KiB - image/png - Not just the font, b/c gui_bad2.png has the same font as my gnome-terminal)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by cbm
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by ttl (Posted a comment)
  • -email is unavailable- added by cbm (Updated the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-01-04 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2019-12-14 mmuetzel StatusPatch Submitted Ready For Test
    2019-07-20 mmuetzel Attached File- Added bug42136_box_drawing_chars.patch, #47243
        StatusConfirmed Patch Submitted
    2019-03-23 apjanke Attached File- Added box drawing on current dev QTerminal 0.14.1 2019-03-23.png, #46630
    2019-03-22 rik5 Carbon-CopyRemoved 72865 -
    2019-03-22 apjanke Attached File- Added box drawing on 5.1.0 on macOS 10.14 in iTerm2 CLI - apj angharad.png, #46609
    2019-03-22 apjanke Attached File- Added box drawing on 5.1.0 on macOS 10.14- apj angharad.png, #46608
    2019-03-21 cbm Carbon-Copy- Added apjanke
    2019-03-10 cbm Attached File- Added unpatched_crazy_nonfont_boxdrawing.png, #46486
        Attached File- Added patched_to_just_use_fonts.png, #46487
    2019-03-04 cbm Attached File- Added konsole_2019_fedora29.png, #46411
    2019-03-04 cbm Attached File- Added gui_2019_fedora29.png, #46409
        Attached File- Added gui_2019_win10_bad.png, #46410
    2019-03-03 mtmiller Carbon-CopyRemoved 80942 -
    2015-12-08 rik5 StatusNeed Info Confirmed
        Release3.8.1 dev
    2014-04-18 cbm Attached File- Added unicode_align1.txt, #31199
        Attached File- Added unicode_align3.m, #31200
        Attached File- Added unicode_align2.m, #31201
    2014-04-17 mtmiller Item GroupNone Incorrect Result
        StatusNone Need Info
    2014-04-17 cbm Attached File- Added konsole_also_bad.png, #31198
    2014-04-17 None Attached File- Added gnometerm-good.png, #31194

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code