bugGNU nano - Bugs: bug #52954, on a console, nano shows junk in...


You are not allowed to post comments on this tracker with your current authentication level.

bug #52954: on a console, nano shows junk in front of line numbers

Submitted by:  Benno Schulenberg <bens>
Submitted on:  Mon 22 Jan 2018 08:31:24 PM UTC  
Severity:  3 - Normal Status:  Fixed
Assigned to:  Benno Schulenberg <bens> Open/Closed:  Closed

( Jump to the original submission)

Sun 09 Feb 2020 10:20:28 AM UTC, comment #8: 

Released in 4.8.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Wed 15 Jan 2020 05:58:54 PM UTC, comment #7: 

A workaround has been added to master, commit f47ef539.

(Could it be that the VT needs to configured in some way in order to treat zero-width characters properly?  Because I find it weird that the Linux console is not able to handle combining accents.  How long do we now have general support for UTF-8 in just about all tools?  But not on the Linux console?)

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Wed 15 Jan 2020 04:23:45 PM UTC, comment #6: 

The thing is that a VT apparently simply cannot cope with zero-width characters, not even with well-known combining accents.  Open for example a new buffer, type "ete", and then, with the cursor after each e, type the combining acute accent: M-V 000301.  On a terminal emulator, it displays fine as "été".  But on a VT, it gets displayed as "e♦te♦".  And when you move the cursor over this line from start to end, the t gets 'copied' to one place earlier, resulting in the display of "ette♦", which is even more wrong.  Typing ^L then restores the original display.

So... it seems that the solution would be to just skip any zero-width character when running on a Linux console.  That would get rid of any junk in front of line numbers, and would skip any combining accents that the console is not able to display anyway.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Tue 17 Jul 2018 07:56:56 PM UTC, comment #5: 

That's good to know.  Although if the problem no longer occurs on X terminals, but still occurs on the console, maybe that means the real problem is on the console in some way?

David Lawrence Ramsey <dolorous>
Project Member
Sun 15 Jul 2018 07:54:48 AM UTC, comment #4: 

With more recent versions, the problem with the soft hyphen on X seems to have disappeared.  With Xfce-Terminal, libvte 0.52.2, and ncurses 6.1, the dollar sign in front of line 281 and the shifting of "jrks" no longer occurs here.

But the putting of junk in front of line numbers on the console is still just as bad.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Thu 08 Feb 2018 07:19:09 PM UTC, comment #3: 

Inserting a soft-hyphen between an a and a b causing the b to "disappear"... yes, that is a manifestation of the same problem: the terminal and ncurses disagree about the width of the displayed character.

(Sure, it happens also with files other than binary.  It's just that binary files contain a wide range of possible and impossible characters, something one doesn't find in any normal text file present on the average computer.  So, a binary file is a nice test.)

The problem is not limited to the soft hyphen.  On the console, many characters cannot be displayed -- the font apparently does not contain any glyph for them -- so it gets replaced by a diamond, and this is problematic.  Because when this happens for a zero-width character... the rest of the line gets displayed one column too far, and thus the junk problem ensues.

An example of a zero-width character that becomes a diamond on my Linux console: U+06E2.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Tue 06 Feb 2018 05:25:02 PM UTC, comment #2: 

Addendum: There are a few bits I forgot to mention:

In step 5, pressing Ctrl-L to refresh the screen will redisplay the "b", but the cursor will be shifted left and placed on a new blank character between "a" and "b".  This behavior takes place on Xfce4-terminal (and KDE5 Konsole behaves the same way).

xterm behaves slightly differently: after step 4, there will be a "-" displayed where "b" was instead of a blank space (so "b" won't seem to disappear, but will seem to be replaced by a hyphen).  urxvt also behaves differently: after step 4, "b" is still displayed at the same place, but a byphen is overlaid over the "a".  So part of this seems to be terminal-dependent.

David Lawrence Ramsey <dolorous>
Project Member
Tue 06 Feb 2018 04:22:50 PM UTC, comment #1: 

The problems that nano has displaying the Unicode soft hyphen aren't limited to just the console, line numbering mode, or binary files.  With either nano 2.9.3 or current git (d865d7ac), try the following:

1. Assuming the binary is in the current directory, run:

./nano --ignore

2. In the new buffer, type:


3. Press <Left> to move the cursor back on top of the "b".

4. Use verbatim input mode (Meta-V 0 0 0 0 A D) to insert a soft hyphen there.  Note that the cursor will remain in the same place, while the "b" will seem to disappear.

5. Press Ctrl-L to refresh the screen.  The "b" will be displayed again, this time one character to the right of the cursor.

If this is a different manifestation of the same problem, maybe the subject of this bug should be adjusted?

David Lawrence Ramsey <dolorous>
Project Member
Mon 22 Jan 2018 08:31:24 PM UTC, original submission:  

To reproduce, switch to a Linux console (for example with Ctrl+Alt+F2) and then run:

    nano --linenum  nano-2.9.2.tar.xz

(yes, we're going to look at a binary file full of "random" junk).

Then type <PageDown> a few times, and see how in front of many line numbers some seemingly random character appear -- mostly a dollar sign ($), sometimes another character followed by a dollar sign.

When running the same command on a terminal emulator running on X, the same problem does not happen.  At least, so it seems.  But... press <PageDown> until line 280 is onscreen.  See how there is dollar sign in front of line 281.  Now move the cursor down to line 280, then press <Right> eight times, to put the cursor on the space between "2" and "jrks".  Now press <Right> again.  The cursor does not move -- apparently there is a zero-width character there.  Press <Right> again -- note that "j" under the cursor gets 'copied' to the left.  The same 'copying' happens for the next three <Right>s.  Now press ^L -- the "jrks" gets moved one position to the right again.

Nano is confused about how wide this invisible, zero-width character is.  After a ^L, it gets displayed as a space; but when you move the cursor over it, it disappears.  Maybe there is a discrepancy between what mbwidth() says and how wide ncurses thinks the character is, or how the terminal shows it when it is fed that character?

The relevant character in this case is U+00AD (0xC2 0xAD), a soft hyphen.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.


No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by dolorous (Posted a comment)
  • -email is unavailable- added by bens (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-02-09 bens Open/ClosedOpen => Closed
    2020-01-15 bens StatusIn Progress => Fixed
    2020-01-15 bens StatusNone => In Progress
        Assigned toNone => bens

    Back to the top

    Powered by Savane 3.7