bugGNU nano - Bugs: bug #48489, when a deletion creates a valid...


You are not allowed to post comments on this tracker with your current authentication level.

bug #48489: when a deletion creates a valid character from invalid bytes, nano should adapt

Submitted by:  Benno Schulenberg <bens>
Submitted on:  Tue 12 Jul 2016 07:42:20 PM UTC  
Severity:  2 - Minor Status:  Wont Fix
Assigned to:  None Open/Closed:  Closed

Sun 17 May 2020 12:23:48 PM UTC, comment #5: 

It's possible to fix this (see attached patch), but it's too much code for an issue that should normally never occur.

Furthermore, the cursor not moving when pressing the left or right arrow can also occur when dealing with combining characters.  It would be more important to get that working in a less surprising way.

(And if the user has configured their terminal so that they can enter raw, invalid bytes, well, they get to live with the problem they themselves have created.)

(file #49102)

Benno Schulenberg <bens>
Project Administrator
Thu 30 Mar 2017 03:59:19 PM UTC, comment #4: 

Specific example: Run rxvt-unicode 9.22, set URxvt.meta8 to True in your .Xdefaults file, and press Alt-whatever, and it will insert the raw byte with that value (instead of the UTF-8 character with that value as xterm does).

If you press Meta-V to activate verbatim input mode, and then press Ctrl-Alt-whatever (assuming, of course, that there's no rxvt-unicode function bound to that key combination), it will insert the raw byte with that value as well, even if it's a control character.

So it is definitely possible to add raw bytes via adding text as well as deleting it (although it might also be considered a bug in rxvt-unicode).

David Lawrence Ramsey <dolorous>
Project Member
Wed 29 Mar 2017 06:13:01 PM UTC, comment #3: 

Addendum: it's also possible to output a high-bit control character in do_output() via verbatim input mode if the terminal is configured to set the high bit when, say, Alt is held, so that case could also create a valid UTF-8 sequence.  So both input and output need to be covered by whatever fixes this bug.

David Lawrence Ramsey <dolorous>
Project Member
Wed 29 Mar 2017 03:51:33 PM UTC, comment #2: 

I just had a thought: This problem isn't just caused by character deletion; it can also be caused by character addition.

do_output() should filter out the bytes of a broken UTF-8 sequence if you try to type them in (they're high-bit control characters, after all).  But, if those bytes are already in the file, you get them into the cutbuffer via cutting or copying text, and then paste them into the middle of another broken sequence elsewhere in the file, you could possibly get a valid UTF-8 sequence as a result.

David Lawrence Ramsey <dolorous>
Project Member
Sat 25 Feb 2017 09:14:22 PM UTC, comment #1: 

Since I use constant cursor position display by default, I've noticed another problem with this: aside from making sure that we're not in the middle of a character after a deletion, the character count needs to be adjusted as well.

In your test case, the text in portoroz is originally read in as separate bytes, which increases the character count by one for each byte, for a total of 11 characters out of 11, as expected.

Then, after deleting the space between the two invalid bytes, the text is reinterpreted as a full UTF-8 sequence (which, despite being several bytes, is only one character).  However, the character count will only decrease by one, due to the deletion, for a total of 9 characters out of 10.

David Lawrence Ramsey <dolorous>
Project Member
Tue 12 Jul 2016 07:42:20 PM UTC, original submission:  

To reproduce, run 'echo "0000000: 506f 7274 6f72 6fc5 20be 2e" | xxd -r - >portoroz', and then run 'src/nano portoroz' in a UTF8 locale.  Move the cursor till it is between the two invalid bytes, then press Delete.  The valid character "ž" appears out of nowhere (because the two bytes of its code had artificially been separated by a space, which has now been deleted).  Note that the cursor now seems to be positioned on the final period, but... it is not, which you will see when you press Delete again.  Instead of deleting the period, the second byte of ž is deleted, making its first byte an invalid one again.

Press M-U to restore the two deleted bytes.  Now delete the space between the two invalid bytes again.  Then press the Right arrow.  The cursor does not appear to move -- because it has moved from the second byte of ž to the actual period.  Only a second strike of the Right arrow will move the cursor to after the period.

Expected behavior: nano should not have its x position in the middle of valid multibyte characters, but instead the x position should correspond to where the cursor appears to be.  So... probably for every type of deletion (and when x is not zero), nano should do: move a char left, move a char right -- just to make sure we're not in the middle of one.

Benno Schulenberg <bens>
Project Administrator


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by dolorous (Posted a comment)
  • -email is unavailable- added by bens (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-05-30 bens Open/ClosedOpen => Closed
    2020-05-17 bens Attached File- => Added 0001-editing-an-attempt-to-shy-away-from-the-middle-of-a-.patch, #49102
        StatusNone => Wont Fix

    Back to the top

    Powered by Savane 3.5