bugGNU nano - Bugs: bug #48489, when a deletion creates a valid...

 
 

bug #48489: when a deletion creates a valid character from invalid bytes, nano should adapt

Submitted by:  Benno Schulenberg <bens>
Submitted on:  Tue 12 Jul 2016 07:42:20 PM UTC  
 
Severity: 2 - MinorStatus: None
Assigned to: NoneOpen/Closed: Open

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 30 Mar 2017 03:59:19 PM UTC, comment #4:

Specific example: Run rxvt-unicode 9.22, set URxvt.meta8 to True in your .Xdefaults file, and press Alt-whatever, and it will insert the raw byte with that value (instead of the UTF-8 character with that value as xterm does).

If you press Meta-V to activate verbatim input mode, and then press Ctrl-Alt-whatever (assuming, of course, that there's no rxvt-unicode function bound to that key combination), it will insert the raw byte with that value as well, even if it's a control character.

So it is definitely possible to add raw bytes via adding text as well as deleting it (although it might also be considered a bug in rxvt-unicode).

David Lawrence Ramsey <dolorous>
Project Administrator
Wed 29 Mar 2017 06:13:01 PM UTC, comment #3:

Addendum: it's also possible to output a high-bit control character in do_output() via verbatim input mode if the terminal is configured to set the high bit when, say, Alt is held, so that case could also create a valid UTF-8 sequence. So both input and output need to be covered by whatever fixes this bug.

David Lawrence Ramsey <dolorous>
Project Administrator
Wed 29 Mar 2017 03:51:33 PM UTC, comment #2:

I just had a thought: This problem isn't just caused by character deletion; it can also be caused by character addition.

do_output() should filter out the bytes of a broken UTF-8 sequence if you try to type them in (they're high-bit control characters, after all). But, if those bytes are already in the file, you get them into the cutbuffer via cutting or copying text, and then paste them into the middle of another broken sequence elsewhere in the file, you could possibly get a valid UTF-8 sequence as a result.

David Lawrence Ramsey <dolorous>
Project Administrator
Sat 25 Feb 2017 09:14:22 PM UTC, comment #1:

Since I use constant cursor position display by default, I've noticed another problem with this: aside from making sure that we're not in the middle of a character after a deletion, the character count needs to be adjusted as well.

In your test case, the text in portoroz is originally read in as separate bytes, which increases the character count by one for each byte, for a total of 11 characters out of 11, as expected.

Then, after deleting the space between the two invalid bytes, the text is reinterpreted as a full UTF-8 sequence (which, despite being several bytes, is only one character). However, the character count will only decrease by one, due to the deletion, for a total of 9 characters out of 10.

David Lawrence Ramsey <dolorous>
Project Administrator
Tue 12 Jul 2016 07:42:20 PM UTC, original submission:

To reproduce, run 'echo "0000000: 506f 7274 6f72 6fc5 20be 2e" | xxd -r - >portoroz', and then run 'src/nano portoroz' in a UTF8 locale. Move the cursor till it is between the two invalid bytes, then press Delete. The valid character "ž" appears out of nowhere (because the two bytes of its code had artificially been separated by a space, which has now been deleted). Note that the cursor now seems to be positioned on the final period, but... it is not, which you will see when you press Delete again. Instead of deleting the period, the second byte of ž is deleted, making its first byte an invalid one again.

Press M-U to restore the two deleted bytes. Now delete the space between the two invalid bytes again. Then press the Right arrow. The cursor does not appear to move -- because it has moved from the second byte of ž to the actual period. Only a second strike of the Right arrow will move the cursor to after the period.

Expected behavior: nano should not have its x position in the middle of valid multibyte characters, but instead the x position should correspond to where the cursor appears to be. So... probably for every type of deletion (and when x is not zero), nano should do: move a char left, move a char right -- just to make sure we're not in the middle of one.

Benno Schulenberg <bens>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by dolorous (Posted a comment)
  • -unavailable- added by bens (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    No Changes Have Been Made to This Item

    Back to the top


    Powered by Savane 3.1-cleanup1