bugGNU nano - Bugs: bug #50403, statusbar character count should...

 
 

bug #50403: statusbar character count should count chars/codepoints not bytes

Submitted by:  David Lawrence Ramsey <dolorous>
Submitted on:  Sat 25 Feb 2017 09:27:30 PM UTC  
 
Severity: 3 - NormalStatus: Fixed
Assigned to: Benno Schulenberg <bens>Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 28 Feb 2017 03:49:20 PM UTC, comment #8:

Fixed in git, 91910b23.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Mon 27 Feb 2017 10:48:47 PM UTC, comment #7:

That patch works too.

David Lawrence Ramsey <dolorous>
Project Administrator
Mon 27 Feb 2017 08:23:57 PM UTC, comment #6:

Yes, that seems to work. See attached patch.

(file #39841)

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Mon 27 Feb 2017 08:18:18 PM UTC, comment #5:

The offending commit is 08c51cfd. So... this suggests to me that the solution is to not return a count of 1, but to return a count of 2 or 3 or 4. (I haven't tested this yet, but things worked before this commit, so...). Looking into it...

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Sun 26 Feb 2017 06:23:15 PM UTC, comment #4:

The attached patch should fix it. It's not that Unicode input only returns the first byte, it's that verbatim input outputs whatever it gets byte by byte, which is fine for everything except Unicode input, which needs the whole sequence read at once.

(file #39835)

David Lawrence Ramsey <dolorous>
Project Administrator
Sun 26 Feb 2017 11:46:41 AM UTC, comment #3:

Bug exists since nano-2.6.3.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Sun 26 Feb 2017 09:59:02 AM UTC, comment #2:

It works fine when typing Unicode characters directly. For example, when typing «ŝámpú» (the chevrons included) the character count is 7 -- which you can verify by copy/pasting the text.

Nice find. So Unicode Input should return not just the first byte, but all of the bytes of the codepoint. If you want to attempt a patch... I am busy elsewhere.

Benno Schulenberg <bens>
Project AdministratorIn charge of this item.
Sat 25 Feb 2017 09:48:59 PM UTC, comment #1:

Addendum: I think this problem may also apply to typing in UTF-8 characters normally via the keyboard, instead of only in verbatim input mode, but with my keyboard and default settings, I have no way to test that at the moment.

David Lawrence Ramsey <dolorous>
Project Administrator
Sat 25 Feb 2017 09:27:30 PM UTC, original submission:

(This is a similar problem to that of bug 48489, hence the similar title.)

In current git, Unicode input mode (Meta-V, plus a 6-digit hex sequence) reads in text properly, but, in parse_verbatim_kbinput(), adds the bytes to the buffer one by one after the sequence is complete. Those bytes are, of course, invalid UTF-8 individually, but create a valid sequence as a whole. However, their being added individually means that the character count increases by one for each byte, when it should only increase by one for each character.

You can reproduce one test case as follows:

1. In a UTF-8 locale, run "nano --ignore --const --nonewlines".

2. Type: Meta-V "0 0 2 6 3 8". This will add the Unicode character U+2638 to the empty buffer.

3. Note that the character count in the cursor position display now says you're on character 1 out of 3, when it should say 1 out of 1. This is because the UTF-8 sequence for character U+2638 is three bytes long.

David Lawrence Ramsey <dolorous>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by vapier (Updated the item)
  • -unavailable- added by bens (Posted a comment)
  • -unavailable- added by dolorous (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 9 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 02 Apr 2017 09:48:54 AM UTCbensOpen/ClosedOpen=>Closed
      Summarystatus bar character count should count chars/codepoints not bytes=>statusbar character count should count chars/codepoints not bytes
    Tue 07 Mar 2017 11:37:53 PM UTCvapierSummarywhen using Unicode Input, the bytes should count as a single character=>status bar character count should count chars/codepoints not bytes
    Tue 28 Feb 2017 03:49:20 PM UTCbensStatusIn Progress=>Fixed
    Mon 27 Feb 2017 08:23:57 PM UTCbensAttached File-=>Added minimal-needed-change.patch, #39841
      StatusNone=>In Progress
    Mon 27 Feb 2017 08:18:18 PM UTCbensAssigned toNone=>bens
    Sun 26 Feb 2017 06:23:15 PM UTCdolorousAttached File-=>Added 0001-input-fix-wrong-character-count-after-verbatim-Unico.patch, #39835
    Sun 26 Feb 2017 09:59:02 AM UTCbensSummarywhen Unicode input creates a valid character from invalid bytes, nano should adapt=>when using Unicode Input, the bytes should count as a single character

    Back to the top


    Powered by Savane 3.1-cleanup1