bugGNU nano - Bugs: bug #50403, statusbar character count should...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #50403: statusbar character count should count chars/codepoints not bytes

Submitter:  David Lawrence Ramsey <dolorous>
Submitted:  Sat 25 Feb 2017 09:27:30 PM UTC
   
 
Severity:  3 - Normal Status:  Fixed
Assigned to:  bens Open/Closed:  Closed

Jump to the original submission

Tue 28 Feb 2017 03:49:20 PM UTC, comment #8: 

Fixed in git, 91910b23.

Benno Schulenberg <bens>
Group administrator
Mon 27 Feb 2017 10:48:47 PM UTC, comment #7: 

That patch works too.

David Lawrence Ramsey <dolorous>
Group Member
Mon 27 Feb 2017 08:23:57 PM UTC, comment #6: 

Yes, that seems to work.  See attached patch.

(file #39841)

Benno Schulenberg <bens>
Group administrator
Mon 27 Feb 2017 08:18:18 PM UTC, comment #5: 

The offending commit is 08c51cfd.  So... this suggests to me that the solution is to not return a count of 1, but to return a count of 2 or 3 or 4.  (I haven't tested this yet, but things worked before this commit, so...).  Looking into it...

Benno Schulenberg <bens>
Group administrator
Sun 26 Feb 2017 06:23:15 PM UTC, comment #4: 

The attached patch should fix it.  It's not that Unicode input only returns the first byte, it's that verbatim input outputs whatever it gets byte by byte, which is fine for everything except Unicode input, which needs the whole sequence read at once.

(file #39835)

David Lawrence Ramsey <dolorous>
Group Member
Sun 26 Feb 2017 11:46:41 AM UTC, comment #3: 

Bug exists since nano-2.6.3.

Benno Schulenberg <bens>
Group administrator
Sun 26 Feb 2017 09:59:02 AM UTC, comment #2: 

It works fine when typing Unicode characters directly.  For example, when typing «ŝámpú» (the chevrons included) the character count is 7 -- which you can verify by copy/pasting the text.

Nice find.  So Unicode Input should return not just the first byte, but all of the bytes of the codepoint.  If you want to attempt a patch... I am busy elsewhere.

Benno Schulenberg <bens>
Group administrator
Sat 25 Feb 2017 09:48:59 PM UTC, comment #1: 

Addendum: I think this problem may also apply to typing in UTF-8 characters normally via the keyboard, instead of only in verbatim input mode, but with my keyboard and default settings, I have no way to test that at the moment.

David Lawrence Ramsey <dolorous>
Group Member
Sat 25 Feb 2017 09:27:30 PM UTC, original submission:  

(This is a similar problem to that of bug 48489, hence the similar title.)

In current git, Unicode input mode (Meta-V, plus a 6-digit hex sequence) reads in text properly, but, in parse_verbatim_kbinput(), adds the bytes to the buffer one by one after the sequence is complete.  Those bytes are, of course, invalid UTF-8 individually, but create a valid sequence as a whole.  However, their being added individually means that the character count increases by one for each byte, when it should only increase by one for each character.

You can reproduce one test case as follows:

1. In a UTF-8 locale, run "nano --ignore --const --nonewlines".

2. Type: Meta-V "0 0 2 6 3 8".  This will add the Unicode character U+2638 to the empty buffer.

3. Note that the character count in the cursor position display now says you're on character 1 out of 3, when it should say 1 out of 1.  This is because the UTF-8 sequence for character U+2638 is three bytes long.

David Lawrence Ramsey <dolorous>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by vapier (Updated the item)
  • -email is unavailable- added by bens (Posted a comment)
  • -email is unavailable- added by dolorous (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-04-02 bens Open/ClosedOpen Closed
        Summarystatus bar character count should count chars/codepoints not bytes statusbar character count should count chars/codepoints not bytes
    2017-03-07 vapier Summarywhen using Unicode Input, the bytes should count as a single character status bar character count should count chars/codepoints not bytes
    2017-02-28 bens StatusIn Progress Fixed
    2017-02-27 bens Attached File- Added minimal-needed-change.patch, #39841
        StatusNone In Progress
    2017-02-27 bens Assigned toNone bens
    2017-02-26 dolorous Attached File- Added 0001-input-fix-wrong-character-count-after-verbatim-Unico.patch, #39835
    2017-02-26 bens Summarywhen Unicode input creates a valid character from invalid bytes, nano should adapt when using Unicode Input, the bytes should count as a single character

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code