bugGNU nano - Bugs: bug #47626, composed and decomposed characters...


You are not allowed to post comments on this tracker with your current authentication level.

bug #47626: composed and decomposed characters are not seen as equal

Submitted by:  Benno Schulenberg <bens>
Submitted on:  Tue 05 Apr 2016 12:15:18 PM UTC  
Severity:  2 - Minor Status:  None
Assigned to:  None Open/Closed:  Open

Tue 05 Apr 2016 01:23:51 PM UTC, comment #1: 

Proposed solution: when reading in a file (in a Unicode locale), nano should normalize the encoding.  And, when any normalization occurs, it should note this on the statusbar (just like when a DOS or Mac file is converted -- the difference is that now writing out the file does modify it).  This normalization step should probably be behind a configuration and/or runtime flag.

Benno Schulenberg <bens>
Project Administrator
Tue 05 Apr 2016 12:15:18 PM UTC, original submission:  

When a Unicode-encoded text contains characters that are canonically equivalent but encoded in differing manners, then nano will not see them as the same when searching for them.

To reproduce, use nano to open the attached file, which contains the words caña and coöp, each encoded in two ways: once with single code point for the ñ and ö, and once with a normal n/o followed by a combining tilde/diaeresis.  Then, when searching for caña or coöp, nano will find only the first occurrence, not the equivalent second one.

(As a small consolation: vim and emacs and geany and pico don't get this right either.  But gedit does.)

(To confuse matters further: what I call "composed" is what Unicode calls "decomposed": two or more code points that together make up a single character.  And what I call "precomposed" is what Unicode calls "composed": a single code point that signifies a single character that consists of two or more visual elements.)

Benno Schulenberg <bens>
Project Administrator


Attached Files
file #36840:  equivalents added by bens (29B - application/octet-stream)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by bens (Submitted the item)

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.




    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-04-05 bens Summarycomposed and precomposed characters are not seen as equal => composed and decomposed characters are not seen as equal
    2016-04-05 bens Attached File- => Added equivalents, #36840

    Back to the top

    Powered by Savane 3.5