bugGNU TeXmacs - Bugs: bug #34209, Unicode characters in GUI interface

 
 

You are not allowed to post comments on this tracker with your current authentification level.

bug #34209: Unicode characters in GUI interface

Submitted by:  Joris van der Hoeven <vdhoeven>
Submitted on:  Mon 05 Sep 2011 07:11:02 AM UTC  
 
Category: Qt portPriority: 5 - Normal
Item Group: ErrorStatus: Confirmed
Privacy: PublicAssigned to: Miguel de Benito <mdbenito>
Originator Name: Open/Closed: Open
Release: Release: None
Fixed Release: NoneFixed Release: 
Keywords: 

Tue 03 Dec 2013 10:00:41 AM UTC, comment #5:

Changes in url handling are extremely sensitive, so I would first need a minimal patch in order to study your proposal. For the moment, your patch contains various unessential changes, which makes it hard to focus on the main proposal.

Joris van der Hoeven <vdhoeven>
Project Administrator
Mon 02 Dec 2013 09:11:49 PM UTC, comment #4:

I've attached a proof-of-concept patch which adresses these issues in a more consistent way. The basic idea is to use the system's default encoding internally in the url class, then convert to and from cork (the encoding currently assumed everywhere else) whenever conversions to and from strings are required (constructors, concretize(), etc.)

To this end, we implement two functions, convert_from_filesystem_encoding() and convert_to_filesystem_encoding(), which should follow the guidelines explained in comment #3. In the patch they are little more than placeholders which assume filenames in UTF8.

Using them across url creation and conversion routines makes the use of heuristic detection unnecessary under Qt and fixes displaying of file names in menus under X11. There are several problems with the X11 version, though:

  • I'm not sure what encoding is appropriate for X11's XSetIconName, which is the function ultimately used to set the window name in x_window_rep::set_name. The man page seems to indicate that it is the "Host Portable Character Encoding", but this doesn't support most international charsets AFAIK.
  • Some WidKit widgets don't work well with the new internal representation using cork of urls converted to strings. As a result, saving a file with special chars in it (e.g. "hé.tm") creates a filename (e.g. "h%E9.tm") which under MacOS 10.9 crashes the Finder.

There are also some questions open:

  • Will these changes break support for some special encodings? For instance, the changes involve undoing Zou Hou's patch for chinese file names, so we should test with chinese again.
  • Using the encoding converter mechanism, which relies on urls to open its dictionaries, is a chicken and egg situation which results in many "convert-error, Couldn't open encoding dictionary..." upon TeXmacs startup.

(file #29761)

Miguel de Benito <mdbenito>
Project MemberIn charge of this item.
Mon 02 Dec 2013 11:33:51 AM UTC, comment #3:

This bug is not present under the Qt port thanks to François' heuristic detection of encodings.

The problem is that the encoding of file names is ignored by the url system and therefore file names are handled internally "as-is", whereas strings for widgets are expected to be in a specific encoding.

We need to convert all file names we read according to the OS / filesystem:

  • NTFS uses unicode16. Info for Windows filesystems
  • Extfs doesn't set any default encoding. Most applications use and assume utf8. QFile uses the user locale (this may be overriden)
  • MacOS uses a "decomposed" version of UTF-8 (some comments here)
Miguel de Benito <mdbenito>
Project MemberIn charge of this item.
Thu 07 Feb 2013 01:10:31 AM UTC, comment #2:

This has been "fixed" in svn r6254. Corrections are probably not robust and are only for Qt.

The reason is that encoding of file names is sometimes done as cork, sometimes as-is (utf8 in my system). It is not clear to me at which points in the url system internal encoding is expected, and at which it's the system's, so I think this is not a entirely a GUI issue.

Miguel de Benito <mdbenito>
Project MemberIn charge of this item.
Tue 04 Sep 2012 06:50:05 PM UTC, comment #1:

Note to self: The problem seems to be restricted to file names: the OS returns them as utf8 strings which TeXmacs' url system uses as-is. See SLOT_FILE in qt_tm_widget_rep.

Miguel de Benito <mdbenito>
Project MemberIn charge of this item.
Mon 05 Sep 2011 07:11:02 AM UTC, original submission:

In several cases, unicode characters are rendered incorrectly in GUI elements, such as title bars. Please check that corrections are robust and also work for the X11 version.

Joris van der Hoeven <vdhoeven>
Project Administrator

 

Attached Files
file #29761:  fix-unicode-in-gui-7984.patch added by mdbenito (16kB - application/octet-stream - See comment #4)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mdbenito (Posted a comment)
  • -unavailable- added by vdhoeven (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 2 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 02 Dec 2013 09:11:49 PM UTCmdbenitoAttached File-=>Added fix-unicode-in-gui-7984.patch, #29761
    Tue 04 Sep 2012 06:50:05 PM UTCmdbenitoAssigned tomgubi=>mdbenito

    Back to the top


    Powered by Savane 3.1-cleanup