bugGNU TeXmacs - Bugs: bug #34209, Unicode characters in GUI interface

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #34209: Unicode characters in GUI interface

Submitter:  Joris van der Hoeven <vdhoeven>
Submitted:  Mon 05 Sep 2011 07:11:02 AM UTC
   
 
Category:  Qt port Priority:  5 - Normal
Item Group:  Error Status:  Fixed
Privacy:  Public Assigned to:  mdbenito
Originator Name:  Open/Closed:  Closed
Release:  None Release: 
Fixed Release:  None Fixed Release: 
Keywords: 

Jump to the original submission

Fri 05 Mar 2021 06:31:14 PM UTC, comment #10: 

Without further feedback, I assume that the problem has been solved.

Joris van der Hoeven <vdhoeven>
Group administrator
Sun 01 Nov 2020 05:12:59 PM UTC, comment #9: 

Is this bug still present?

Please give me a detailed way to reproduce it with a recent version of TeXmacs. Otherwise, I will close it.

Joris van der Hoeven <vdhoeven>
Group administrator
Sun 01 Nov 2020 11:29:27 AM UTC, comment #8: 

Is this bug still present?

Joris van der Hoeven <vdhoeven>
Group administrator
Thu 02 Mar 2017 11:04:38 PM UTC, comment #7: 

Well, reopening myself; I was slightly too optimistic...
There are still some instances where utf8 is misrendered in the GUI, for instance in tm-files.scm when giving informative message on the footer.
As soon as the windows version uses utf8 in urls (which is underway) this will be an easy fix.

Philippe Joyez <pjoyez>
Group Member
Wed 01 Mar 2017 08:54:45 PM UTC, comment #6: 

I think most of the changes proposed by Miguel have been implemented since the last comment. I also fixed an issue with image filenames. Closing for clarity. Reopen and comment accordingly if this is still an issue.

Philippe Joyez <pjoyez>
Group Member
Tue 03 Dec 2013 10:00:41 AM UTC, comment #5: 

Changes in url handling are extremely sensitive, so I would first need a minimal patch in order to study your proposal.  For the moment, your patch contains various unessential changes, which makes it hard to focus on the main proposal.

Joris van der Hoeven <vdhoeven>
Group administrator
Mon 02 Dec 2013 09:11:49 PM UTC, comment #4: 

I've attached a proof-of-concept patch which adresses these issues in a more consistent way. The basic idea is to use the system's default encoding internally in the url class, then convert to and from cork (the encoding currently assumed everywhere else) whenever conversions to and from  strings are required (constructors, concretize(), etc.)

To this end, we implement two functions, convert_from_filesystem_encoding() and convert_to_filesystem_encoding(), which should follow the guidelines explained in comment#3. In the patch they are little more than placeholders which assume filenames in UTF8.

Using them across url creation and conversion routines makes the use of heuristic detection unnecessary under Qt and fixes displaying of file names in menus under X11. There are several problems with the X11 version, though:

  • I'm not sure what encoding is appropriate for X11's XSetIconName, which is the function ultimately used to set the window name in x_window_rep::set_name. The man page seems to indicate that it is the "Host Portable Character Encoding", but this doesn't support most international charsets AFAIK.
  • Some WidKit widgets don't work well with the new internal representation using cork of urls converted to strings. As a result, saving a file with special chars in it (e.g. "hé.tm") creates a filename (e.g. "h%E9.tm") which under MacOS 10.9 crashes the Finder.


There are also some questions open:

  • Will these changes break support for some special encodings?  For instance, the changes involve undoing Zou Hou's patch for chinese file names, so we should test with chinese again.
  • Using the encoding converter mechanism, which relies on urls to open its dictionaries, is a chicken and egg situation which results in many "convert-error, Couldn't open encoding dictionary..." upon TeXmacs startup.


(file #29761)

Miguel de Benito <mdbenito>
Group Member
Mon 02 Dec 2013 11:33:51 AM UTC, comment #3: 

This bug is not present under the Qt port thanks to François' heuristic detection of encodings.

The problem is that the encoding of file names is ignored by the url system and therefore file names are handled internally "as-is", whereas strings for widgets are expected to be in a specific encoding.

We need to convert all file names we read according to the OS / filesystem:

  • NTFS uses unicode16. Info for Windows filesystems
  • Extfs doesn't set any default encoding. Most applications use and assume utf8. QFile uses the user locale (this may be overriden)
  • MacOS uses a "decomposed" version of UTF-8 (some comments here)


Miguel de Benito <mdbenito>
Group Member
Thu 07 Feb 2013 01:10:31 AM UTC, comment #2: 

This has been "fixed" in svn r6254. Corrections are probably not robust and are only for Qt.

The reason is that encoding of file names is sometimes done as cork, sometimes as-is (utf8 in my system). It is not clear to me at which points in the url system internal encoding is expected, and at which it's the system's, so I think this is not a entirely a GUI issue.

Miguel de Benito <mdbenito>
Group Member
Tue 04 Sep 2012 06:50:05 PM UTC, comment #1: 

Note to self: The problem seems to be restricted to file names: the OS returns them as utf8 strings which TeXmacs' url system uses as-is. See SLOT_FILE in qt_tm_widget_rep.

Miguel de Benito <mdbenito>
Group Member
Mon 05 Sep 2011 07:11:02 AM UTC, original submission:  

In several cases, unicode characters are rendered incorrectly in GUI elements, such as title bars. Please check that corrections are robust and also work for the X11 version.

Joris van der Hoeven <vdhoeven>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #29761:  fix-unicode-in-gui-7984.patch added by mdbenito (16KiB - application/octet-stream - See comment #4)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pjoyez (Posted a comment)
  • -email is unavailable- added by mdbenito (Posted a comment)
  • -email is unavailable- added by vdhoeven (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-03-05 vdhoeven StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2020-11-01 vdhoeven StatusConfirmed Ready For Test
    2017-03-02 pjoyez Open/ClosedClosed Open
    2017-03-01 pjoyez Open/ClosedOpen Closed
    2013-12-02 mdbenito Attached File- Added fix-unicode-in-gui-7984.patch, #29761
    2012-09-04 mdbenito Assigned tomgubi mdbenito

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code