bugGNUstep - Bugs: bug #34486, gnustep has a problem with...

Group
 
 

bug #34486: gnustep has a problem with non-ascii string constants

Submitter:  julian <julian_>
Submitted:  Wed 05 Oct 2011 05:41:30 PM UTC
Votes: 2
 
Category:  Base/Foundation Severity:  1 - Wish
Item Group:  Bug Status:  Fixed
Privacy:  Public Assigned to:  None
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 15 Oct 2011 09:56:52 AM UTC, comment #8: 

thanks can confirm its working now as it should.

julian <julian_>
Fri 14 Oct 2011 07:20:03 AM UTC, comment #7: 

Since we formally dropped release for older compilers after the last release in September, we are now free to assume that we are using a compiler which supports UTF-8 in string constants ... so at last it is possible to support it in gnustep-base.

I've modified the constant string handling to assume that the compiler has provided UTF-8 strings (but that does mean perhaps we should also modify gnustep-make to either alert about or strip out any attempt to tell the compiler to use a different character encoding).

Richard Frith-Macdonald <CaS>
Group Member
Wed 05 Oct 2011 09:24:17 PM UTC, comment #6: 

i can confirm that works fine. thanks for another nice workaround.

julian <julian_>
Wed 05 Oct 2011 09:03:30 PM UTC, comment #5: 

Oh, you can do this: (UTF8 macro just for clarity)

#define UTF8(x) [NSString stringWithUTF8String: x]
#define kTrackNames [NSArray arrayWithObjects: UTF8("Tannhäuser Stargate"), UTF8("Soylent Black"), UTF8("Quantum Circuit"), UTF8("Blaspheme Quarantine"), UTF8("Frogpill Quasar"), UTF8("Terminal Velocity"), nil]

That works by default in gcc for me. It should work as long as the compiler stores C constant strings as UTF-8.

Eric Wasylishen <ericw>
Group Member
Wed 05 Oct 2011 08:48:08 PM UTC, comment #4: 

ugh if there is no way at all to get it into a string constant it really is a pain. i've currently this code and see no way how i'd be able to convert that

#define kTrackNames            [NSArray arrayWithObjects:@"Tannhäuser Stargate", @"Soylent Black", @"Quantum Circuit", @"Blaspheme Quarantine", @"Frogpill Quasar", @"Terminal Velocity", nil]

and kTrackNames is used throughout the app.


anyway, i'd be happy if it would work using clang trunk and nothing else.

julian <julian_>
Wed 05 Oct 2011 08:36:28 PM UTC, comment #3: 

Pretty much the two workarounds right now are putting the utf-16 codepoints in an array and then using -initWithCharacters:length:, or else loading the string from a file (maybe using NSLocalizedString?)

unichar string[2] = { 0x0068, 0x00e }; // utf-16 codepoints for "hé"
NSLog(@"%@", [[[NSString alloc] initWithCharacters: string length: 2] autorelease]);

That's a pain, though.

The \u escape sequence is probably treated the same by the compiler as loading a utf-8 source file, so it makes sense that it doesn't work at the moment.

btw, here is a mailing list post that may have some useful info: http://lists.apple.com/archives/cocoa-dev/2008/Nov/msg02226.html


Eric Wasylishen <ericw>
Group Member
Wed 05 Oct 2011 08:08:55 PM UTC, comment #2: 


>historically non-ascii data in ObjC string constants has been disallowed


yes, historically, but this has been changed with Mac OS X 10.5 and i think GNUstep should follow here. source files are encouraged to be UTF-8 and objc string constants with special chars work now - though i couldn't find an official apple document encouraging it. but the fact that they changed things to make it work and  Xcode also doesn't warn on this even with all warnings turned on - suggests its here to stay.

> discouraged all uses of anything which implies a default C string encoding elsewhere


yes but i think C string encoding is a completely different issue.


anyway i think this shouldn't be that high priority.

whats more disturbing is that even the workaround, manually specifying the unicode escape sequences doesn't work. e.g.

[mybutton setTitle:@"Her name is M\u00E4dchen"]; // ä

how is this done in gnustep?


julian <julian_>
Wed 05 Oct 2011 07:23:30 PM UTC, comment #1: 

I think this is probably an application bug since historically non-ascii data in ObjC string constants has been disallowed (ie strings documented as having to be ascii and behavior for non-ascii documented as unsupported/undefined).
Apple have also active discouraged all uses of anything which implies a default C string encoding elsewhere, and have afaik deprecated all such methods ... so they seem serious about being well behaved and requiring you to specify the encoding of all C strings generally.

However ...
1. you may know of some recent documentation defining a policy change and specifying how non-ascii data is to be handled in future or
2. it might be worth reverse engineering the latest OSX behavior and implementing that even if apple don't want developers to depend on it.

Richard Frith-Macdonald <CaS>
Group Member
Wed 05 Oct 2011 05:41:30 PM UTC, original submission:  

gnustep has a problem with non-ascii string constants e.g.

[myButton setTitle: @"hello umlaut ä, ö, ü "];

will result in garbage, even if the line is  in a file encoded as UTF-8.

this obviously works fine on Mac OS X / Cocoa.

trivial sample project attached.

julian <julian_>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #24104:  GNUstepTest.tar.gz added by julian_ (2KiB - application/x-gzip)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rmottola (Updated the item)
  • -email is unavailable- added by ericw (Posted a comment)
  • -email is unavailable- added by CaS (Posted a comment)
  • -email is unavailable- added by julian_ (Voted in favor of this item)
  • -email is unavailable- added by julian_ (Submitted the item)
  •  

    There are 2 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-10-21 rmottola Open/ClosedIn Test Closed
    2011-10-14 CaS StatusConfirmed Fixed
        Open/ClosedOpen In Test
    2011-10-05 ericw CategoryGui/AppKit Base/Foundation
        Severity3 - Normal 1 - Wish
        StatusNone Confirmed
    2011-10-05 julian_ Carbon-Copy- Added julian_
    2011-10-05 julian_ Attached File- Added GNUstepTest.tar.gz, #24104

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code