bugGNUstep - Bugs: bug #39900, unar: Uncaught exception...

Group
 
 

bug #39900: unar: Uncaught exception NSCharacterConversionException, reason: Can't get cString from Unicode string.

Submitter:  - <_92319>
Submitted:  Fri 30 Aug 2013 12:59:48 PM UTC
   
 
Category:  Base/Foundation Severity:  3 - Normal
Item Group:  Bug Status:  Ready For Test
Privacy:  Public Assigned to:  CaS
Open/Closed:  In Test
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 10 Sep 2013 07:32:12 PM UTC, comment #13: 

Fred, I've compiled and installed version 20130910 from daily-snapshots (ftp://ftp.gnustep.org/pub/daily-snapshots/) and I'm using unarchiver without the (file #28969) patch. Everything seems OK, unar does not crash.

- <_92319>
Mon 09 Sep 2013 09:40:36 PM UTC, comment #12: 

Richard just committed a change to base that re-adds the setlocale() call when getting the default encoding. Could you please try again with that?

Maybe a framework has to call that function after all? But has to make sure to undo the effects afterwards?

Fred Kiefer <FredKiefer>
Group Member
Sat 31 Aug 2013 01:03:53 PM UTC, comment #11: 

I am the author of unar, and I'd like to add that I am not sure it makes sense to require a setlocale() for this to work correctly. The called method is fileSystemRepresentation, and it does seem to me that it would be the job of that function to work out the correct encoding to use without requiring help from the rest of the program.

I'll add that patch for now, though, to deal with currently available versions, but I'd suggest looking into how to make fileSystemRepresentation work correctly without it.

Dag Ă…gren <waha_06x36>
Sat 31 Aug 2013 12:31:45 PM UTC, comment #10: 

Yes, the patch solved my problems! Thanks to all of you!

(file #28969)

- <_92319>
Sat 31 Aug 2013 12:13:29 PM UTC, comment #9: 

So it is a problem with the unar program, after all. What should I tell the author of the unar? Just to point him to this issue?

- <_92319>
Sat 31 Aug 2013 11:32:24 AM UTC, comment #8: 

I think that Eirc was correct with his answer. I found this link when searching for issues with nl_langinfo:

http://stackoverflow.com/questions/1558379/whi-is-nl-langinfocodeset-different-from-locale-charmap

It explains that you need to call setlocale(LC_ALL, "") first.

I have attached a small progam that demonstrated this. If you comment out that line, you will get the wrong result from nl_langinfo.

Eric is also correct in that a framework should not call such an important function  somewhere in the depths of its implementation. And as GNUstep is just a set of frameworks we go to great pain in base to not set the locale permanently.
Most likely it belongs into the main() function of the real application in this case unar.

We could thing about adding this call to NSApplicationMain, but that wont help you with a command line program.

(file #28966)

Fred Kiefer <FredKiefer>
Group Member
Fri 30 Aug 2013 11:00:14 PM UTC, comment #7: 

Hi Eric.

No, I'm running the latest stable version, 1.24.5, but I've also tried versions 1.24.0 (with which the unar worked fine before) through 1.24.4.

And yes, unar is a command line tool, you can get its sources from here:
https://code.google.com/p/theunarchiver/downloads/list
("unar1.7_src.zip" is what you want).

- <_92319>
Fri 30 Aug 2013 10:17:09 PM UTC, comment #6: 

Hey Daimonion,
Are you running a recent svn build of gnustep-back?
This change I made could potentially mess up character encoding related things:

"r37002 | ericwa | 2013-08-21 16:30:15 -0600 (Wed, 21 Aug 2013) | 8 lines

  • Source/x11/XIMInputServer.m: Add the setlocale(LC_CTYPE, "") call

back that I removed last year (r35152).

I tested the fcitx input method, and without the setlocale call,
XOpenIM would fail. I'm not sure what the best plan is in the long
term, I wanted to avoid calling setlocale from the core frameworks
but in this case xlib seems to require it."

But if unar is a command line tool, that code probably isn't even running.

Btw, I couldn't figure out an easy way to build unar under GNUstep. Could you post a tgz of your source or a GNUmakefile please?

Eric Wasylishen <ericw>
Group Member
Fri 30 Aug 2013 09:23:56 PM UTC, comment #5: 

I've tried gnustep-base v1.24.0 (which I definitely had no issues with), and the same error. Downgrading to unar v1.6 (which definitely worked) solved nothing, issue is still present. The only conclusion is that some other thing (which recently got upgraded) is causing this. Any thoughts on what could it be?

- <_92319>
Fri 30 Aug 2013 08:43:09 PM UTC, comment #4: 

Fred, still no backtrace:
http://pastebin.com/hjcfdUih

The encoding should be UTF-8:

% echo $LANG
sr_RS.UTF-8

Again, these problems started recently. It used to work before. I'm gonna try previous versions of gnustep-base as well as unar.

- <_92319>
Fri 30 Aug 2013 08:18:09 PM UTC, comment #3: 

I am no gdb expert, but I normally start it with just the program as argument, set my breakpoints and then give the arguments to the r command. In this case something like:

gdb unar
b GSPrivateDefaultCStringEncoding
r arhiva.zip


Which encoding would you expect as the default for your system? I just thought it might be UTF-8, but it could as well be ISO8859-5 or what ever.

The code in GSPrivateDefaultCStringEncoding() uses nl_langinfo(CODESET) to detect this. But we could extend this to try ucnv_getDefaultName() frist, as corebase is doing.

Fred Kiefer <FredKiefer>
Group Member
Fri 30 Aug 2013 06:14:41 PM UTC, comment #2: 

Fred, it works with 'export GNUSTEP_STRING_ENCODING=UTF-8'. I have gdb installed, but I'm not sure how to start the debugging. I've tried with

gdb --args unar arhiva.zip

but it ended before the backtrace could be generated.

- <_92319>
Fri 30 Aug 2013 02:46:19 PM UTC, comment #1: 

Most likely the problem is that GNUstep detects your system encoding differently from what you expect.
Now the best way to see what gets detected and why, is to go through the function GSPrivateDefaultCStringEncoding() with a debugger. If this isn't an option for you the next best thing to try is to set the environment variable GNUSTEP_STRING_ENCODING to UTF-8 and see whether the code works after this. That way you still don't know why it isn't working :-(

Fred Kiefer <FredKiefer>
Group Member
Fri 30 Aug 2013 12:59:48 PM UTC, original submission:  

unar, application to unpack various archives failes upon extracting an UTF-8 named file, or when extracting to a UTF-8 named folder. The error is:

unar: Uncaught exception NSCharacterConversionException, reason: Can't get cString from Unicode string.

Version of gnustep-base I've tested are 1.24.3, 1.24.4 and 1.24.5. on Arch Linux.

I've noticed this only recently. The program did work very well, few months ago.

I've reported bug to the unar author, he said I need to report the bug here:

https://code.google.com/p/theunarchiver/issues/detail?id=697

- <_92319>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #28969:  locale-detect-fix.patch added by _92319 (366B - application/octet-stream)
file #28966:  default_encoding.c added by FredKiefer (515B - text/x-csrc)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by waha_06x36 (Posted a comment)
  • -email is unavailable- added by ericw (Posted a comment)
  • -email is unavailable- added by FredKiefer (Posted a comment)
  • -email is unavailable- added by _92319 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-09-09 FredKiefer CategoryNone Base/Foundation
        StatusNone Ready For Test
        Assigned toNone CaS
        Open/ClosedOpen In Test
    2013-08-31 _92319 Attached File- Added locale-detect-fix.patch, #28969
    2013-08-31 FredKiefer Attached File- Added default_encoding.c, #28966

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code