bugPSPP - Bugs: bug #33255, Non-ascii filenames not properly...

 
 

bug #33255: Non-ascii filenames not properly handled.

Submitted by:  John Darrington <jmd>
Submitted on:  Sat 07 May 2011 05:02:39 AM UTC  
 
Category: Internationalization/LocalizationSeverity: 5 - Average
Status: Need InfoAssigned to: None
Open/Closed: OpenRelease: Future
Effort: 0.00

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Tue 03 Dec 2013 05:02:18 PM UTC, comment #7:

This seems to be the same, or very similar problem reported by abraham laredo sicsu <alsicsu@gmail.com>

John Darrington <jmd>
Project Administrator
Wed 24 Jul 2013 03:40:44 PM UTC, comment #6:

Harry Thijssen <pspp@sjpaes.nl> writes in email:

> This is a really old bug and there are so many changes in all the software
> around. Maybe we can ask the list if anybody still encounters this problem
> with the latest MSWindows builds.
>
> I wouldn't be surprised if the problem has gone.

Ben Pfaff <blp>
Project Administrator
Wed 24 Jul 2013 05:28:19 AM UTC, comment #5:

> Recent incarnations of windows use UTF-16LE encoding not UTF-8.


I'm pretty sure that Windows has had the "W" versions of library functions, that accept 16-bit Unicode encoding, practically forever. Since Windows NT 3.x anyhow.

I've heard that there was a more recent change from using UCS-2 encoding (which only supports characters in the Unicode Basic Multilingual Plane) to UTF-16 encoding (which supports all Unicode characters via surrogate pairs). But that wouldn't affect support for simple accented characters.

It might be worthwhile to bring this up on the gnulib mailing list (but Bruno, who would really know, has been MIA for a year or more).

Ben Pfaff <blp>
Project Administrator
Tue 14 Jun 2011 07:22:34 AM UTC, comment #4:

I got this from a user with the latest PSPP 0.7.8. for MSWindows version. I guess this is the same problem:

-words accented: New thing I discovered, when the name of a directory
contains a word accented, then the pspp can't open a file from inside.
Before, this didn't happen.

The directory is named 'Ficheros estadísticos'

Harry Thijssen <unknown_one>
Sat 28 May 2011 04:45:34 PM UTC, comment #3:

I had a chat with some of the mingw guys on irc and found the root of the problem: Recent incarnations of windows use UTF-16LE encoding not UTF-8.

The attached two patches (one against pspp, one against gnulib) demonstrate one way to circumvent the problem. Obviously a proper fix will require more thought (especially if its to continue to work with older windoze).

(file #23460, file #23461)

John Darrington <jmd>
Project Administrator
Sun 15 May 2011 04:49:59 AM UTC, comment #2:

I had another look. Debugging on windows is not easy because it lacks even basic tools. However I did ascertain that it is not a GUI related issue. The problem exists on the terminal interface too. I'm suspecting, in fact, that it is the open command which is at fault. (this is provided by gnulib?)
If a version with some diagnostic info can be hacked up it might help.

Something else I noticed (I don't think it's related to this problem, but one never knows):

In file-handle-def.c the comment to fh_create_file says that the
ID must be UTF-8, but doesn't say anything about the FILE_NAME.
However since the implementation copies them both into the same string, then either we must insist that FILE_NAME is also UTF8 or we must convert it inside the function.

John Darrington <jmd>
Project Administrator
Fri 13 May 2011 04:34:14 AM UTC, comment #1:

John, I don't yet see where the problem is occurring. It looks to me that load_file() in psppire-data-window.c is converting from glib file name encoding into UTF-8 and using the UTF-8 version to compose the syntax string. I'm having a lot of trouble seeing what's going wrong here; everything looks consistent.

Would you mind taking a closer look?

Thanks,

Ben.

Ben Pfaff <blp>
Project Administrator
Sat 07 May 2011 05:02:39 AM UTC, original submission:

Using 0.7.7-gfa1fed on w32, tried to save, then reopen a dataset, using a filename containing a non-ascii filename.

The file could be saved, but the name got corrupted. It was then not possible to open this file.

The log from pspp.jnl show what is happening:

SAVE OUTFILE="C:\Users\Martin.Wagner\öl.sav".

GET FILE="C:\Users\Martin.Wagner\öl.sav".

error: An error occurred while opening `C:\Users\Martin.Wagner\öl.sav': No such file or directory.

John Darrington <jmd>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #23460:  rename.patch added by jmd (1kB - text/x-patch)
file #23461:  make-file.patch added by jmd (566B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jmd
  • -unavailable- added by unknown_one (Posted a comment)
  • -unavailable- added by blp (Posted a comment)
  • -unavailable- added by jmd (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 03 Dec 2013 05:02:18 PM UTCjmdCarbon-Copy-=>Added sicsu
    Thu 23 Feb 2012 07:19:44 AM UTCjmdReleaseBefore 0.8.0=>Future
    Sat 28 May 2011 04:45:34 PM UTCjmdAttached File-=>Added rename.patch, #23460
      Attached File-=>Added make-file.patch, #23461
    Sat 14 May 2011 04:51:02 AM UTCblpReleaseNone=>Before 0.8.0
    Fri 13 May 2011 04:34:14 AM UTCblpStatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup