bugPSPP - Bugs: bug #12984, need i18n support for identifiers,...

 
 

bug #12984: need i18n support for identifiers, system & portable files

Submitted by:  Ben Pfaff <blp>
Submitted on:  Thu 05 May 2005 02:23:25 PM UTC  
 
Category: Syntax ParserSeverity: 5 - Average
Status: FixedAssigned to: Ben Pfaff <blp>
Open/Closed: ClosedRelease: Post-0.6.0
Effort: 0.00

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 09 Apr 2009 01:19:26 AM UTC, comment #5:

Charset has been merged with master. Closing this item.

John Darrington <jmd>
Project Administrator
Wed 08 Apr 2009 01:36:09 AM UTC, comment #4:

Most of this has now been fixed in the charset branch.

John Darrington <jmd>
Project Administrator
Thu 21 Dec 2006 09:30:27 PM UTC, comment #3:

A large part of this is now complete.

The parts outstanding that I'm aware of are:

  1. Strings entered in syntax files cannot be non-ascii.
  2. We don't set or inspect the codepage value which we've recently discovered (but we shouldn't rely on it, since old versions don't use it).
  3. The LOCALE and NOCONVERT options to GET as mentioned in iii below haven't been implemented.

I'm changing this to Post next release. If anyone thinks it's very important feel free to change it back.

John Darrington <jmd>
Project Administrator
Wed 31 May 2006 06:53:37 AM UTC, comment #2:

From the dev-pspp mailing list:
On Sun, Mar 19, 2006 at 05:26:47PM -0800, Ben Pfaff wrote:

Let me elaborate. Here is the plan that I envision:

i. PSPP adopts a single locale that defaults to the system locale
but can be changed with SET LOCALE. (I'll call this the "PSPP
locale".)

ii. All string data in all casefiles and dictionaries is in the
PSPP locale, or at least we make that assumption.

iii. The GET command assumes by default that data read in is in
the PSPP locale. If the user provides a LOCALE subcommand
specifying something different, then missing values and
value label keys are converted as the dictionary is read and
string case data is converted "on the fly" as data is read
from the file. We can also provide a NOCONVERT subcommand
(with a better name, I hope) that flags string variables
that are not to be converted.

iv. The SAVE command assumes by default that data written out is
to be in the PSPP locale. If the user provides a LOCALE
subcommand specifying something different, then we convert
string data, etc., as we write it, and again exceptions can
be accommodated.

v. Users who want accurate translations, as in your survey
example, choose a reasonable PSPP locale, e.g. something based
on UTF-8.

vi. We look into the possibility of tagging system files with a
locale. The system file format is extensible enough that
this would really just be a matter of testing whether SPSS
will complain loudly about our extension records or just
silently ignore them.

John Darrington <jmd>
Project Administrator
Fri 20 May 2005 01:32:34 AM UTC, comment #1:

The support that SPSS affords is pretty poor.

I started putting non ascii characters in SPSS v12 using the data editor. It crashed very quickly.

I suggest we don't try to emulate that behaviour!

John Darrington <jmd>
Project Administrator
Thu 05 May 2005 02:23:25 PM UTC, original submission:

SPSS supports international characters in variable names, vector names, etc. We should too.

Internally, I expect we'll want to use UTF-8. I don't know what encoding we should expect to see in command files. I also don't know what encoding is used in system files. We should do some investigation for purposes of compatibility.

(Also, the SPSS 12 manual, in the section on variable names, implies that UCS-2 encoding is used internally for variable names. I don't know whether to believe that or not.)

Ben Pfaff <blp>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jmd (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 5 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 09 Apr 2009 01:19:26 AM UTCjmdOpen/ClosedOpen=>Closed
    Wed 08 Apr 2009 01:36:09 AM UTCjmdStatusIn Progress=>Fixed
    Thu 21 Dec 2006 09:30:27 PM UTCjmdReleaseBefore 0.6.0=>Post-0.6.0
    Wed 31 May 2006 06:53:37 AM UTCjmdStatusNone=>In Progress
    Sat 07 May 2005 04:31:31 AM UTCblpReleaseNone=>Before 0.6.0

    Back to the top


    Powered by Savane 3.1-cleanup1