bugPSPP - Bugs: bug #12984, need i18n support for identifiers,...

 
 

bug #12984: need i18n support for identifiers, system & portable files

Submitter:  Ben Pfaff <blp>
Submitted:  Thu 05 May 2005 02:23:25 PM UTC
   
 
Category:  Syntax Parser Severity:  5 - Average
Status:  Fixed Assigned to:  blp
Open/Closed:  Closed Release:  Post-0.6.0
Effort:  0.00
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 09 Apr 2009 01:19:26 AM UTC, comment #5: 

Charset has been merged with master.  Closing this item.

John Darrington <jmd>
Group administrator
Wed 08 Apr 2009 01:36:09 AM UTC, comment #4: 

Most of this has now been fixed in the charset branch.

John Darrington <jmd>
Group administrator
Thu 21 Dec 2006 09:30:27 PM UTC, comment #3: 

A large part of this is now complete.

The parts outstanding that I'm aware of are:

  1. Strings entered in syntax files cannot be non-ascii.
  2. We don't set or inspect the codepage value which we've recently discovered (but we shouldn't rely on it, since old versions don't use it).
  3. The LOCALE and NOCONVERT options to GET as mentioned in iii below haven't been implemented.


I'm changing this to Post next release.  If anyone thinks it's very  important feel free to change it back.

John Darrington <jmd>
Group administrator
Wed 31 May 2006 06:53:37 AM UTC, comment #2: 

From the dev-pspp mailing list:
On Sun, Mar 19, 2006 at 05:26:47PM -0800, Ben Pfaff wrote:

     Let me elaborate.  Here is the plan that I envision:

     i. PSPP adopts a single locale that defaults to the system locale
        but can be changed with SET LOCALE.  (I'll call this the "PSPP
        locale".)

     ii. All string data in all casefiles and dictionaries is in the
         PSPP locale, or at least we make that assumption.

     iii. The GET command assumes by default that data read in is in
          the PSPP locale.  If the user provides a LOCALE subcommand
          specifying something different, then missing values and
          value label keys are converted as the dictionary is read and
          string case data is converted "on the fly" as data is read
          from the file.  We can also provide a NOCONVERT subcommand
          (with a better name, I hope) that flags string variables
          that are not to be converted.

     iv. The SAVE command assumes by default that data written out is
         to be in the PSPP locale.  If the user provides a LOCALE
         subcommand specifying something different, then we convert
         string data, etc., as we write it, and again exceptions can
         be accommodated.

     v. Users who want accurate translations, as in your survey
        example, choose a reasonable PSPP locale, e.g. something based
        on UTF-8.

     vi. We look into the possibility of tagging system files with a
         locale.  The system file format is extensible enough that
         this would really just be a matter of testing whether SPSS
         will complain loudly about our extension records or just
         silently ignore them.

John Darrington <jmd>
Group administrator
Fri 20 May 2005 01:32:34 AM UTC, comment #1: 

The support that SPSS  affords is pretty poor.

I started putting non ascii characters in SPSS v12 using the data editor.  It crashed very quickly.

I suggest we don't try to emulate that behaviour!

John Darrington <jmd>
Group administrator
Thu 05 May 2005 02:23:25 PM UTC, original submission:  

SPSS supports international characters in variable names, vector names, etc.  We should too.

Internally, I expect we'll want to use UTF-8.  I don't know what encoding we should expect to see in command files.  I also don't know what encoding is used in system files.  We should do some investigation for purposes of compatibility.

(Also, the SPSS 12 manual, in the section on variable names, implies that UCS-2 encoding is used internally for variable names.  I don't know whether to believe that or not.)

Ben Pfaff <blp>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jmd (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2009-04-09 jmd Open/ClosedOpen Closed
    2009-04-08 jmd StatusIn Progress Fixed
    2006-12-21 jmd ReleaseBefore 0.6.0 Post-0.6.0
    2006-05-31 jmd StatusNone In Progress
    2005-05-07 blp ReleaseNone Before 0.6.0

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code