bugPSPP - Bugs: bug #41864, Exporting SPSS metadata with...

 
 

bug #41864: Exporting SPSS metadata with "DISPLAY DICTIONARY" from codepage DIN_66003 writes "Custom attributes:" to each variable

Submitter:  Andre Müller <andrem>
Submitted:  Thu 13 Mar 2014 06:14:37 PM UTC
   
 
Category:  System/Portable File I/O Severity:  5 - Average
Status:  Invalid Assigned to:  None
Open/Closed:  Closed Release:  None
Effort:  0.00
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 19 Mar 2014 12:15:34 PM UTC, comment #3: 

Oh, and re using our files for the "scientific purpose"
of software development:
That really is OK --
for one thing this is software development aiding our
archiving tasks (thanks indeed). We do provide
the files to other developers as well, on the
condition of not redistributing.
Posting the file to public bugzilla would
set me in violation of the rules, that's why I rather
send links.

In perspective, we are exploring ways how to get
rid of the license restrictions. The licences
were thought liberal some thirty years ago,
and now it's not easy to change it because
we have piles of agreements with data providers.

Andre Müller <andrem>
Mon 17 Mar 2014 01:52:47 PM UTC, comment #2: 

Ah, OK.
Thanks a lot for looking at it. I could not make heads or tails from the output. Now I see more clearly.

As to the file being bizarre: Yes, agreed, and there's
a dozen files that do the same thing (or I would not have
reported it).

A main reason for files not exporting properly actually
is that somebody "fixed" some file while editing, say
an IBM850 encoded file as a windows-1252 encoded one.
Codepage mixups are big fun...

Thanks again,
Andre


Andre Müller <andrem>
Sun 16 Mar 2014 06:41:36 PM UTC, comment #1: 


> An example file (download needs registration):
> https://dbk.gesis.org/dbksearch/download.asp?db=D&id=40413


This file is bizarre.  It self-identifies as being encoded in windows-1252, but clearly it is not windows-1252.  You tell me that it is in DIN_66003, and I believe you, but it has internal contradictions.  The particular one that you see is that each variable has an attribute that, if one interprets the strings in the file as windows-1252, is named "$@Role".  Such attributes are the way that SPSS identifies the role of a variable.  But DIN_66003 has no @, and in place of @ the DIN_66003 encoding has §, so when you tell PSPP that the file is in DIN_66003 you get a $§Role attribute for every variable.

I assume what happened here is that some version of SPSS (probably version 18, that's the version that wrote the file) read the DIN_66003 encoded file as if it were encoded in windows-1252, then wrote it out, and in the process added the really windows-1252 attributes.

There isn't any way to reasonably and correctly interpret the whole file as any single encoding.  One would have to pick and choose which fields are which encoding.  I can't reasonably add that feature to PSPP.

By the way, do you interpret downloading a file from gesis to figure out why it isn't read correctly to be a form of scientific research?  I always feel like I'm lying when I tell it that, but if I tell it that I'm downloading it for non-scientific reasons, it won't let me do it.  Is software development a kind of scientific research?

Ben Pfaff <blp>
Group administrator
Thu 13 Mar 2014 06:14:37 PM UTC, original submission:  

When exporting SPSS metadata from a DIN_66003 encoded file,
each variable's output gets two extra lines:
"Custom attributes:
        $§Role  0"

The export is done using
GET FILE="source.sav" ENCODING="DIN_66003"
DISPLAY DICTIONARY

The variables actually just have the input role,
and that's what both SPSS and PSPP tell me in the GUI.

Also, this does not show up when exporting this file
with another encoding option.

(I can't seem to use the encoding in psppire, the
codepage is not on offer in the file dialog.)

Apart from these "bonus" lines, the export is flawless,
the codepage is used; the textual output is fine.
This same behaviour is found for a series of studies.

An example file (download needs registration):
https://dbk.gesis.org/dbksearch/download.asp?db=D&id=40413

Some example output:

zeitnot2        Label: ZEITNOT BEI 16 FREIZEITAKTIVITäTEN               270
        Format: F8.2
        Measure: Scale
        Display Alignment: Right
        Display Width: 10
        Custom attributes:
        $§Role  0

Note: DIN_66003 is really odd and outdated,
a 7bit-safe encoding for Germany.
I found it used in a dozen datasets from
different sources, though.
That's just to say that it may not be something
worth spending a lot of time on getting it right.
For my work, I can just discard these lines.

Andre Müller <andrem>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by blp (Posted a comment)
  • -email is unavailable- added by andrem (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-03-16 blp CategoryOutput Driver System/Portable File I/O
        StatusNone Invalid
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-bb6a.
    Corresponding source code