bugPSPP - Bugs: bug #48040, GLM produces wrong output

 
 

bug #48040: GLM produces wrong output

Submitted by:  Alan Mead <amead>
Submitted on:  Fri 27 May 2016 02:57:26 PM UTC  
 
Category: Numerical ErrorsSeverity: 5 - Average
Status: FixedAssigned to: None
Open/Closed: ClosedRelease: None
Effort: 0.00

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Mon 15 Aug 2016 05:39:34 AM UTC, comment #2:

I think this has been resolved now. So I'm closing as Fixed. If you think there are still issues outstanding, then please re-open it.

John Darrington <jmd>
Project Administrator
Fri 27 May 2016 04:24:10 PM UTC, comment #1:

Thanks for the bug report.

PSPP (unlike SPSS) does not show the "intercept" if the model is
unbalanced, since it has no meaning (just what calculation SPSS does in this case is a mystery).

The negative F value (and consequently NaN df) are more of a concern, possibly also related to missing values.

Also the speed issues concern me too.

I suggest we discuss this further on pspp-dev.

John Darrington <jmd>
Project Administrator
Fri 27 May 2016 02:57:26 PM UTC, original submission:

The attached data ("personality.sav") are three personality scores on a 50-item measure. They are moderately intercorrelated (0.35 - 0.51). There are a large number of missing values.

Using the attached data, I ran:
GLM agree_score BY caution_score extra_score

The processing took 6 minutes (compared to SPSS < 1 sec) and produced attached "glm_output.txt" that doesn't match SPSS and which is nonsensical in many places. For example, the tests for intercept are simply missing, the F values are all negative (which isn't possible), the p-values are all NaN, the error degrees of freedom are negative as are the error mean square.

I assume that a problem (not the only problem, see below) with these data are that there are many levels for the independent variables (about 50; they aren't really factors). This SHOULD NOT be a problem for GLM, but when I do a median split on the dependent variables:

recode caution_score (lo thru 35=1) (36 thru hi=2) (else=copy) into x1.
recode extra_score (lo thru 31=1) (32 thru hi=2) (else=copy) into x2.
execute.
GLM agree_score BY x1 x2.

This GLM runs very quickly but still produces incorrect output. For example, the df for x1 and x2 should be 1 (they have 2 levels, 2-1=1). The only way PSPP could be calculating df=2 is if it senses 3 levels which probably means that it's treating missing as a level, which is obviously incorrect. the two independent variables should still have significant effects on the dep. variable but the SS is calculated as 0.00. Also, the p-value for one effect is NaN, which shouldn't happen.

If the internal mechanics of PSPP's GLM cannot handle multiple levels then the routine should count the levels and refuse to run when the levels are too many (or, better yet, use an algorithm that doesn't fail because the "general linear model" shouldn't choke when you feed it continuous variables... that's what "general" in GLM means...).

Clearly there are several other serious bugs in the routine: (1) df are calculated incorrectly. (2) Under some circumstances, error SS is wrong. (3) When F=0 or F=1, significance values should be printed as 1.000 and 0.000, not as "NaN"

I ran this using an old version of PSPP on Windows 7 but I have since installed GNU pspp 0.10.1-g1082b8 and verified that I have identical results.

Alan Mead <amead>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #37288:  glm_output2.odt added by amead (21KiB - application/vnd.oasis.opendocument.text - the .ODT file contains both PSPP and SPSS output (the only way to past ethe SPSS output was as formatted text) and the SAV file was created by SPSS 24.)
file #37289:  personality.sav added by amead (5KiB - application/x-spss-sav - the .ODT file contains both PSPP and SPSS output (the only way to past ethe SPSS output was as formatted text) and the SAV file was created by SPSS 24.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jmd (Posted a comment)
  • -unavailable- added by amead (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 15 Aug 2016 05:39:34 AM UTCjmdStatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Fri 27 May 2016 02:57:26 PM UTCameadAttached File-=>Added glm_output2.odt, #37288
      Attached File-=>Added personality.sav, #37289

    Back to the top


    Powered by Savane 3.1-cleanup1