bugPSPP - Bugs: bug #26936, T-TEST reports 2-p instead of p

 
 

bug #26936: T-TEST reports 2-p instead of p

Submitted by:  Ben Pfaff <blp>
Submitted on:  Wed 01 Jul 2009 10:35:30 PM UTC  
 
Category: Numerical ErrorsSeverity: 7 - Major
Status: FixedAssigned to: None
Open/Closed: ClosedRelease: None
Effort: 0.00

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 15 Jul 2009 06:47:03 PM UTC, comment #6:

OK. Thanks for clearing this up. There was indeed a rather
embarrassing bug in the paired samples test. I've pushed a fix to
the stable branch.

(For any future bug reports, you can easily generate syntax from
the gui, by using the "Paste" button. This makes it easier to
report the exact problem.)

John Darrington <jmd>
Project Administrator
Wed 15 Jul 2009 04:07:23 PM UTC, comment #5:

John

Thanks for your reply. I think there are a couple of misunderstandings here.

1. If you have a choice between results assuming equal variances and non-equal variances, you must be running an independent-samples test. My point related to a paired-samples test (Analyse - Compare Means - Paired Samples T test).
2. Neither of my data sets had only one member. (For a paired-samples test they have to be of equal size anyway.) In column A the values, reading downwards, were 11, 1 and 1. In column B the values, reading downwards, were 2, 1 and 1.
3. I am not using syntax, I am using the drop-down menu.
4. In view of all the above I assume you don't need a different example, but I can willingly provide others if needed.
5. I am running PSPP (psppire.exe) 0.7.2 under Windows XP.

Hope this helps,
Mike

Mike Griffiths <mikegriffiths>
Wed 15 Jul 2009 06:21:37 AM UTC, comment #4:

Okay - thanks for clarifying the problem.

When I run the example you gave, I get a T value of -Infinity for the Equal Variances case, and NaN for the Unequal Variances case. Consequently all other figures are NaN (except the Mean Difference).

I'm rather intrigued that you get a finite T-Value at all. What platform are you running on?

The results I'm seeing are probably not strictly correct, but are understandable since, as one of the groups has only 1 member, the variance is zero - and division by zero is undefined.

But I'm concerned when you say that "other paired-samples tests" are giving the wrong answer too. Do any of these other examples
have group sizes larger than 1 ? If so, can you post example syntax files which demonstrate the problem.

John Darrington <jmd>
Project Administrator
Tue 14 Jul 2009 07:23:36 PM UTC, comment #3:

>Unfortunately the person who originally logged my item
>incorrectly summarised it in the title. The p-value calculated by
>PSPP is not 2*p, but 2-p.


Sorry about that mistake. I updated the title.

Ben Pfaff <blp>
Project Administrator
Tue 14 Jul 2009 07:06:09 PM UTC, comment #2:

I'm afraid the answer given is incorrect. Furthermore, I think this is a serious error in PSPP.

Unfortunately the person who originally logged my item incorrectly summarised it in the title. The p-value calculated by PSPP is not 2*p, but 2-p. This is nothing to do with the difference between one and two tailed values. If you correctly calculate the p-value from the inverse CDF you can never come up with a p-value greater than 1, whether 1 or 2 tailed. (The circumstances when you are likely to calculate a p-value greater than 1 are when you are doing corrections for multiple comparisons, and certainly the value then should be capped at 1.00. Probability can never be more than 100%.)

To reiterate what I originally reported. My test data were: variable A = 11, 1, 1. Variable B = 2, 1, 1. The correct p-value for a paired-samples t-test (from SPSS) is .423 2-tailed, hence .212 1-tailed. The p-value reported by PSPP is 1.58 2-tailed.

Mike Griffiths <mikegriffiths>
Tue 14 Jul 2009 12:41:39 PM UTC, comment #1:

T-Test (in common with most PSPP commands) displays the 2-tailed significance, which is twice the 1-tailed significance. So the
reported p-values are correct. - Perhaps it should be clamped to a
maximum of 1.00 ...

John Darrington <jmd>
Project Administrator
Wed 01 Jul 2009 10:35:30 PM UTC, original submission:

Mike Griffiths <mike_g50@hotmail.com> reports, at http://lists.gnu.org/archive/html/bug-gnu-pspp/2009-06/msg00026.html:

> 3. I entered the following (admittedly nonesense) data and called up a
> paired-samples t-test. PSPP gave the correct value of t (1.00) but for the
> p-value gave 1.577, which is bigger not only than the correct figure but than
> the maximum theoretically possible p-value (which is 1). (A little
> experimenting with other paired-samples tests suggests that the programme is
> reporting 2-p, where p is the correct value.)
>
> a b
> 11 2
> 1 1
> 1 1

Ben Pfaff <blp>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mikegriffiths (Posted a comment)
  • -unavailable- added by jmd (Posted a comment)
  • -unavailable- added by blp (Submitted the item)
  • -unavailable- added by blp (reporter)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 15 Jul 2009 06:47:03 PM UTCjmdSeverity3 - Ordinary=>7 - Major
      StatusNone=>Fixed
      Open/ClosedOpen=>Closed
    Tue 14 Jul 2009 07:23:36 PM UTCblpSummaryT-TEST reports 2*p instead of p=>T-TEST reports 2-p instead of p
    Tue 14 Jul 2009 12:41:38 PM UTCjmdSeverity5 - Average=>3 - Ordinary
    Wed 01 Jul 2009 10:35:30 PM UTCblpCarbon-Copy-=>Added mike griffiths

    Back to the top


    Powered by Savane 3.1-cleanup1