bugGNU Octave - Bugs: bug #42671, [PATCH] corr() does not have...

 
 

bug #42671: [PATCH] corr() does not have p-values output, returns 1.0 with one observation.

Submitted by:  Philipp Kutin <pkutin>
Submitted on:  Thu 03 Jul 2014 12:36:46 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Matlab Compatibility
Status: Patch SubmittedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: Any

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Fri 04 Jul 2014 12:03:24 PM UTC, comment #3:

One more update. The patch attached with this comment is from 'hg export' instead of my local MQ, er, stack and as such includes author info. I find MQ very convenient to cleanly separate my patches as I can't know whether/when they're going to be committed to Octave HG. In case anyone's interested, I keep my .hg/patches directory here: https://github.com/helixhorned/pk-octave-patches .

Anyhow, I think I've gotten the hang of the occasional contributor workflow:
- work locally on some source code
- save a new MQ patch
- hg qpop -a
- commit the topmost MQ patch to the real HG repo
- hg export, this one will be uploaded
- hg rollback
- hg qpush -a

This is hopefully my last noisy comment. Sorry!

(file #31672)

Philipp Kutin <pkutin>
Thu 03 Jul 2014 01:09:48 PM UTC, comment #2:

Alright... an updated patch is attached that supersedes the first one. Currently, Octave is incompatible to MATLAB in another way, not handled with this patch. In MATLAB, the input arguments are interpreted strictly as stated in the doc: columns are variables, rows are observations:

Octave, in contrast, makes an expection for vector input(s), always handling each one as one variable with as many observations as it has elements. This special case is not documented and may cause confusion. Personally, I prefer stricter semantics whenever possible.

(file #31671)

Philipp Kutin <pkutin>
Thu 03 Jul 2014 12:52:18 PM UTC, comment #1:

Oops, there are unhandled corner cases related to the ambiguity when passing row vector inputs and one '==' that should read '<='. Don't use yet.

Philipp Kutin <pkutin>
Thu 03 Jul 2014 12:36:46 PM UTC, original submission:

The current corr.m is behind MATLAB's in various ways. First, there's no p-values output with the second 'PVAL' outarg. Because of this, there's also no option as to which kind of alternative hypothesis to consider ('both', 'left' or 'right').

The attached patch adds the PVAL output for the both-sided case. As stated in the MATLAB docs, a transformation from r to values that are t-distributed (assuming the input variables are uncorrelated bivariate Gaussian) is used there.

Additionally, when correlating data sets with one observation, return NaN instead of 1 -- the Pearson correlation coefficient is not defined in this case since the variance of either variable isn't.

Patch message:

Future directions:
For corr() to accept key/value pairs like 'KIND' it would be nice to have a factored system to extract these from a varargin passed to a function. Searching for the K/V pattern in the Octave code, it seems like these are done by hand each time now.
The remaining measures of assiciation -- spearman() and kendall() -- are there, so dispatching to those could then be done from corr(), too. Estimating p-values for them is a different story.

Tests on MATLAB R2013a:

Philipp Kutin <pkutin>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #31672:  corr-pval-3.patch added by pkutin (2kB - text/x-patch)
file #31671:  corr-pval-2.patch added by pkutin (2kB - text/x-patch)
file #31670:  corr-pval-1.patch added by pkutin (2kB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Updated the item)
  • -unavailable- added by pkutin (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Fri 04 Jul 2014 12:03:24 PM UTCpkutinAttached File-=>Added corr-pval-3.patch, #31672
    Thu 03 Jul 2014 02:34:54 PM UTCmtmillerStatusNone=>Patch Submitted
    Thu 03 Jul 2014 01:09:48 PM UTCpkutinAttached File-=>Added corr-pval-2.patch, #31671
    Thu 03 Jul 2014 12:36:46 PM UTCpkutinAttached File-=>Added corr-pval-1.patch, #31670

    Back to the top


    Powered by Savane 3.1-cleanup