bugGNU Octave - Bugs: bug #43721, [octave forge] (statistics)...

 
 

bug #43721: [octave forge] (statistics) statistical CDF functions lack "upper" argument support

Submitted by:  Falk Tannhäuser <fata>
Submitted on:  Fri 28 Nov 2014 01:50:30 PM UTC  
 
Category:  Octave Forge Package Severity:  1 - Wish
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  fata Open/Closed:  Open
Release:  dev Operating System:  Any

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sat 21 Apr 2018 05:44:40 PM UTC, comment #10: 

All of the CDF functions in Octave have been moved to the Octave Forge statistics package, reassigning.

Mike Miller <mtmiller>
Project Administrator
Thu 12 Nov 2015 03:31:12 AM UTC, comment #9: 

I think this bug should be fixed generally.  It appears that just about all of the cumulative distribution functions in the Statistics Toolbox support the 'upper' argument.  I've re-titled the bug to reflect that.

Rik <rik5>
Project Administrator
Thu 11 Dec 2014 08:47:24 PM UTC, comment #8: 

Hi,

Here is a binocdf.m file for review, based on Falk Tannhäuser
implementation of the 'upper' option.

Changes:
1. "upper" was changed to 'upper' (better matlab compatability)
2. A few words of documentation on the 'upper' option
3. New tests

Please check this.

Thanks,

Avinoam 

(file #32651)

Avinoam Kalma <avinoam>
Project Member
Fri 05 Dec 2014 09:40:10 PM UTC, comment #7: 

If there is a complete changeset posted I can review it and push it to Mercurial.

Rik <rik5>
Project Administrator
Tue 02 Dec 2014 03:53:58 PM UTC, comment #6: 

Hello,
Well, for x >= n, binocdf(x, n, p) is exactly 1 and hence binocdf(x, n, p, "upper") is exactly 0. For x < n but x being "close" to n, binocdf(x, n, p) tends to be close to 1, especially if p is close to 0.
Conversely, for x < 0 we have exactly binocdf(x, n, p) == 0 and binocdf(x, n, p, "upper") == 1. For x >= 0 but close to 0 and p close to 1, binocdf(x, n, p) is close to 0 and binocdf(x, n, p, "upper") is close to 1 (possibly equal to 1 within machine precision).
I think it is best to always use betainc(p, x + 1, n - x) for the "upper" case and betainc(1 - p, n - x, x + 1) otherwise.
The user should call binocdf(x, n, p) if he is interested in the probability of a binomially distributed random variable being <= x and binocdf(x, n, p, "upper") for the probability of it being > x.

Falk

Falk Tannhäuser <fata>
Sun 30 Nov 2014 08:08:42 PM UTC, comment #5: 

Hi,

better than my quick & dirty patch :-)

We should add

## @deftypefnx {Function File} {} binocdf (@var{x}, @var{n}, @var{p}, "upper")

and some explanation about the "upper" paameter. I will try
to write something.

Is there a way to know in advance which of

betainc (p, x + 1, n - x)
betainc (1 - p, n - x, x + 1)

Has a better accuracy? We can use it also for the non-lower case.

Avinoam

Avinoam Kalma <avinoam>
Project Member
Sun 30 Nov 2014 01:18:17 PM UTC, comment #4: 

Well, that's a quick&dirty fix for someone needing just approximate Matlab compatibility; however it doesn't solve the problem of accuracy - results below 1e-16 will be wrongly returned as 0, which for many practical purposes is not acceptable.
I attach another version of binocdf.m where in case of "upper",

betainc (p, x + 1, n - x)

is called instead of

betainc (1 - p, n - x, x + 1)

which avoids this loss of accuracy.

For instance,

1 - binocdf(0:10, 10, 1e-5)

yields

[ 9.9996e-05   4.4998e-09   1.2002e-13   0.0000e+00   0.0000e+00   0.0000e+00   0.0000e+00   0.0000e+00   0.0000e+00   0.0000e+00   0.0000e+00 ]

while with my version

binocdf(0:10, 10, 1e-5, "upper")

yields

[ 9.9996e-05   4.4998e-09   1.1999e-13   2.0999e-18   2.5199e-23   2.0999e-28   1.2000e-33   4.4999e-39   9.9999e-45   1.0000e-50   0.0000e+00 ]

I also added a test case

%!assert (binocdf(99, 100, 0.1, "upper"), 1e-100, 1e-112)

Of course, binocdf() is just one among many other CDF and analogous fixes have to be applied for all the other xxxcdf functions listed on https://www.gnu.org/software/octave/doc/interpreter/Distributions.html#Distributions - that's a considerable amount of work!

(file #32575)

Falk Tannhäuser <fata>
Sat 29 Nov 2014 11:00:08 PM UTC, comment #3: 

Hi,

Please check the attached modified binocdf.m

If the 4th argument is "upper", the result is
1-cdf (X,N, p).

There is no documentation yet.
There is no treatment in special points.

Thanks,

Avinoam

(file #32571)

Avinoam Kalma <avinoam>
Project Member
Fri 28 Nov 2014 04:02:27 PM UTC, comment #2: 

Note however that CDF of other probability distributions (see https://www.gnu.org/software/octave/doc/interpreter/Distributions.html#Distributions for a list) are also concerned by this Matlab incompatibility / missing feature. Compare with the CDF in the "functions" sections of the individual probability distributions listed under http://fr.mathworks.com/help/stats/continuous-distributions.html or http://fr.mathworks.com/help/stats/discrete-distributions.html

Falk Tannhäuser <fata>
Fri 28 Nov 2014 02:39:25 PM UTC, comment #1: 

This function is Octave core so I'm adjusting the title to avoid confusion with the statistics package.

Carnë Draug <carandraug>
Project Member
Fri 28 Nov 2014 01:50:30 PM UTC, original submission:  

In Matlab the cumulative distribution functions (probability / statistics) accept a special argument string "upper" which causes the function to return the complement of the CDF - see for example http://fr.mathworks.com/help/stats/binocdf.html
For example

binocdf(x, N,p , 'upper')

is equivalent to

1 - binocdf(x, N, p)

except that extreme upper tail probabilities are calculated more accurately.
It seems desirable to have the same feature in Octave.

Falk Tannhäuser <fata>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #32651:  binocdf.m added by avinoam (4KiB - application/octet-stream)
file #32575:  binocdf.m added by fata (4KiB - application/octet-stream)
file #32571:  binocdf.m added by avinoam (3KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gyom
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by avinoam (Updated the item)
  • -email is unavailable- added by avinoam
  • -email is unavailable- added by carandraug (Arno Onken - maintainer of the statistics package)
  • -email is unavailable- added by fata (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-03-18 gyom Carbon-Copy- => Added gyom
    2018-04-21 mtmiller CategoryOctave Function => Octave Forge Package
        Summarystatistical CDF functions lack "upper" argument support => [octave forge] (statistics) statistical CDF functions lack "upper" argument support
    2015-11-12 rik5 Item GroupFeature Request => Matlab Compatibility
        StatusNone => Confirmed
        Summarybinocdf - cumulative distribution functions lack "upper" argument => statistical CDF functions lack "upper" argument support
    2014-12-11 avinoam Attached File- => Added binocdf.m, #32651
    2014-11-30 fata Attached File- => Added binocdf.m, #32575
    2014-11-29 avinoam Attached File- => Added binocdf.m, #32571
    2014-11-28 avinoam Carbon-Copy- => Added avinoam
    2014-11-28 carandraug Severity3 - Normal => 1 - Wish
        Item GroupMatlab Compatibility => Feature Request
        Release3.8.2 => dev
        Operating SystemMicrosoft Windows => Any
        SummaryStatistics: cumulative distribution functions lack "upper" argument => binocdf - cumulative distribution functions lack "upper" argument
        Carbon-Copy- => Added asnelt@asnelt.org

    Back to the top


    Powered by Savane 3.5