bugGNU Octave - Bugs: bug #44130, [octave forge] (statistics) mvncdf...

 
 

bug #44130: [octave forge] (statistics) mvncdf algorithm needs performance and accuracy improvement

Submitter:  Pham <dkgpoctave>
Submitted:  Fri 30 Jan 2015 06:29:28 AM UTC
   
 
Category:  Octave Package Severity:  1 - Wish
Priority:  5 - Normal Item Group:  Inaccurate Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 18 Aug 2022 10:34:27 PM UTC, comment #4: 

A new implementation of mvncdf has been pushed at https://github.com/gnu-octave/statistics/commit/3578095d0a867442634353f66c0a5c391bcb11bd

The accuracy has been improved greatly and it is in par with Matlab. Moreover, the calculation of the bivariate normal cdf is adapted from earlier work by Thomas H. Jørgensen (with his permission) and it produces more accurate results than Matlab built-in function.

This can be closed.

Andreas Bertsatos <pr0m1th3as>
Tue 23 Nov 2021 05:50:27 PM UTC, comment #3: 

actually trying other matlab doc examples i get the following:

Octave 6.4.0, statistics 1.4.2

>> firstDim = (-2:2)';
>> X = repmat(firstDim,1,4)
X =

  -2  -2  -2  -2
  -1  -1  -1  -1
   0   0   0   0
   1   1   1   1
   2   2   2   2

>> p = mvncdf(X)
p =

   0
   0
   0
   0
   0


Matlab 2021a:

>> firstDim = (-2:2)';
>> X = repmat(firstDim,1,4)

X =

    -2    -2    -2    -2
    -1    -1    -1    -1
     0     0     0     0
     1     1     1     1
     2     2     2     2

>> p = mvncdf(X)
p =

    0.0000
    0.0006
    0.0625
    0.5011
    0.9121


so maybe it's a bit more than an accuracy issue?

Nicholas Jankowski <nrjank>
Group Member
Tue 23 Nov 2021 05:47:13 PM UTC, comment #2: 

The differences are confirmed, but so is the explanation. The current implementation of mvncdf uses a method with inherent variability in output within the tolerances allowed.  It is already extremely slower than the matlab equivalent (the comment #0 test code takes ~5000x longer in octave, i tried the multidimensional matlab example that matlab solved in a millisecond and thought octave had crashed), and even if tightening the tolerances would provide acceptable convergent levels of discrepancy, the performance penalty would likely be extreme.

Noting that octave has adaptive quadrature integration routines available, the best think is probably to rework this function for performance improvement.  Retitling and dropping severity, but leaving it open.


Nicholas Jankowski <nrjank>
Group Member
Wed 22 Jul 2015 06:02:09 PM UTC, comment #1: 

Hi Pham,

I believe this is not a bug but a matter of precision.

The function always(*) uses a Monte Carlo method to approximate the integrals, that runs until error is smaller than a predefined precision of 1e-3. Look at the variable 'error'
 
[p,error] = mvncdf(x, mu, sigma).


best, Andrej 

(*) unlike Matlab that uses adaptive quadrature for 2- and 3- dimensional data. It would be nice to have this in Octave, too, since it is faster than Monte Carlo.

Andrej <mangosauce35>
Fri 30 Jan 2015 06:29:28 AM UTC, original submission:  

Hi all,
I've just "discovered" that executing mvncdf several times with the exact same input parameters yields different numerical results.

I expected that the function yield exactly the same result.

Here is a minimal working example (from the doc) :

x = [1 2];
mu = [0.5 1.5];
sigma = [1.0 0.5; 0.5 1.0];
p1 = mvncdf (x, mu, sigma)
p2 = mvncdf (x, mu, sigma)


The outputs are :

p1 = 0.54658
p2 = 0.54624


I get other (different) results when exiting and restarting Octave.

Computer setup :
Xubuntu 14.04.1 LTS x86_64
octave from official repos

Pham <dkgpoctave>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pr0m1th3as (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by jwe (Updated the item)
  • -email is unavailable- added by mangosauce35 (Posted a comment)
  • -email is unavailable- added by dkgpoctave (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-08-19 nrjank StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2021-11-23 nrjank Severity3 - Normal 1 - Wish
        StatusNone Confirmed
        Operating SystemGNU/Linux Any
        Summary[octave forge] (statistics) mvncdf outputs different results for same input parameters [octave forge] (statistics) mvncdf algorithm needs performance and accuracy improvement
    2018-04-27 mtmiller Carbon-CopyRemoved 80942 -
    2018-04-27 mtmiller Release3.8.1 other
    2017-08-13 jwe Summarymvncdf outputs different results for same input parameters [octave forge] (statistics) mvncdf outputs different results for same input parameters

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code