# bug #44130: [octave forge] (statistics) mvncdf algorithm needs performance and accuracy improvement

 Submitter: Pham Submitted: Fri 30 Jan 2015 06:29:28 AM UTC Category: Octave Forge Package Severity: 1 - Wish Priority: 5 - Normal Item Group: Inaccurate Result Status: Fixed Assigned to: None Originator Name: Open/Closed:  * Closed Release:  * other Operating System:  * Any Fixed Release: None Planned Release: None
* Mandatory Fields

Comment Type & Canned Response:

## Discussion

Thu 18 Aug 2022 10:34:27 PM UTC, comment #4:

A new implementation of mvncdf has been pushed at https://github.com/gnu-octave/statistics/commit/3578095d0a867442634353f66c0a5c391bcb11bd

The accuracy has been improved greatly and it is in par with Matlab. Moreover, the calculation of the bivariate normal cdf is adapted from earlier work by Thomas H. Jørgensen (with his permission) and it produces more accurate results than Matlab built-in function.

This can be closed.

Andreas Bertsatos <pr0m1th3as>
Tue 23 Nov 2021 05:50:27 PM UTC, comment #3:

actually trying other matlab doc examples i get the following:

Octave 6.4.0, statistics 1.4.2

>> firstDim = (-2:2)';
>> X = repmat(firstDim,1,4)
X =

-2  -2  -2  -2
-1  -1  -1  -1
0   0   0   0
1   1   1   1
2   2   2   2

>> p = mvncdf(X)
p =

0
0
0
0
0

Matlab 2021a:

>> firstDim = (-2:2)';
>> X = repmat(firstDim,1,4)

X =

-2    -2    -2    -2
-1    -1    -1    -1
0     0     0     0
1     1     1     1
2     2     2     2

>> p = mvncdf(X)
p =

0.0000
0.0006
0.0625
0.5011
0.9121

so maybe it's a bit more than an accuracy issue?

Nicholas Jankowski <nrjank>
Tue 23 Nov 2021 05:47:13 PM UTC, comment #2:

The differences are confirmed, but so is the explanation. The current implementation of mvncdf uses a method with inherent variability in output within the tolerances allowed.  It is already extremely slower than the matlab equivalent (the comment #0 test code takes ~5000x longer in octave, i tried the multidimensional matlab example that matlab solved in a millisecond and thought octave had crashed), and even if tightening the tolerances would provide acceptable convergent levels of discrepancy, the performance penalty would likely be extreme.

Noting that octave has adaptive quadrature integration routines available, the best think is probably to rework this function for performance improvement.  Retitling and dropping severity, but leaving it open.

Nicholas Jankowski <nrjank>
Wed 22 Jul 2015 06:02:09 PM UTC, comment #1:

Hi Pham,

I believe this is not a bug but a matter of precision.

The function always(*) uses a Monte Carlo method to approximate the integrals, that runs until error is smaller than a predefined precision of 1e-3. Look at the variable 'error'

[p,error] = mvncdf(x, mu, sigma).

best, Andrej

(*) unlike Matlab that uses adaptive quadrature for 2- and 3- dimensional data. It would be nice to have this in Octave, too, since it is faster than Monte Carlo.

Andrej <mangosauce35>
Fri 30 Jan 2015 06:29:28 AM UTC, original submission:

Hi all,
I've just "discovered" that executing mvncdf several times with the exact same input parameters yields different numerical results.

I expected that the function yield exactly the same result.

Here is a minimal working example (from the doc) :

x = [1 2];
mu = [0.5 1.5];
sigma = [1.0 0.5; 0.5 1.0];
p1 = mvncdf (x, mu, sigma)
p2 = mvncdf (x, mu, sigma)

The outputs are :

p1 = 0.54658
p2 = 0.54624

I get other (different) results when exiting and restarting Octave.

Computer setup :
Xubuntu 14.04.1 LTS x86_64
octave from official repos

Pham <dkgpoctave>

## Attached Files

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Comment:

No files currently attached

## Dependencies

Depends on the following items: None found

Items that depend on this one: None found

Carbon-Copy List
• -email is unavailable- added by pr0m1th3as (Posted a comment)
• -email is unavailable- added by nrjank (Posted a comment)
• -email is unavailable- added by jwe (Updated the item)
• -email is unavailable- added by mangosauce35 (Posted a comment)
• -email is unavailable- added by dkgpoctave (Submitted the item)
•

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only project members can vote.

## History

Date Changed by Updated Field Previous Value => Replaced by
2022-08-19 nrjank StatusConfirmed Fixed
Open/ClosedOpen Closed
2021-11-23 nrjank Severity3 - Normal 1 - Wish
StatusNone Confirmed
Operating SystemGNU/Linux Any
Summary[octave forge] (statistics) mvncdf outputs different results for same input parameters [octave forge] (statistics) mvncdf algorithm needs performance and accuracy improvement
2018-04-27 mtmiller Carbon-CopyRemoved 80942 -
2018-04-27 mtmiller Release3.8.1 other
2017-08-13 jwe Summarymvncdf outputs different results for same input parameters [octave forge] (statistics) mvncdf outputs different results for same input parameters