bugGNU Octave - Bugs: bug #41008, empirical_pdf incorrect...

 
 

bug #41008: empirical_pdf incorrect implementation

Submitter:  None
Submitted:  Fri 27 Dec 2013 12:24:57 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  HP Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 3.6.4
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 23 Jul 2014 08:38:10 AM UTC, comment #6: 

A yet terser version with a more complete, rounding-proof unit test for ties.

(file #31760)

Alexander Klein <matalex>
Wed 23 Jul 2014 08:15:10 AM UTC, comment #5: 

By the way, bug 39280 is basically a duplicate of this one.

Alexander Klein <matalex>
Wed 23 Jul 2014 08:08:24 AM UTC, comment #4: 

I just came across this bug report after I'd fixed the issue myself. In fact, unique already does all the necessary counting, so there's no need to do it all over again by calling histc, which seems to be a function optimised for generality rather than speed.

Maybe you'd like to consider the attached file as an alternative solution.

(file #31759)

Alexander Klein <matalex>
Thu 02 Jan 2014 09:25:32 PM UTC, comment #3: 

I changed the calculation to account for repeat values in the input data in this changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/1461b9cfac4e).  It will be a part of the 4.2 release.  If you need it sooner you can just grab the file empirical_pdf.m from Mercurial.

Rik <rik5>
Group administrator
Fri 27 Dec 2013 05:30:20 PM UTC, comment #2: 

If you check discrete_pdf you'll find that it normalizes the probability so there is no need to divide by the length of the data in empirical_pdf.m.  The all ones vector is just a convenient way to assume that each value in the empirical distribution is equally likely.

There is an issue if there are duplicates in the data that determines the empirical pdf.  There are several different strategies to tackle this.  The simplest is just to check for any duplicates at the start of the function and abort with a warning message if any are found.  Something like this would work


if (numel (data) != numel (unique (data)))
  error ("empirical_pdf: DATA has duplicate observed values");
endif


Alternatively, we could use the current code when all elements are unique and try to code up a solution for duplicate elements.

Rik <rik5>
Group administrator
Fri 27 Dec 2013 02:28:33 AM UTC, comment #1: 

would a call to unique() remedy the situation?

Felipe G. Nievinski <fgnievinski>
Fri 27 Dec 2013 12:24:57 AM UTC, original submission:  

Line #40 of empirical_pdf.m reads
pdf = discrete_pdf (x, data, ones (size (data)));

However, the function discrete_pdf requires that the last argument
be a vector of probabilities.
This can be remedied as follows:

pdf = discrete_pdf (x, data, (ones (size (data))/length(data)));

More basically, however, this implementation does not consider the possibility of ties in the data. In other words, the implementation is limited to data with distinct observations.
I suggest this should be addressed.

Best
-HP

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #31760:  empirical_pdf.m added by matalex (2KiB - application/octet-stream)
file #31759:  empirical_pdf.m added by matalex (2KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by matalex (Updated the item)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by fgnievinski (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-07-23 matalex Attached File- Added empirical_pdf.m, #31760
    2014-07-23 matalex Attached File- Added empirical_pdf.m, #31759
    2014-01-02 rik5 StatusNone Fixed
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code