bugGNU Octave - Bugs: bug #41008, empirical_pdf incorrect...

 
 

bug #41008: empirical_pdf incorrect implementation

Submitted by:  None
Submitted on:  Fri 27 Dec 2013 12:24:57 AM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: FixedAssigned to: None
Originator Name: HPOriginator Email: -unavailable-
Open/Closed: ClosedRelease: 3.6.4
Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 23 Jul 2014 08:38:10 AM UTC, comment #6:

A yet terser version with a more complete, rounding-proof unit test for ties.

(file #31760)

Alexander Klein <matalex>
Wed 23 Jul 2014 08:15:10 AM UTC, comment #5:

By the way, bug 39280 is basically a duplicate of this one.

Alexander Klein <matalex>
Wed 23 Jul 2014 08:08:24 AM UTC, comment #4:

I just came across this bug report after I'd fixed the issue myself. In fact, unique already does all the necessary counting, so there's no need to do it all over again by calling histc, which seems to be a function optimised for generality rather than speed.

Maybe you'd like to consider the attached file as an alternative solution.

(file #31759)

Alexander Klein <matalex>
Thu 02 Jan 2014 09:25:32 PM UTC, comment #3:

I changed the calculation to account for repeat values in the input data in this changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/1461b9cfac4e). It will be a part of the 4.2 release. If you need it sooner you can just grab the file empirical_pdf.m from Mercurial.

Rik <rik5>
Project Administrator
Fri 27 Dec 2013 05:30:20 PM UTC, comment #2:

If you check discrete_pdf you'll find that it normalizes the probability so there is no need to divide by the length of the data in empirical_pdf.m. The all ones vector is just a convenient way to assume that each value in the empirical distribution is equally likely.

There is an issue if there are duplicates in the data that determines the empirical pdf. There are several different strategies to tackle this. The simplest is just to check for any duplicates at the start of the function and abort with a warning message if any are found. Something like this would work

Alternatively, we could use the current code when all elements are unique and try to code up a solution for duplicate elements.

Rik <rik5>
Project Administrator
Fri 27 Dec 2013 02:28:33 AM UTC, comment #1:

would a call to unique() remedy the situation?

Felipe G. Nievinski <fgnievinski>
Fri 27 Dec 2013 12:24:57 AM UTC, original submission:

Line #40 of empirical_pdf.m reads
pdf = discrete_pdf (x, data, ones (size (data)));

However, the function discrete_pdf requires that the last argument
be a vector of probabilities.
This can be remedied as follows:

pdf = discrete_pdf (x, data, (ones (size (data))/length(data)));

More basically, however, this implementation does not consider the possibility of ties in the data. In other words, the implementation is limited to data with distinct observations.
I suggest this should be addressed.

Best
-HP

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #31760:  empirical_pdf.m added by matalex (2KiB - application/octet-stream)
file #31759:  empirical_pdf.m added by matalex (2KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by matalex (Updated the item)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by fgnievinski (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 23 Jul 2014 08:38:10 AM UTCmatalexAttached File-=>Added empirical_pdf.m, #31760
    Wed 23 Jul 2014 08:08:24 AM UTCmatalexAttached File-=>Added empirical_pdf.m, #31759
    Thu 02 Jan 2014 09:25:32 PM UTCrik5StatusNone=>Fixed
      Open/ClosedOpen=>Closed

    Back to the top


    Powered by Savane 3.1-cleanup1