bugGNU Octave - Bugs: bug #31474, autocor.m & autocov.m are wrong

 
 

bug #31474: autocor.m & autocov.m are wrong

Submitter:  None
Submitted:  Wed 27 Oct 2010 02:01:42 AM UTC
   
 
Category:  Libraries Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Wont Fix Assigned to:  None
Originator Name:  Reginald Beardsley Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 3.2.4
Operating System:  * BSD Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 29 Oct 2010 01:21:13 AM UTC, comment #9: 

Amen!

My apologies to all for being so cranky.  I'm fighting a respiratory infection I picked up at a conference last week and got blindsided by the multiplicity of definitions. Not that it excuses being rude.

I would like to urge that periodogram.m also be deprecated as it has the same issue as autocov.m and autocor.m and since it's nothing more than the Fourier transform of the autocorrelation, however defined, not of any real benefit.  It's actually a very poor way to perform a spectral estimate as the variance does not decrease with increasing sample length.

However, the real problem is the multiplicity of definitions. Box & Jenkins and company subtract the mean whereas Oppenheim & Schafer and company do not.  So it is another pit fall for the unwary in an area which is quite difficult enough already.

Thank you.

Have Fun!
Reg

Reginald Beardsley <rhb>
Fri 29 Oct 2010 12:42:43 AM UTC, comment #8: 

After doing some research, I find there is very little definitional certainty about how to calculate the cross covariance (and hence the autocovariance).  As you have noted, in the field of statistics it tends to be calculated one way, and in signal processing in a slightly different manner.  Even with mostly similar algorithms there is an abundance of normalization factors which can be applied.

To illustrate the ridiculousness of it all, Matlab, can calculate the cross-covariance in no less than 4 different manners (http://www.mathworks.com/help/toolbox/signal/xcov.html).  For Matlab, the default option, 'none', produces what you were expecting.  Octave uses the equivalent of the 'unbiased' option.  Octave is not wrong, just different.

After some discussion on the octave-maintainers mailing list, it was decided that these two functions do not really belong in the core of Octave.  They are not needed by the general user, and the choice of a particular method for calculating the cross covariance relies on the user's understanding of their dataset and interests.  The signal package from Octave-Forge contains the two functions xcorr and xcov which reproduce the Matlab functions and their variety of normalizations.  I suggest using those in place of autocor and autocov so that you may explicitly choose which algorithm normalization you desire.

Since NOW, these functions are deprecated.  A warning message will be displayed when they are used for the first time.  The documentation will now explain the particular calculation that autocov and autocorr use, and will also point to the Octave Forge signal packages.  They will be removed in Octave 3.6.

Rik <rik5>
Group administrator
Thu 28 Oct 2010 04:57:30 PM UTC, comment #7: 

John,

Out of curiosity I did some research.  It turns out I was both right and wrong at the same time.

Friedrich Leisch is a statistician.  In one of the perverse accidents of mathematics, the statisticians have very different definitions for autocorrelation and autocovariance than commonly used in the signal processing community.  While I have a few statistically oriented texts, they are not where I learned the mathematics.  If I was ever aware of the discrepancy, it was forgotten long ago.

The presence of autocor.m and autcov.m in the signal directory led me to expect a signal processing discipline definition as presented by:

Random Data
Bendat & Piersol

Digital Signal Processing
Oppenheim & Schaefer

Theory and Application of Digital Signal Processing
Rabiner & Gold

The Fourier Transform & its Applications
Bracewell

The Fourier Integral & its Applications
Papoulis

and many more.

In all of these, the autocorrelation is the inverse Fourier transform of the power spectrum and the autocovariance is the inverse Fourier transform of the power spectrum w/ the DC component set to zero.  If the peak values are normalized to 1 it is explicitly described as "normalized".

However, Box & Jenkins in "Time Series Analysis" defined the autocorrelation and autocovariance as implemented by Friedrich Leisch.

Rather than using my "corrected" versions, I'd like to suggest that the documentation for autocor(), autocov() and periodogram()  explicitly  note that they implement the Box & Jenkins definitions rather than the Oppenheim & Schaefer/Bendat & Piersol definitions.

All of Leisch's contributions probably belong in the statistics directory, however, most are unlikely to be used by anyone other than a statistician, so there is little risk of confusion.

Have Fun!
Reg

Reginald Beardsley <rhb>
Wed 27 Oct 2010 09:26:34 PM UTC, comment #6: 

John,

Not meant as an insult.  Curt yes.  I'd wasted much of a day already.  Please remember, I didn't just complain.  I fixed it and provided a reference and a test case.  I would have provided the test case at the start, but I thought the issue was obvious.

I did not say it was a failure to understand basic math.  I said it was a failure to understand "the math". It's a basic operation in signal processing and the analysis of random data, but I would not describe it as "basic math".  It may have arisen from excessive enthusiasm for programming or from problems translating English language terminology into German.  The author's English is certainly better than my German.  However, nothing changes the fact that the functions were seriously wrong and that a quick inquiry w/ google found it had been accurately reported long ago and another comparison showed it had not been corrected in the most recent version of source I had.

For condescension, I plead guilty.  I got subjected to a "you must answer this question even if the answer you give is wrong" interface.  Obviously the person who coded that didn't consider that there are more operating systems than are on the list.  I've worked w/ over a dozen flavors of Unix and another dozen or so non-Unix operating systems.  I suggest adding "other" to the list if you control it.  Even MVS is Posix compliant and should run Octave.  In any case, combining pedanticism and ignorance will pretty much guarantee the same response from me every time.

I'm actually much more concerned about the general direction that Octave has taken over the last 5-10 years.  I don't particularly care about matching Matlab syntax & semantics.  It's a nice convenience, but hardly of any real consequence from my perspective.  I use Octave as a prototyping tool to verify I understand the math before I implement a major code.  For me reliability is paramount. A tool that forces me to devise test cases to verify the functions I use is of no help.  That's the main reason not to code it myself.  The test cases take considerable time.

I've still not been able to compile anything later than 2.1.73 w/ more than partial success despite spending most of a day on the task on more than one occasion.  I plan to try again soon while I still remember the issues I encountered. The chain of dependencies on non-standard conforming language features and other packages becomes very arduous, especially when autoconf gets it wrong or the dependencies make different choices (e.g. 32 vs 64 bit).  I could avoid the problem by firing up a Gnu/Linux machine, but I don't particularly like Gnu/Linux.  There are too many gratuitous changes to old programs that break things.  So I'd pretty much be trading one set of problems for another.

I'm willing to endure some pretty arduous requirements to compile Octave if it's reliable.  However, there's a lot of arcane math in it and that becomes a concern if there is not a regression test suite to verify that they are correct.  I could write tests for a significant portion, but nowhere near all and it is a Herculean task as a whole.  Many years ago writing a test suite for Octave was a major task.   Now it's mind boggling.  The only solution I know of is to write a test case every time a new function is added.  I'm facing the same issues w/ several other large, complex packages.  I don't have a good answer.  It's a hard problem.  If you've got ideas, I'd like to discuss them.

Have Fun!
Reg

Reginald Beardsley <rhb>
Wed 27 Oct 2010 05:05:32 PM UTC, comment #5: 

There's no need to insult us (failure to understand basic math) or be condescending ("it may come as a shock").

John W. Eaton <jwe>
Group administrator
Wed 27 Oct 2010 01:40:26 PM UTC, comment #4: 

Here's the jpeg compressed a bit more


Reginald Beardsley <rhb>
Wed 27 Oct 2010 01:37:49 PM UTC, comment #3: 

Lest there be any doubt about my assertion that they were wrong, the reference cited in my comment in the corrected scripts is attached.

This really isn't a bug.  It's a failure to understand the math.  Which given the very basic nature of the operations is difficult to comprehend.

rhb

Reginald Beardsley <rhb>
Wed 27 Oct 2010 02:37:39 AM UTC, comment #2: 

The old bug report:

   http://osdir.com/ml/gnu.octave.bugs/2004-01/msg00055.html

Listed:

   The Savanah bug submission form which forced me to list the wrong OS.

Test case:

   Attached.

(file #21841)

Reginald Beardsley <rhb>
Wed 27 Oct 2010 02:06:34 AM UTC, comment #1: 

What bug?  What is the error?  How about providing a test case that demonstrates the bug, or a direct link to the old bug report that you claim was not fixed?

"BTW it may come as a shock, but there are more flavors of Unix than those listed. I'm using Solaris 10 x86."

What do you mean by "those listed"?  Listed where?

John W. Eaton <jwe>
Group administrator
Wed 27 Oct 2010 02:01:42 AM UTC, original submission:  

autocor was reported to the bug mailing list a long time ago against 2.1.50.  The bug is still present in 3.2.4. In addition, autocov.m is wrong.

I've attached corrected versions, but suggest checking them against the info description to make certain I didn't miss something as it's been a while since I used octave.

I find it rather scary that something so basic could be wrong or  go uncorrected for so long.

rhb

BTW it may come as a shock, but there are more flavors of Unix than those listed.  I'm using Solaris 10 x86.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #21844:  IMG_3854.JPG added by rhb (260KiB - image/jpeg)
file #21841:  fubar.m added by rhb (116B - text/x-objcsrc)
file #21839:  autocor.m added by None (2KiB - text/x-objcsrc)
file #21840:  autocov.m added by None (2KiB - text/x-objcsrc)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by rhb (Updated the item)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-10-29 rik5 StatusNone Wont Fix
        Open/ClosedOpen Closed
    2010-10-27 rhb Attached File- Added IMG_3854.JPG, #21844
    2010-10-27 rhb Attached File- Added fubar.m, #21841
    2010-10-27 None Attached File- Added autocor.m, #21839
        Attached File- Added autocov.m, #21840

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code