bugGNU Octave - Bugs: bug #60076, [octave forge] (signal) xcorr bug...

 
 

bug #60076: [octave forge] (signal) xcorr bug for large datasets

Submitter:  Mark van Rossum <cosine>
Submitted:  Thu 18 Feb 2021 05:50:57 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 5.2.0 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 20 Feb 2021 03:33:03 PM UTC, comment #10: 

OK, thanks, closing with "Fixed" (maybe not the best status Id)

Philip Nienhuis <philipnienhuis>
Group Member
Sat 20 Feb 2021 10:02:20 AM UTC, comment #9: 

I guess we can close it.

In retrospect this is/was an Octave FFT bug.

Mark van Rossum <cosine>
Fri 19 Feb 2021 10:05:58 PM UTC, comment #8: 

5.2.0 has been superseded by 6.1.0 anyway and 6.2.0 seems to be in the making right now so if it works for you with 6.1.0 I suppose this bug report can be closed.
Would you agree?

Philip Nienhuis <philipnienhuis>
Group Member
Fri 19 Feb 2021 12:42:53 PM UTC, comment #7: 

I have compiled Octave 6.1.0.
No bug there, so that is a work-around.

Mark van Rossum <cosine>
Fri 19 Feb 2021 12:37:04 PM UTC, comment #6: 

Try to self-compile fftw and LD_PRELOAD this library prior to
starting octave.

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 19 Feb 2021 12:21:59 PM UTC, comment #5: 

Thank you Dmitri, I thought I was going crazy.
I see the bug on Ubuntu 20.04, but not on a different system with Debian 10.

Here is the core of the xcorr.m code.
 
%%%% xcorr.m code:
M = 2^nextpow2(N + maxlag)
post = fft( postpad(X(:),M) );
cor = ifft( post .* conj(post) );
R = [ conj(cor(maxlag+1:-1:2)) ; cor(1:maxlag+1) ];

Whenever the data length surpasses 4096, the ftt behaves oddly.
R gets large imaginary components, which should not happen.


Mark van Rossum <cosine>
Fri 19 Feb 2021 11:48:04 AM UTC, comment #4: 

We saw some other reports of fft behaving weird:

 https://octave.discourse.group/t/fft-function-random-results-with-n-signal-length/641

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 19 Feb 2021 11:39:54 AM UTC, comment #3: 

Hi
Thanks for looking into this and sorry for not attaching a plot.
I attach it now.

Very oddly, for the longer time-series the plot changes from run to run.

The xcorr function uses a Fourier transform, and perhaps there is a rounding bug in some versions of that library.
What system do you use?

This is on a fresh (reinstall) of Octave and "pkg -forge install signal".
"which xcorr" returns
'xcorr' is a function from the file /usr/share/octave/packages/signal-1.4.1/xcorr.m



Mark van Rossum <cosine>
Fri 19 Feb 2021 11:08:18 AM UTC, comment #2: 

I'm unsure about what you refer to, you didn't supply the faulty plots you mentioned.
With Octave-5.2.0 & signal-1.4.1 and Octave-7.0.0 (dev) also with signal-1.4.1 (admittedly both on Windows) I get the attached plot for sightly amended plot code; I don't see any difference between the underlying xcorr results.

a=mod(1:1000,2);               % 0,1,0,1,0,.....
plot(xcorr(a,a,20,'unbiased')) % looks fine

a=mod(1:10000,2);  % longer time-series
hold on
plot(xcorr(a,a,20,'unbiased') + 1, 'g') % same except for values + 1


Also the outputs o the .m file you supplied don't look suspicious to me.

Would you maybe have some stray xcorr.m in your path?
just type 'which xcorr' (w/o quotes)

(status -> Need Info)


Philip Nienhuis <philipnienhuis>
Group Member
Thu 18 Feb 2021 06:38:20 PM UTC, comment #1: 

Even simpler example highlighting that the data-size matters:

a=mod(1:1000,2);               % 0,1,0,1,0,.....
plot(xcorr(a,a,20,'unbiased')) % looks fine


a=mod(1:10000,2);  % longer time-series
plot(xcorr(a,a,20,'unbiased')) % divergencies

Mark van Rossum <cosine>
Thu 18 Feb 2021 05:50:57 PM UTC, original submission:  

Hi

The attached file creates a telegraph noise process (a timeseries that flips between two values).

Matlab produces the expected outcome when calling 'xcov' or 'xcorr',
but with Octave's signal-1.4.1 package I get wild fluctuations in the plot.

For simple tests such as
a=1:10; xcorr(a,a,'unbiased')
Matlab and Octave return the same result.

Mark

Mark van Rossum <cosine>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #50885:  Screenshot@at@2021-02-19@11-16-01.png added by cosine (11KiB - image/png - a=mod(1:10000,2);plot(xcorr(a,a,20,'unbiased')))
file #50884:  bug_60076.png added by philipnienhuis (39KiB - image/png)
file #50879:  telegraph_noise_bug.m added by cosine (477B - application/vnd.wolfram.mathematica.package - script showing the effect)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by siko1056 (Updated the item)
  • -email is unavailable- added by cosine (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-02-20 philipnienhuis StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2021-02-19 cosine Attached File- Added Screenshot@at@2021-02-19@11-16-01.png, #50885
    2021-02-19 philipnienhuis Attached File- Added bug_60076.png, #50884
        StatusNone Need Info
    2021-02-19 siko1056 Summarysignal xcorr bug for large datasets [octave forge] (signal) xcorr bug for large datasets
    2021-02-18 cosine Attached File- Added telegraph_noise_bug.m, #50879

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code