bugGNU Octave - Bugs: bug #49470, [MXE] eig.cc-tst failure in...

 
 

bug #49470: [MXE] eig.cc-tst failure in __run_test_suite__.m with 64-bit-Octave

Submitter:  Philip Nienhuis <philipnienhuis>
Submitted:  Fri 28 Oct 2016 03:45:04 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Philip Nienhuis Open/Closed:  * Closed
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 26 Aug 2017 08:23:31 AM UTC, comment #18: 

The test now passes on Windows 64bit (and still does on Linux). Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Fri 25 Aug 2017 06:34:34 PM UTC, comment #17: 

I went ahead and pushed the patch to the BIST here: http://hg.savannah.gnu.org/hgweb/octave/rev/9c736cc2ee94

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Sun 13 Aug 2017 10:30:30 AM UTC, comment #16: 

After reading Barbara's blog [1], I conclude that the test was intended to check whether the correct default decomposition method is used for symmetric input.
The failing test on Octave for Windows 64bit basically comes down to the following equality failing there (see also Avinoam's comment #7):

>> isequal(1e-15, 10^-15)
ans = 0


In the attached patch, I changed the failing test to use numeric literals instead of the power. With that change it produces the same results for "chol" and "qz" on Windows 64bit and Linux. I also added a comment that clarifies what is checked, and merged the three similar tests into one.

I also tried to find a non-symmetric matrix that produces different results for "chol" and "qz" but was unsuccessful to find one. Maybe someone that is more familiar with the algorithms/numerics involved can add a test for this case, too.

[1]: http://barbaralocsigsoc2016.blogspot.de/2016/07/cholqz-and-matrixvector-flags.html

(file #41503)

Markus Mützel <mmuetzel>
Group administrator
Sun 26 Feb 2017 11:21:45 PM UTC, comment #15: 

I agree with Avinoam's suggestion in Comment #5. It is enough to test that both methods give a correct result within tolerance and we do not need to check whether the results are identical.

Nir Krakauer <nir_krakauer>
Sun 26 Feb 2017 04:47:29 PM UTC, comment #14: 

Trying to add Nir and Carnë again since it did not work the last time.

Markus Mützel <mmuetzel>
Group administrator
Sun 26 Feb 2017 04:43:31 PM UTC, comment #13: 

This test still fails in Octave 4.2.1 64bit on Windows 10.
If I understand the discussion in patch #8960 correctly, the failing test was added to check if the correct algorithm (qz or chol) is used.
The failing test could indicate one of the following:
- The wrong algorithm is used on Windows 64bit in that particular case.
- The indicated algorithm is used but the results match nevertheless. Maybe due to lower rounding errors on the higher bit system (wild guess!).

In the first case, something might be wrong with the code in eig.
In the second case, the test does not really check what it is supposed to and should be changed for a better one.

I do not know by far enough about matrix decomposition to help in any of these cases.
Adding Nir and Carnë in CC since they were involved in the original patch.

Markus Mützel <mmuetzel>
Group administrator
Thu 17 Nov 2016 05:15:27 PM UTC, comment #12: 

The bug is still present because no one has worked on it.  See my comment #9.  We know which cset introduced the failing test.  The BIST test appear to need re-writing so that they use a tolerance.

Rik <rik5>
Group administrator
Thu 17 Nov 2016 06:46:48 AM UTC, comment #11: 

Still present in 4.3.0+, so I assume 4.2.0 is also affected.

My build dates from Nov. 15, latest revision included =
http://hg.savannah.gnu.org/hgweb/octave/rev/5b1fdeb1aa04 (remove oct-alloc.h)

Release => dev

Philip Nienhuis <philipnienhuis>
Group Member
Thu 17 Nov 2016 03:57:46 AM UTC, comment #10: 

Updating release to 4.2.0, I'm assuming this still affects the stable release.

Mike Miller <mtmiller>
Group Member
Wed 02 Nov 2016 03:56:48 PM UTC, comment #9: 

These tests are a new addition in this cset:


changeset:   22305:510886d03ef2
user:        Barbara Locsi <locsi.barbara@gmail.com>
date:        Tue Aug 16 02:07:58 2016 +0100
summary:     eig: new options for choice of algorithm, balancing, and output (patch #8960)


I've added Barbara to the CC list for this bug.  Maybe she knows why the tests were written in this manner.

It seems odd to me that is_qz_accurate and is_chol_accurate are always either true or false.  Won't this depend on the actual HW/SW system?  There is no prior test that would make these values dependent on the actual platform Octave is being used on.

One way around this, as Avinoam suggests, is to compare with a tolerance in assert, rather than using isequal which requires absolute equality.


Rik <rik5>
Group administrator
Tue 01 Nov 2016 11:21:14 PM UTC, comment #8: 

The difference could be due to sse (default on 64-bit)
 vs fp87 (on 32-bit) floating point path.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 01 Nov 2016 09:50:46 PM UTC, comment #7: 

I do not know if the computations are the same for 32 and 64-bit.
try for example


1.e-10-10^(-10)


On 32-bit version and 64-bit version (on the same machine).

Avinoam Kalma <avinoam>
Group Member
Tue 01 Nov 2016 08:36:16 PM UTC, comment #6: 

This makes me wonder why there's a difference at all between 32-bit and 64-bit Windows builds; isn't it only integer math that is affected?
AFAIK the word size for doubles and singles is equal for both 32-bit and 64-bit builds.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 01 Nov 2016 11:32:43 AM UTC, comment #5: 

The problem is with


function chol_qz_accuracy (A, B, is_qz_accurate, is_chol_accurate)
  [V1,D1] = eig (A,B, 'qz');
  [V2,D2] = eig (A,B); %default is chol
  assert (isequal (A*V1,A*V1*D1), is_qz_accurate)
  assert (isequal (A*V2, A*V2*D2), is_chol_accurate)
endfunction


The test is


>> A = diag([10^-16, 10^-15]);
>> chol_qz_accuracy (A, A, true, false);
error: ASSERT errors for:  assert (isequal (A * V2, A * V2 * D2),is_chol_accurate)

  Location  |  Observed  |  Expected  |  Reason
     ()           1            0         Abs err 1 exceeds tol 0


In 32 bit version


debug> D1
D1 =

Diagonal Matrix

   1   0
   0   1

debug> D2
D2 =

Diagonal Matrix

   1.00000         0
         0   1.00000

debug> D2 - [1 0; 0 1]
ans =

  -1.1102e-016  -0.0000e+000
  -0.0000e+000  2.2204e-016


So


assert (isequal (A*V2, A*V2*D2), is_chol_accurate)


pass.

In 64 bit


debug> D1
D1 =

Diagonal Matrix

   1   0
   0   1

debug> D2
D2 =

Diagonal Matrix

   1   0
   0   1

debug> D2 - [1 0; 0 1]
ans =

   0  -0
  -0   0


so A*V2 = A*V2*D2 and the test fails.

Actually this test looks wrong. it does not check the correct values of V1, V2. It checks if D1, D2 are close enough to the unit matrix, and if D2 is close enough, the test fails.

I suggest to replace the test with checking the values of V1, V2 with tolernce, something like


 assert (V1, diag ([1,1]), 0)
 assert (V2, diag([10^8,3.16227766016838*10^7]),10^(-7))


Avinoam Kalma <avinoam>
Group Member
Mon 31 Oct 2016 10:04:20 PM UTC, comment #4: 

Confirmed with octave-4.2.0-rc3-w64-installer.exe

Avinoam Kalma <avinoam>
Group Member
Sun 30 Oct 2016 03:08:45 PM UTC, comment #3: 

Title adapted.

(w. 32-bit octave on 64-bit Windows things work well, but not with a 64-bit octave build, Note: not 64-bit indexing I haven't tried that)

Philip Nienhuis <philipnienhuis>
Group Member
Sat 29 Oct 2016 07:02:55 PM UTC, comment #2: 

Same here: works fine on a 32-bit mxe-octave 4.3.0+

Philip Nienhuis <philipnienhuis>
Group Member
Sat 29 Oct 2016 01:34:44 PM UTC, comment #1: 

I did not see this problem with 32bit version


>> __octave_config_info__ ("hg_id")
ans = 340e37fd3c20
>> test eig.cc-tst
PASSES 75 out of 75 tests



Avinoam Kalma <avinoam>
Group Member
Fri 28 Oct 2016 03:45:04 PM UTC, original submission:  

The last months I consistently see an eig.cc-tst failure with --enable-windows-64 configured MXE cross-builds when running _run_test_suite_.m.
From fntests.log:

:
>>>>> processing C:\Programs\Octave\Octave-4.3.0+VEp_w64_qt4_20161028\share\octave\4.3.0+\etc\tests\libinterp\corefcn\eig.cc-tst
***** test
 A = diag([10^-16, 10^-15]);
 chol_qz_accuracy (A, A, true, false);
!!!!! test failed
ASSERT errors for:  assert (isequal (A * V2, A * V2 * D2),is_chol_accurate)

  Location  |  Observed  |  Expected  |  Reason
     ()           1            0         Abs err 1 exceeds tol 0


Philip Nienhuis <philipnienhuis>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #41503:  bug49470_eig_test.patch added by mmuetzel (2KiB - application/octet-stream)

 

Carbon-Copy List
  • -email is unavailable- added by nir_krakauer (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Could you take a look at this bug related to a patch for eig you submitted?)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by philipnienhuis (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 18 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-08-26 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2017-08-25 mmuetzel StatusConfirmed Ready For Test
    2017-08-13 mmuetzel Attached File- Added bug49470_eig_test.patch, #41503
    2017-02-26 mmuetzel Carbon-Copy- Added nir_krakauer
        Carbon-Copy- Added carandraug
    2017-02-26 mmuetzel Releasedev 4.2.1
        Dependencies- Depends on patch #8960
    2017-02-26 mmuetzel Dependencies- bugs #50402 is dependent
    2016-11-17 philipnienhuis Release4.2.0 dev
    2016-11-17 mtmiller Release4.2.0-rc3 4.2.0
    2016-11-02 mtmiller Dependencies- bugs #49517 is dependent
    2016-11-02 rik5 Carbon-Copy- Added -email is unavailable-
    2016-10-31 avinoam StatusNone Confirmed
        Releasedev 4.2.0-rc3
    2016-10-30 philipnienhuis Summary[MXE] eig.cc-tst failure in _run_test_suite_.m on 64-bit machines [MXE] eig.cc-tst failure in _run_test_suite_.m with 64-bit-Octave
    2016-10-28 rik5 Carbon-CopyRemoved 72865 -
    2016-10-28 rik5 Summary[ME] eig.cc-tst failure in _run_test_suite_.m [MXE] eig.cc-tst failure in _run_test_suite_.m on 64-bit machines

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code