bugGNU Octave - Bugs: bug #65020, BLAS inv() incorrect on new intel...

 
 

bug #65020: BLAS inv() incorrect on new intel architecture

Submitter:  None
Submitted:  Tue 12 Dec 2023 05:17:39 PM UTC
   
 
Category:  Libraries Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Need Info Assigned to:  None
Originator Name:  Sukanta Basu Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 6.4.0
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 30 Jan 2024 09:08:56 PM UTC, comment #11: 

I have not resolved the BLAS issue with version 6.4.0. However, I can confirm that the issue does NOT exist for version 7.1.0.

Anonymous
Tue 19 Dec 2023 09:11:34 AM UTC, comment #10: 

I build 6.4 from the source as well. In my work, I use the octave-mpi package (https://github.com/carlodefalco/octave-mpi), which is not compatible with the newer versions of octave.

I will post my comments on the openBLAS forum. Before I do that, I want to rebuild octave with a manually compiled version of openBLAS. I am facing some technical difficulties. Hopefully, I will be able to resolve them shortly.

Anonymous
Mon 18 Dec 2023 11:56:37 PM UTC, comment #9: 

I agree with Dmitri.  This feels like an upstream problem with BLAS or LAPACK libraries.

For what it's worth, I also run Ubuntu 22.04.  I'm using the default libblas3:amd64 3.10.0-2ubuntu1.  From 'cat /proc/cpuinfo' my cpu is


model name      : 12th Gen Intel(R) Core(TM) i9-12900H


Using your test code I get 0.4608 as the first entry of either inv() or pinv().  In other words, it works.

I would also note that version 6.4 of Octave is very, very old and is no longer supported.  You might try a newer version just in case that helps.  I see that you said you did build Octave from source, but I don't know if that was 6.4 source or something more recent.

Rik <rik5>
Group administrator
Wed 13 Dec 2023 04:26:50 PM UTC, comment #8: 

I suggest reporting the problem to
https://github.com/OpenMathLib/OpenBLAS/issues

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Wed 13 Dec 2023 01:38:23 PM UTC, comment #7: 

On the Xeon system,

OPENBLAS_VERBOSE=2 octave

produces:
Core: Cooperlake

Please note that I have tested this problem on two different installations of octave on the Xeon system. First, I used the pre-compiled binary via apt-get. Next, I also compiled octave from the source files. Both lead to an identical problem.

Anonymous
Wed 13 Dec 2023 05:46:06 AM UTC, comment #6: 

Could you do some more diagnostic?
Start octave as:

$ OPENBLAS_VERBOSE=2 octave


it should report the underlying kernel it is using.
On my compiter I see:

$ OPENBLAS_VERBOSE=2 octave
Core: Zen


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Dec 2023 10:49:55 PM UTC, comment #5: 


comment #4:

> On the Xeon system:
> octave:1> randn('seed',0);
> octave:2> A = randn(100);
> octave:3> B = inv(A);
> octave:4> max(max(abs(A*B - eye(100))))
> ans = 2.8047e+04


This is bad.
But there is nothing octave can do about it. It is an issue with
an underlying library. You should report it to OpenBlas and/or lapack/blas people.

Sincerely,

Dmitri.
--


Dmitri A. Sergatskov <dasergatskov>
Tue 12 Dec 2023 09:04:46 PM UTC, comment #4: 

On the Xeon system:
octave:1> randn('seed',0);
octave:2> A = randn(100);
octave:3> B = inv(A);
octave:4> max(max(abs(A*B - eye(100))))
ans = 2.8047e+04

On the AMD system:
octave:1> randn('seed',0);
octave:2> A = randn(100);
octave:3> B = inv(A);
octave:4> max(max(abs(A*B - eye(100))))
ans = 6.0140e-14

Here is a bit of background. I have written a computational fluid dynamics code in Octave about 15+ years ago. It has been working well all these years on various platforms. It contains one matrix inversion; the matrix is well-conditioned, and I have never had any issues.

In order to diagnose the root of this problem, I am creating the random number array. In the actual code, it is not random.

Anonymous
Tue 12 Dec 2023 05:54:09 PM UTC, comment #3: 

The A is an ill-conditioned matrix, so calculating inv will be noisy and may depend on instruction set (SSE/AVX/AVX512) used.

What is max(max(abs(A*B - eye(100)))) on the affected machine?

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Dec 2023 05:44:30 PM UTC, comment #2: 

This is what I get this time. The value of inv() is different this time.

sukanta@HAL:/ssd/OCTLES/Neutral_f14_Nz100$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/blas/libblas.so.3 octave
GNU Octave, version 6.4.0
Copyright (C) 2021 The Octave Project Developers.
This is free software; see the source code for copying conditions.
There is ABSOLUTELY NO WARRANTY; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  For details, type 'warranty'.

Octave was configured for "x86_64-pc-linux-gnu".

Additional information about Octave is available at https://www.octave.org.

Please contribute if you find this software useful.
For more information, visit https://www.octave.org/get-involved.html

Read https://www.octave.org/bugs.html to learn how to submit bug reports.
For information about changes from previous versions, type 'news'.

octave:1> randn('seed',0);
octave:2> A = randn(100);
octave:3> B = inv(A);
octave:4> C = pinv(A);
octave:5> B(1)
ans = 1.1232
octave:6> C(1)
ans = 0.4608
octave:7>

Anonymous
Tue 12 Dec 2023 05:40:46 PM UTC, comment #1: 

It is most likely OpenBlas issue.
Can you repeat the test on Xeon preloading reference blas library?
Start octave as:

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/blas/libblas.so.3 octave


You can try to download and compile the latest openblas library and preload it the same way. It should also help with performance, your Xeon is probably too new for your Openblas library (and it uses some generic kernel instead).


Sincerely,
Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 12 Dec 2023 05:17:39 PM UTC, original submission:  

I am on a new workstation (Intel Xeon w9-3495X (105MB Cache, 56 cores, 112 threads, 1.9GHz to 4.8GHz Turbo 350W) running Ubuntu 22.04. Please see the inv() and pinv() computations below. I repeat them on a different workstation (AMD Ryzen Threadripper PRO 5995WX: 64 cores, 2.7 GHz, 256 MB cache, PCIe 4.0).

----------------------------------------------------------------------
GNU Octave Version: 6.4.0 (hg id: 8d7671609955)
GNU Octave License: GNU General Public License
Operating System: Linux 6.1.0-1027-oem #27-Ubuntu SMP PREEMPT_DYNAMIC Mon Nov 20 15:08:14 UTC 2023 x86_64
----------------------------------------------------------------------
Package Name  | Version | Installation directory
--------------+---------+-----------------------
       image *|  2.14.0 | /home/sukanta/octave/image-2.14.0
         mpi *|   3.1.0 | /home/sukanta/octave/mpi-3.1.0
octave:2> randn('seed',0);
octave:3> A = randn(100);
octave:4> B = inv(A);
octave:5> C = pinv(A);
octave:6> B(1)
ans = -172.63
octave:7> C(1)
ans = 0.4608

The same computation on AMD system:

octave:13> randn('seed',0);
octave:14> A = randn(100);
octave:15> B = inv(A);
octave:16> C = pinv(A);
octave:17> B(1)
ans = 0.4608
octave:18> C(1)
ans = 0.4608
octave:19> version -lapack
ans = Linear Algebra PACKage Version 3.10.0
octave:20> version -blas
ans = OpenBLAS (config: OpenBLAS 0.3.20 NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Zen MAX_THREADS=64)

You will notice that the pinv() results match from both the systems. Furthermore, inv() and pinv() from AMD match. But, inv() and pinv() on intel are completely different.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-12-18 rik5 CategoryOctave Function Libraries
        StatusNone Need Info
        Open/ClosedOpen Closed
        SummaryIssues with inv() on new intel architecture BLAS inv() incorrect on new intel architecture

    Back to the top

    Powered by Savane 3.13-04b1.
    Corresponding source code