bugGNU Octave - Bugs: bug #46830, Multiplication about 4x slower...

 
 

bug #46830: Multiplication about 4x slower than Matlab

Submitter:  Ceral Paquet <octavebugs>
Submitted:  Wed 06 Jan 2016 10:03:28 AM UTC
   
 
Category:  Performance Severity:  1 - Wish
Priority:  5 - Normal Item Group:  Feature Request
Status:  Works For Me Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * 4.2.0 Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 20 Sep 2022 06:39:58 AM UTC, comment #28: 

IIUC, hyperthreads on the same physical CPU core share low-level CPU caches (at least some of them). Those CPU caches can be accessed much faster than "regular" memory.
Optimized BLAS implementations gain some of their performance by efficiently using those caches.

If multiple hyper-threads are running on the same CPU that can lead to more cache misses. In that case, the data is accessed over the slower bus to the "regular" memory. It might also be that the CPU caches are refreshed in that case. That in turn leads to the other hyperthread being slowed down when it tries to access the data that was in the CPU cache before the refresh. All of this is likely happening multiple times. Hence the "rivaling" hyperthreads are slowing down each other.

Afaiu, that is the main reason why using more threads than physical cores doesn't lead to much performance improvement (or none at all). It can even be slower than using only the physical cores for some operations (if fast access to the cached data is more advantageous than pure "calculation power").

Markus Mützel <mmuetzel>
Group administrator
Mon 19 Sep 2022 09:46:04 PM UTC, comment #27: 

@Markus:
Thanks, I knew that thread, it was about Intel Core CPU and I wasn't sure if it would apply equally to AMD Ryzen CPUs.
The intruiging thing is that building Octave & crossbuilding on that Ryzen-7 box with "-j16" and (make nsis-installer) JOBS=16 resp., works much faster than with "-j8" / JOBS=8, so it is just OpenBLAS that is affected by hyperthreading.

I don't know much about CPU internals, but I suspect that hyperthreading applies more to "regular" instructions rather than the less basic mathematical operations.  The latter are usually delegated to dedicated parts of the CPU that often contain "pipelines" e.g., operations for division, roots and trig, all of which, unlike read/write from/to memory, cannot be handled in a couple of clock ticks.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 19 Sep 2022 11:49:34 AM UTC, comment #26: 
Markus Mützel <mmuetzel>
Group administrator
Thu 08 Sep 2022 07:33:54 PM UTC, comment #25: 

W.r.t. my comment #13

> On that (Ryzen 5800U) laptop the same code runs much slower (...).


I usually copy the complete crossbuilt tree over to a btrfs partition that I can then access directly in Windows using winbtrfs.
However copying libopenblas.dll to libblas.dll wasn't implemented in that scenario yet ... I'll adapt my local <mxe-octave>/binary-dist-rules.mk to also fix this.

With that fixed "installation" I get a performance that is about 10 % faster than what I wrote in comment #9.
Intriguingly, increasing the number in environment setting "OpenBLAS_NUM_THREADS" beyond 8 makes no difference (Ryzen 5800U has 8 cores / 16 threads). Only thing I see with higher numbers is that initially CPU utilization peaks shortly at 70 to 80 % but immediately goes down to ~63 % for the rest of the computation time; which is the same CPU utilization as when using 8 threads.
FTR OpenBLAS is at 3.20 in mxe-octave.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 08 Sep 2022 05:54:58 PM UTC, comment #24: 

OT: added a specific note to all the other info at https://wiki.octave.org/Octave_for_Microsoft_Windows#Installers_for_Microsoft_Windows for those using the zip/7z archives. probably sufficient.

Nicholas Jankowski <nrjank>
Group Member
Thu 08 Sep 2022 03:52:12 PM UTC, comment #23: 

OT: Imho, "normal" users should use the installer on Windows. The .7z and .zip packages are for "more advanced" users.
But I'd guess that the BLAS switcher could be mentioned on the Wiki (if it isn't already).

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Sep 2022 03:40:51 PM UTC, comment #22: 

this is going off topic a bit, but if the default suggested to the user using the installer is openblas, and failing to change any default presented options is openblas, would it make sense for the zip package to also be set openblas as the no user intervention, out of the box, blas? The goal primarily being uniformity of user experience above anything else.

I also noticed the readme with the windows versions only mentions the shortcut in the start menu as method of switching BLAS, and makes no mention of the location/process of the dll's or the blas_switch.exe program for people who didn't use the installer.

Nicholas Jankowski <nrjank>
Group Member
Thu 08 Sep 2022 03:34:58 PM UTC, comment #21: 

confirming that after switching from reference to openblas, the same system performs the a*a operation in ~3-4 seconds.

Nicholas Jankowski <nrjank>
Group Member
Thu 08 Sep 2022 03:02:30 PM UTC, comment #20: 

That depends how you define "default".
If you use the installer, OpenBLAS is selected as the default. If OpenBLAS is selected, the installer deletes libblas.dll (the reference implementation by netlib) and replaces it with libopenblas.dll.
The user can later switch between the reference BLAS implementation and OpenBLAS with the tool that lostbard created. (That tool appears in the Start menu, or it can be found at `bin\blas_switch.exe` of the Octave installation.)

Octave for Windows links to libblas.dll (which is normally the reference implementation). But it doesn't care if that is actually the reference BLAS or any other BLAS implementation (like, e.g., OpenBLAS). Renaming libraries allows switching the BLAS implementation that Octave uses. If you just unzip a tarball and don't rename anything, libblas.dll is the reference BLAS implementation. So, in that case the reference BLAS implementation is the "default".

The reference implementation is usually the one with the "more numerically stable" algorithms. But it can be orders of magnitudes slower as implementations of the BLAS library that are optimized for certain hardware.
The OpenBLAS that is included in Octave for Windows is built for dynamic architecture. I.e., it detects at runtime which CPU features it can use. It is usually much faster than the reference implementation. But the results might be "less accurate" in some situations compared to the reference implementation.

(Having written that, there can still be bugs in either of them that could lead to the reverse behavior to the one described before. I.e., more accurate results with OpenBLAS or faster with the reference implementation. But the "expected" behavior is that OpenBLAS is faster, and that the reference implementation might be more accurate.)


Markus Mützel <mmuetzel>
Group administrator
Thu 08 Sep 2022 02:51:04 PM UTC, comment #19: 

You can switch it with "blas switcher" helper program.
Octave installer would set it up with Openblas by default --
at least for me.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 08 Sep 2022 02:39:30 PM UTC, comment #18: 

clicked submit too fast.

this is on Windows 10
Octave 7.2.0, 'installed' by extracting the 7z zip archive and running post-install.bat.

maybe I'm misunderstanding comment #16, Reference BLAS is supposed to be the default (it ships_with OpenBLAS, but using it is left to the user), correct?

Nicholas Jankowski <nrjank>
Group Member
Thu 08 Sep 2022 02:16:09 PM UTC, comment #17: 


> For the users that see slow multiplication: What does `version -blas` return for you?



>> version -blas
ans = unknown or reference BLAS


Nicholas Jankowski <nrjank>
Group Member
Thu 08 Sep 2022 09:50:28 AM UTC, comment #16: 

Subject slowdown on Ryzen:

The Windows installer "GNU Octave, version 7.2.0" is shipping with OpenBLAS 0.3.18 dating back to October 2021.


OpenBLAS 0.3.19 fixes a bug that an unknown / newly released x64 CPU will fall back to Prescott = Pentium IV features.

"DYNAMIC_ARCH builds now fall back to the cpu with most similar capabilities
when an unknown CPUID is encountered, instead of defaulting to Prescott"

https://github.com/xianyi/OpenBLAS/releases/tag/v0.3.19


The most recent version would be 0.3.21

Armin Müller <arm_in>
Thu 08 Sep 2022 09:23:50 AM UTC, comment #15: 

For the users that see slow multiplication: What does `version -blas` return for you?

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Sep 2022 05:51:13 AM UTC, comment #14: 

On a linux machine with 6 cores, matrices 8000x8000, octave 6.3.91 takes about 4 seconds for .* , using 1 core. matlab 2019a takes about 1.5 seconds, using all the cores.

Marco Caliari <caliari>
Group Member
Wed 07 Sep 2022 09:50:18 PM UTC, comment #13: 

@nrjank:
Just FTR, I also saw on Windows, with OpenBlas, that for a*a at first CPU usage is ~30 %, then a short while 100 %, then a much longer time hovering around 30-40 %.
'getenv ("OPENBLAS_NUM_THREADS ")' gives "4" on my i5 core desktop and "8" on my new HP Ryzen 5800U laptop
On that laptop the same code runs much slower (...). Note that these days I make the crossbuilds on that HP laptop so I suppose Openblas gets optimized for the crossbuilds but no, under my old desktop's Windows Octave-8 is faster.  I'll investigate but no time now.)

Philip Nienhuis <philipnienhuis>
Group Member
Wed 07 Sep 2022 07:12:39 AM UTC, comment #12: 

Today, I neither have the machine nor a Matlab license available from https://savannah.gnu.org/bugs/?46830#comment7 to reproduce the numbers.

Armin Müller <arm_in>
Wed 07 Sep 2022 01:55:34 AM UTC, comment #11: 

It looks to me that your octave does not use openblas.
I got about 20 sec on core2 duo (2 cpu). About ~15 years old laptop (Win7).

Dmitri A. Sergatskov <dasergatskov>
Wed 07 Sep 2022 01:25:19 AM UTC, comment #10: 

looking at the same comparisons in comment #7, on a Windows 10 machine with a modest Intel Core i5-6440HQ  (4 cores):

Matlab R2022a:


a=rand(6000);
tic; a * a; toc
Elapsed time is 4.977654 seconds.

tic; a .* a; toc
Elapsed time is 0.112174 seconds.


just watching task mgr, a*a clearly used all 4 cores. couldn't tell with a.*a.  I created b=rand(20000) array that took much longer and b.*b took ~20sec and clearly used all 4 cores.  cpu also pegged to ~100% utilization during both tasks.

Octave 7.2.0:

a = rand(6000);
tic; a*a; toc
Elapsed time is 1373.01 seconds.
tic; a .* a; toc
Elapsed time is 0.127481 seconds.


i don't know what it's doing under the hood or how to get that detail with vanilla windows, but with a*a all four cores saw some definite CPU load increase. Octave's CPU usage it hovered around 30-40% total CPU utilization even while system total was only 60-70%. never spiked any cpu to 100% unlike matlab which drove all CPUs to 100%.  memory use would ratchet up 250-300MB, similar to matlab memory use jumping up from ~1 to 1.4GB. no gpu on this machine.

so... don't know what might have happened in the past 6 years to go from 2x to 300x slower, but that definitely leaves a lot of opportunity for potential improvement. Ran each a few times to make sure something else wasn't driving a cpu limit, but got similar results.

Nicholas Jankowski <nrjank>
Group Member
Tue 06 Sep 2022 09:17:39 PM UTC, comment #9: 

On that same old Intel core i5 box (comment #5) with 8 GB RAM (under Windows):

Octave-8.0.0 crossbuilt a few days ago:

>> a = rand (1.5e4);
>> tic; a * a; toc
Elapsed time is 43.5511 seconds.
>> tic; a .* a; toc
Elapsed time is 0.8874 seconds.


and Matlab r2020b:

>> a = rand (1.5e4);
>> tic; a * a; toc
Elapsed time is 35.929964 seconds.
>> tic; a .* a; toc
Elapsed time is 2.608581 seconds.


so Octave's speed hasn't changed much.
Why Matlab is now so much slower for element-wise multiplication is beyond me but maybe not that interesting.


Philip Nienhuis <philipnienhuis>
Group Member
Tue 06 Sep 2022 01:47:27 PM UTC, comment #8: 

Reviewing old bug reports. The code in comment #7 takes only 80 milliseconds for me in current Octave 8 for elementwise multiplication and 1.4 seconds for full matrix multiplication.

Is this still a performance problem for current Octave compared to current Matlab?

Arun Giridhar <arungiridhar>
Group Member
Mon 21 Nov 2016 11:45:26 AM UTC, comment #7: 

Another benchmark:

Windows 7, 64 bit, i7-2620M (2 cores, 4 threads)

Octave 4.2.0, 32 bit

a = rand (6000);
tic; a * a; toc
Elapsed time is 21.4271 seconds.

utilizing 4 threads

a = rand (6000);
tic; a .* a; toc
Elapsed time is 0.116012 seconds.

utilizing 1 thread


Matlab R2015b, 64bit

a = rand (6000);
tic; a * a; toc
Elapsed time is 13.481621 seconds.

utilizing 2 threads

a = rand (6000);
tic; a .* a; toc
Elapsed time is 0.097478 seconds.

utilizing 2 threads


So, for me:
1) Both Octave and Matlab use parallelization, to a greater or lesser extent
2) Octave eats +50% runtime and +100% resources when multiplying matrices. => Maybe here, some improvement might be achieved be means of using other library functions, but not "4x slower" as the ticket is saying.
3) When doing element-wise multiplication, both are on a par with runtime and Octave is saving resources. This seems to be consistent with the reports of Philip and Marco.
4) No clue about the GPU
Also, it may be that a bigger difference can be seen when using 6 CPU cores (as Ceral has reported) as compared to 2 CPU cores on my machine. A benchmark with varying the number of available CPU cores might be helpful.

Armin Müller <arm_in>
Thu 17 Nov 2016 12:00:28 AM UTC, comment #6: 

Updating release to 4.2.0. I have no Matlab to compare with, but I see no difference on my system between 3.6.4 and 4.2.0, so I can only assume Octave's performance has not changed relative to itself.

We have some here saying that Octave is significantly slower and single-threaded while Matlab parallelizes the operations, and we have some saying there is no significant difference. Can we at least agree on whether there is a problem in this bug report, what the problem is, and whether we plan on doing anything about it?

Mike Miller <mtmiller>
Group Member
Tue 12 Jan 2016 10:35:44 PM UTC, comment #5: 

FWIW, with Matlab r2016a prerelease I see that it uses all four CPU cores (of my i5 PC with 8 GB RAM) when doing

a = rand (1.5e4);
tic; a * a; toc
Elapsed time is 34.337681 seconds.


but at most two cores (~50 % CPU usage) with

a = rand (1.5e4);
a .* a;
Elapsed time is 0.539022 seconds.


(Octave-4.1.0+ needs ~42 sec and 0.7 sec, resp. and Matlab makes the system fans clearly emit more noise with a * a.  AFAIK Matlab also invokes the GPU.)

All in all I'd conclude that Octave doesn't behave much differently from Matlab as regards CPU usage, but might be a little slower as it doesn't use the GPU.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 12 Jan 2016 12:15:28 PM UTC, comment #4: 

For the test with .*, I see both Octave and Matlab at about 0.4 seconds and about 1.0 second with 5000x5000 matrices.

Marco Caliari <caliari>
Group Member
Mon 11 Jan 2016 05:23:46 PM UTC, comment #3: 

The BLAS function DAXPY and similar could be used for the element-wise addition and subtraction operations and DDOT and similar could be used for the element-by-element multiplication operations. For the ./ et .^ operators I don't see a BLAS function that could be used

D.

Anonymous
Wed 06 Jan 2016 04:03:59 PM UTC, comment #2: 

Octave doesn't use BLAS for element-by-element operations as I don't think those are BLAS operations.

John W. Eaton <jwe>
Group administrator
Wed 06 Jan 2016 10:10:31 AM UTC, comment #1: 

Looking at the top command while running, it's clear that Matlab is using all CPUs for .* wheras Octave is using just one. There are 6 CPUs on my machine.

If I change the .* to a * (matrix multiply) then top shows all the CPUs are being used and the performance is the same.

Ceral Paquet <octavebugs>
Wed 06 Jan 2016 10:03:28 AM UTC, original submission:  

The following code is much slower in Octave than Matlab. I have run it on 2 different machines+verions: v3.81 32-bit from the Ubuntu repo and v4.01 64-bit compiled locally with openblas. Sorry I don't know what package implements the .* operator.


function hello()

A=rand(3000);B=rand(3000);
tic;for k=1:20;C=A.*B;end;toc


Octave:

>> hello

Elapsed time is 0.924918 seconds.

>> hello

Elapsed time is 0.935749 seconds.

>> hello

Elapsed time is 0.95104 seconds.


Matlab:

>> hello

Elapsed time is 0.213288 seconds.

>> hello

Elapsed time is 0.214244 seconds.

>> hello

Elapsed time is 0.214326 seconds.


Ceral Paquet <octavebugs>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by arungiridhar (Posted a comment)
  • -email is unavailable- added by arm_in (Posted a comment)
  • -email is unavailable- added by caliari (Posted a comment)
  • -email is unavailable- added by siko1056 (Updated the item)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by octavebugs (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-01-24 mtmiller Carbon-CopyRemoved 80942 -
    2016-11-17 mtmiller Release4.0.1-rc1 4.2.0
    2016-01-12 caliari StatusPostponed Works For Me
    2016-01-08 siko1056 Severity3 - Normal 1 - Wish
        Item GroupNone Feature Request
        StatusNone Postponed
        Operating SystemGNU/Linux Any
    2016-01-06 jwe CategoryNone Performance

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code