bugGNU Octave - Bugs: bug #44898, octave uses a bunch of system CPU...

 
 

bug #44898: octave uses a bunch of system CPU overhead

Submitted by:  None
Submitted on:  Wed 22 Apr 2015 07:06:55 AM UTC  
 
Category: InterpreterSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Performance
Status: InvalidAssigned to: None
Originator Name: Wiliam EhlhardtOriginator Email: -unavailable-
Open/Closed: ClosedRelease: 3.8.1
Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 23 Apr 2015 12:41:00 PM UTC, comment #7:

Ok, I was able to download the 0.2.8-6 Debian OpenBLAS package and run the example using it, and I see the same thing you are seeing. So this is a fixed OpenBLAS bug, not a bug in Octave.

This bug will be fixed by updating to a newer OpenBLAS, either going to a new Ubuntu release or getting your own copy of OpenBLAS outside of the repositories.

Mike Miller <mtmiller>
Project Administrator
Thu 23 Apr 2015 05:24:35 AM UTC, comment #6:

I am using whatever "apt-get build-dep octave" gave me.

Doing "apt-cache search openblas" turns up a package named libopenblas-dev, which is version 0.2.8 on my system.

William Ehlhardt <orborde>
Thu 23 Apr 2015 01:06:50 AM UTC, comment #5:

Are you using the OpenBLAS library supplied by Ubuntu 14.04 or downloaded or self-compiled? I am working with OpenBLAS 0.2.12 from the Debian unstable repository.

Mike Miller <mtmiller>
Project Administrator
Thu 23 Apr 2015 01:03:10 AM UTC, comment #4:

I can't reproduce this problem, I am not seeing the same sys cpu overhead. I'm testing the same ex4.m example, Octave built from the default branch and using OpenBLAS. When I run it with --no-gui-libs I get no sys cpu at all. If I run without that option, I get a slight amount of sys cpu showing up on the non-busy cores (5-10%, comes and goes). I do not have OPENBLAS_NUM_THREADS or OMP_NUM_THREADS set in my environment.

Mike Miller <mtmiller>
Project Administrator
Thu 23 Apr 2015 12:35:04 AM UTC, comment #3:

Adding Mike Miller to the CC list since he has worked with OMP and OMP_NUM_THREADS and this is superficially related.

Quoting from the OpenBLAS wiki:

Before version 3.8.0, Octave ran a single thread. Now it runs two threads: 1) for the Qt GUI, and 2) for the main interpreter. This is probably why there is a problem now.

Octave requires that a BLAS library be installed, but is not compiled specifically for any library. This suggests that option number two isn't the right approach because we would need to have multiple configure options for OpenBLAS, Intel MKVS BLAS, etc. Also, many Linux distributions make it possible to quickly install a different BLAS version such as OpenBLAS, ATLAS, reference BLAS. If Octave were compiled against a specific library then they would also need to maintain multiple Octave packages for each combination.

Option 3 might be a possibility, but it would mean getting agreement from packaging managers at the various distributions like Debian to always compile with USE_THREAD=0.

It seems like Option 1 may be the best because it is configurable by the user and can therfore adapt to the local computing environment as needed.

Rik <rik5>
Project Administrator
Wed 22 Apr 2015 11:24:11 PM UTC, comment #2:

I downloaded [1], ran ./configure && make, and used ./run_octave ex4.m.

I still observe the "3 cores saturated with SYS CPU according to top". I have not rerun strace.

[1] ftp://alpha.gnu.org/gnu/octave/octave-4.0.0-rc3.tar.bz2

William Ehlhardt <orborde>
Wed 22 Apr 2015 03:35:07 PM UTC, comment #1:

Are you able to build Octave from source? If so, can you try the latest 4.0.0-rc3 release candidate at ftp://alpha.gnu.org/gnu/octave/? I'm pretty sure this has been fixed already.

Rik <rik5>
Project Administrator
Wed 22 Apr 2015 07:06:55 AM UTC, original submission:

On my system (Ubuntu 14.04 on a quad-core Core i5 processor), running a numerics-heavy script [1] causes octave to saturate one CPU core with usermode CPU usage (expected), and the other 3 cores with "sys" CPU usage (as reported by top) (VERY UNEXPECTED).

Some spelunking with strace [2][3] shows ~1.8 million sched_yield syscalls over the course of execution. Further hunting on mailing lists [4] brought me to [5]. Based on [5], I tried the following workaround:

OPENBLAS_NUM_THREADS=1 octave ex4.m

With that environment variable set, octave saturates one core and generates no weird sys CPU overhead, and strace shows no sched_yield calls.

EXPECTED BEHAVIOR: octave should not waste CPU with a zillion useless syscalls. It probably needs to set the OpenBLAS flags correctly.

[1] https://github.com/merwan/ml-class/blob/master/mlclass-ex4/ex4.m ; git clone the whole repository and run "octave ex4.m" to reproduce
[2] See attached strace_output_uniq.txt, generated roughly via strace octave ex4.m 2> /tmp/loge; uniq -c /tmp/loge > strace_output_uniq.txt
[3] On the advice of the inimitable Karl Fogel. Yes, THE Karl Fogel.
[4] Actually, Karl Fogel did the hunting. Isn't he great?
[5] https://github.com/xianyi/OpenBLAS/wiki/faq#multi-threaded

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #33736:  strace_output_uniq.txt added by None (517KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by rik5
  • -unavailable- added by orborde (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 23 Apr 2015 12:41:00 PM UTCmtmillerStatusNeed Info=>Invalid
      Open/ClosedOpen=>Closed
    Thu 23 Apr 2015 12:35:04 AM UTCrik5Carbon-Copy-=>Added mtmiller
    Wed 22 Apr 2015 03:35:07 PM UTCrik5CategoryNone=>Interpreter
      StatusNone=>Need Info
    Wed 22 Apr 2015 07:06:55 AM UTCNoneAttached File-=>Added strace_output_uniq.txt, #33736

    Back to the top


    Powered by Savane 3.1-cleanup1