bugGNU Octave - Bugs: bug #59331, Processor load stays up after...

 
 

bug #59331: Processor load stays up after `__run_test_suite__`

Submitter:  Markus Mützel <mmuetzel>
Submitted:  Sat 24 Oct 2020 12:37:20 PM UTC
   
 
Category:  GUI Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Performance
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 6.0.92 Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 30 Oct 2020 08:38:12 AM UTC, comment #38: 

I can no longer observe the high CPU load after running the test suite with hg id e44e3c2b7eea on Windows 10. Also a script with the commands from comment #24 no longer causes high CPU load.

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Wed 28 Oct 2020 07:32:02 PM UTC, comment #37: 

No worries, there's never any need to apologize about something like that.  Thanks for coming up with the fix!

John W. Eaton <jwe>
Group administrator
Wed 28 Oct 2020 07:03:49 PM UTC, comment #36: 

Thanks for commenting and pushing. Sorry for not having done this earlier (all day job).

I agree that there should be a concept for settings that are used/updated by interpreter and GUI before each setting gets its own solution. I am going to add a task for this.

Torsten Lilge <ttl>
Group Member
Wed 28 Oct 2020 03:22:11 PM UTC, comment #35: 

I added a commit message, pushed the change on stable, and merged with default:

http://hg.savannah.gnu.org/hgweb/octave/rev/8a05763823d1

Marking as ready for test.

John W. Eaton <jwe>
Group administrator
Wed 28 Oct 2020 01:17:29 PM UTC, comment #34: 

Thanks for the explanation.  And no, we don't have anything for settings in the interpreter other than the octaverc startup files.

Yes, I agree the current fix is fine for now and should be applied.

For the future, it would be good to think about a consistent, recommended way to handle settings that

  • belong primarily to the interpreter and might be reflected/cached in the GUI
  • belong primarily to the GUI and might be set/queried by the interpreter
  • might belong somehow to both the interpreter and the GUI, be set from one or the other, etc.


If the idea of a new terminal widget ever happens, I've also been thinking about how the GUI and interpreter should really be connected and how startup should happen, possibly by making it possible to start the GUI from the interpreter command line.  But all of that is getting far away from the topic of this report.

John W. Eaton <jwe>
Group administrator
Wed 28 Oct 2020 01:07:18 PM UTC, comment #33: 

To give a little context how we ended up with the current situation:
The encoding setting originally was a setting only for the editor in the GUI. The interpreter wasn't aware of encodings (which led to different issues).
When we added encoding-awareness to the interpreter, I thought it would make sense to have the two settings synchronized. The reasoning being that a user would probably expect that source files would be interpreted the same way as they are displayed in the built-in editor.
When that happened, the GUI setting already existed. Looking back, it might have made sense to move the setting from the GUI to the interpreter. But afaict, there is no settings file (or something similar) that is read before the first .m file is parsed. That would be a necessary requirement for this to work correctly. The GUI settings seemed like a good replacement for that missing(?) feature of the interpreter.

For Octave 6, Torsten's patch looks good to me imho.

But I agree that it might make sense to think about a better solution going forward.

Markus Mützel <mmuetzel>
Group administrator
Wed 28 Oct 2020 12:27:53 PM UTC, comment #32: 

I was also able to duplicate the problem and the patch fixes it for me.

Adding the parameter to "notice_settings" seems like a reasonable fix to me given that this setting appears to live in both the GUI and the interpreter.  But would it be possible for it to be a property of the interpreter that is simply reflected/cached in the GUI, similar to the way the current working directory is?  Then, changing the setting in the GUI would trigger a call to _mfile_encoding_ (as it does now) and that would use the event manager to notify the GUI that the setting has been updated.  At that point, the GUI would just cache the new value.  But maybe that is not possible (or easy) given that the file encoding is stored inside the GUI settings data structure and we aren't handling them individually, only as a group?

I'm not suggesting that we should handle all the GUI settings individually.  I think the real problem is having settings that are stored in two places.  In this case, the interpreter needs the encoding independent of the GUI, so it seems that the interpreter should own that information and the GUI should be able to set/get that value as needed?

John W. Eaton <jwe>
Group administrator
Wed 28 Oct 2020 07:37:40 AM UTC, comment #31: 

Running the commands from comment #24 in a script indeed triggers the loop for me.
I haven't yet tried the patch from comment #29.

Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 10:41:23 PM UTC, comment #30: 

Thank you very much. From my side I can confirm that the problem is solved with this patch. Both the test as described in comment #27 and the execution of the _run_test_suite_ have been passed.  I have tested with default on fedora.

Hg200 <hg200>
Tue 27 Oct 2020 09:34:46 PM UTC, comment #29: 

I have attached a patch that fixes the issue for me (patch is not yet documented).

It prevents the GUI from triggering another update of an octave internal preference when the update of the GUI preference was in turn triggered by octave.



(file #50143)

Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 08:20:25 PM UTC, comment #28: 

comment #27: that looks like a "hot indicator". If it doesn't work the first time, the script will generally jam the GUI the second time it is run. but it is not guaranteed to work.

Hg200 <hg200>
Tue 27 Oct 2020 08:05:10 PM UTC, comment #27: 

I forgot to mention that the two commands must be executed from within a script. Issuing the commands in the terminal does not result in the endless loop.

Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 07:41:45 PM UTC, comment #26: 

It also doesn't trigger the loop for me.

Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 07:15:33 PM UTC, comment #25: 

Not for me.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Tue 27 Oct 2020 07:04:41 PM UTC, comment #24: 

This one triggers the loop reliably for me


__mfile_encoding__ ("system")
__mfile_encoding__ ("utf-8")


Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 06:13:56 PM UTC, comment #23: 

I have this one


>> system ("locale")
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=C
LC_TIME=C
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=de_DE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=de_DE.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=de_DE.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

>> __locale_charset__
ans = UTF-8


Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 05:47:57 PM UTC, comment #22: 

I see the same "SYSTEM (UTF-8)" in the alphabetical order and "SYSTEM (ANSI_X3.4-1968)" up top.
I would expect to see the latter when the locale would be set to "C". But I have the following:

>> system ("locale")
LANG=en_US.UTF-8
LANGUAGE=en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=C
LC_TIME=C
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_IE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_IE.UTF-8
LC_NAME=en_IE.UTF-8
LC_ADDRESS=en_IE.UTF-8
LC_TELEPHONE=en_IE.UTF-8
LC_MEASUREMENT=en_IE.UTF-8
LC_IDENTIFICATION=en_IE.UTF-8
LC_ALL=
ans = 0
>> __locale_charset__
ans = UTF-8


(Not sure how I ended up with a mixture of US and Ireland...)

Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 05:33:20 PM UTC, comment #21: 

@Markus: I am currently bulding for testing a minor change that might help.

What I do not understand is why my encoding combo box in the preferences dialog shows "SYSTEM (UTF-8)" in the right place regarding alphabetical ordering and "SYSTEM (ANSI_X3.4-1968)" at the very beginning of the list. The latter sounds quite weird to me on a Debian system.

Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 03:33:28 PM UTC, comment #20: 

@Torsten: Do you have an idea how this could be done (have "SYSTEM (locale)" displayed but "system" in the config)?
The way I tried to implement it obviously has issues...

Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 02:34:15 PM UTC, comment #19: 

That might have been my doing.
The changeset that started this is probably this one:
https://hg.savannah.gnu.org/hgweb/octave/rev/a4416cd6bb4f

IIRC, I intended to have "System (UTF-8)" (or something similar) for displaying. I thought that all string starting with "system" could be treated equivalent. But it looks like changes in the part after "system" still trigger some kind of update.



Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 01:56:50 PM UTC, comment #18: 

The encoding is the same but the name is different: "System (UTF-8)" vs. "system".

Using _mfile_encoding_, there is a risk of repeated actions although this should stop after one cycle (see https://savannah.gnu.org/bugs/?54310#comment4). Something might have changed in way such that this cycle does not stop anymore.

Torsten Lilge <ttl>
Group Member
Tue 27 Oct 2020 07:52:29 AM UTC, comment #17: 

@Torsten: I was suspicious of that setting. But I wasn't able to reproduce the bug by calling `__mfile_encoding__` directly.
IIRC, the mfile encoding setting in the GUI changes the corresponding setting in the interpreter and vice versa.
But it's not entirely clear to me why we would end up in an infinite loop...

Can you see which are the different encodings in the two states?

Markus Mützel <mmuetzel>
Group administrator
Tue 27 Oct 2020 06:44:24 AM UTC, comment #16: 

I was now also able to reproduce it. The difference between both ini files is


default_encoding


I guess this is related to the command


__mfile_encoding__


but I will have a closer look tonight (German time).

Torsten Lilge <ttl>
Group Member
Mon 26 Oct 2020 11:39:17 PM UTC, comment #15: 

while octave idling at 63% cpu load:


$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20775 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20775 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20775 Oct 26 19:36 octave-gui.ini
$ ls -go octave-gui.ini
-rw-rw-r-- 1 20784 Oct 26 19:36 octave-gui.ini


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 26 Oct 2020 11:32:59 PM UTC, comment #14: 

I can see that octave-gui.ini keeps changing (actually seems to be alternating between two different sizes).

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 26 Oct 2020 11:11:25 PM UTC, comment #13: 

I xan also reproduce it with stable (c67f39732d14) on CentOS

CPU load ~63%

The gui is responsive.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 26 Oct 2020 10:56:56 PM UTC, comment #12: 

I had it again. Unresponsive GUI when scrolling and typing, high load 20 minutes after test suite has been finished and its continuously writing to octave-gui.ini. I can't reproduce it reliably over here, but I can say that it also occurs under Linux on the default branch.

Hg200 <hg200>
Mon 26 Oct 2020 09:18:46 PM UTC, comment #11: 

I saw quite some static load (70%) after completion of _run_test_suite_ on FC linux with hg id 9c3f4e9a0c83 two times. Then i started to bisect with fresh make and now i can't reproduce the problem any more. Unfortunately i haven't made a copy from my old octave-gui.ini. very well done.



Hg200 <hg200>
Mon 26 Oct 2020 08:44:47 PM UTC, comment #10: 

In reply to comment #9:
Markus, which test was causing the effect that you are describing?

In reply to comment #8:
When the issue occurs, the settings file should already be existing. Therefore, I guess an interaction of the related test with the settings file is triggering an endless loop of changing a setting.

Torsten Lilge <ttl>
Group Member
Mon 26 Oct 2020 06:01:28 PM UTC, comment #9: 

I'm also not familiar with the GUI code. So I don't know if I'm of much help here.

I ran the test suite again. That triggered the familiar state with the laggy GUI.
I attached gdb to the octave process, set at breakpoint in `octave::main_window::notice_settings` and continued execution.
Once the breakpoint was hit, I continued execution step by step until gdb became irresponsive. Execution didn't return from the gdb session to the Octave process I guess. At least, I couldn't interact with the GUI.
At that point, CPU load was high again and it was back to perpetually writing to octave-gui.ini.ABCDEF files.

Attaching a copy of the gdb session.

(file #50114)

Markus Mützel <mmuetzel>
Group administrator
Mon 26 Oct 2020 05:16:52 PM UTC, comment #8: 

In the resource_manager constructor, there is some attempt to copy existing resource files for the new system-dependent file name/location.  It looks like we are leaving that decision to Qt with QSettings::IniFormat.  I don't know exactly how that process is supposed to work.  Maybe some kind of signal/slot loop is happening?

Or are there specific tests that would access the GUI config file and cause Octave to try to rewrite it (as it appears is happening)?

I don't know enough about how this is supposed to work.

Can you use the debugger and step to see what actions are happening while it is in this state?  Is it possible to identify what is causing the apparent infinite loop?

John W. Eaton <jwe>
Group administrator
Mon 26 Oct 2020 03:43:41 PM UTC, comment #7: 

The release candidate is probably not a debug build. And I assume you didn't explicitly disable compiler optimizations.
So, it is possible that the compiler inlined some functions (which might make it difficult to follow the stack).

I deleted "%APPDATA%\octave\octave-gui.ini" (is this the GUI config file you meant?) and ran `__run_test_suite__` again.
This didn't change the high CPU load after the tests have finished. And the "octave-gui.ini.ABCDEF" files are still created perpetually until I close the GUI.

Markus Mützel <mmuetzel>
Group administrator
Mon 26 Oct 2020 02:56:16 PM UTC, comment #6: 

I can reproduce on Win7 Home.
The test also fails at gzip.cc-tst, but may be this is a separate issue.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 26 Oct 2020 02:02:00 PM UTC, comment #5: 

I'm unable to duplicate this problem with the 6.0.92 binary from the installer that I uploaded to alpha.gnu.org.  I am executing it on a Windows 10 system.  After executing _run_test_suite_ the octave-gui.exe process idles at 0.2-0.3% CPU on my system.  Maybe the load should be even lower, but it is not completely unreasonable as it is running the readline input event hook function every 0.1 seconds as it waits for input.

The stack trace appears to show that it is doing something inside main_window::notice_settings, but I don't see exactly how
QApplicationPrivate::notifyLayoutDirectionChange is called from there (the stack trace shows it is called directly?)

Does the problem persist if you delete the Octave GUI config files and try starting a new Octave session?

John W. Eaton <jwe>
Group administrator
Mon 26 Oct 2020 06:22:25 AM UTC, comment #4: 

Confirmed for a Dell laptop Octave 6.0.92 only in the GUI, CLI is not affected.  6.0.90 did not have this problem.

os: MS Windows 10 Home (version 2004)

Kai Torben Ohlhus <siko1056>
Group Member
Sun 25 Oct 2020 04:11:10 PM UTC, comment #3: 

It seems to be reproducible for me.

Markus Mützel <mmuetzel>
Group administrator
Sat 24 Oct 2020 08:39:20 PM UTC, comment #2: 

On my Win10 box I do not see that; CPU load comes back to a few % right after the test suite.

Philip Nienhuis <philipnienhuis>
Group Member
Sat 24 Oct 2020 12:39:03 PM UTC, comment #1: 

CC'ing Torsten because he is our expert in all questions GUI.

@Torsten: Do you have a hunch on what might be going on?

Markus Mützel <mmuetzel>
Group administrator
Sat 24 Oct 2020 12:37:20 PM UTC, original submission:  

After running the test suite with `__run_test_suite__` with Octave 6.0.92 on Windows 10, one of the processor cores stays at full load. At that point, Octave's interface is very slow to respond on any interaction (with mouse or keyboard).

I attached a profiler (Very Sleepy) to Octave while in this state and profiled the thread with the high load for approximately 3 minutes. Attached is the exported stack that produced the highest load (approx. 90% of total) during this time period.

Symbols are mangled. But if I'm reading this correctly, it is the GUI thread that seems to be hanging.

It seems to hang indefinitely. Even after about 20 minutes without interacting with Octave, the CPU load is still up.

Looking on the disc drive activity, it looks like lots of files named "%APPDATA%\octave\octave-gui.ini.ABCDEF" (where ABCDEF is six random letters) are written and discarded immediately in a loop.

This was with a fresh installation of Octave immediately with the instance that was opened by the installer.

Markus Mützel <mmuetzel>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #50143:  bug59331_v01.patch added by ttl (3KiB - text/x-patch)
file #50114:  debug_after_BISTs.txt added by mmuetzel (5KiB - text/plain)
file #50086:  Octave_6.0.92_after_run_test_suite.csv added by mmuetzel (4KiB - application/vnd.ms-excel)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by hg200 (Posted a comment)
  • -email is unavailable- added by ttl (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by mmuetzel (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-10-30 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2020-10-28 jwe StatusPatch Submitted Ready For Test
    2020-10-27 ttl Attached File- Added bug59331_v01.patch, #50143
        StatusConfirmed Patch Submitted
        Operating SystemMicrosoft Windows Any
    2020-10-26 mmuetzel Attached File- Added debug_after_BISTs.txt, #50114
    2020-10-26 siko1056 StatusNone Confirmed
    2020-10-24 mmuetzel Carbon-Copy- Added ttl
    2020-10-24 mmuetzel Attached File- Added Octave_6.0.92_after_run_test_suite.csv, #50086

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code