bugGNU Octave - Bugs: bug #55226, building doc figures in .png...

 
 

bug #55226: building doc figures in .png format occasionally fails with error in getframe: unable to retrieve figure pixels

Submitter:  Mike Miller <mtmiller>
Submitted:  Fri 14 Dec 2018 09:23:45 PM UTC
   
 
Category:  Plotting with OpenGL Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Build Failure
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 21 Dec 2018 06:45:28 PM UTC, comment #14: 

Thank you, this looks fixed for me now.

Mike Miller <mtmiller>
Group Member
Fri 21 Dec 2018 04:45:47 PM UTC, comment #13: 
Pantxo Diribarne <pantxo>
Group Member
Thu 20 Dec 2018 08:11:08 PM UTC, comment #12: 

I attached an updated version of the patch. I added a FIXME note about not being able to restore the previous state of the mutex, but I think it is an overall improvement. So I'll push it if no one opposes.



(file #45707)

Pantxo Diribarne <pantxo>
Group Member
Tue 18 Dec 2018 03:42:16 PM UTC, comment #11: 

@Stefan, Mike: Thanks for testing. I'll rework the patch a bit and assume it is safe to leave the mutex unlocked after the widget has been initialized.

I am also wondering wether this patch could have an effect on the crash observed when building the doc on MAC. I can't find a report so I'll ask on the maintainers list.

Pantxo Diribarne <pantxo>
Group Member
Mon 17 Dec 2018 07:03:26 PM UTC, comment #10: 

Thank you, I'm attaching an edited version of the patch for small style fixes:

  • trailing whitespace
  • it is safe to delete null pointers


The patch works perfectly for me. I've tested both calling "test getframe" in a shell loop as well as generating the user manual png images in a loop, and all works for me reliably now.

I will next test that this also fixes bug #55225.

(file #45681)

Mike Miller <mtmiller>
Group Member
Sat 15 Dec 2018 07:26:41 PM UTC, comment #9: 

The cset applied fine here and I was able to build octave from tip with it, and make -j2.

Stefan Husmann <haawda>
Sat 15 Dec 2018 10:52:34 AM UTC, comment #8: 


>> Anyway, the cset.

should read "Anyway, can you test the cset."

Pantxo Diribarne <pantxo>
Group Member
Sat 15 Dec 2018 10:51:07 AM UTC, comment #7: 

I attached a preliminary cset that makes things work for me. This approach (creating and destroying graphics widgets synchronously) is really how we should proceed, I think.

The problem I am facing though is that we need to unlock graphics resources in the interpreter thread before initializing/finalizing widgets in the GUI thread.
But when unlocking the mutex from the interpreter thread there is currently no way to know if it was previously locked, and thus should be re-locked after operations in the GUI thread are complete.

Anyway, the cset.

(file #45671)

Pantxo Diribarne <pantxo>
Group Member
Sat 15 Dec 2018 07:44:00 AM UTC, comment #6: 

I guess it would be nice to have octave-client (similarly to emacs-client). Probably not for 5.0 though....

Dmitri.
--



Dmitri A. Sergatskov <dasergatskov>
Sat 15 Dec 2018 12:51:22 AM UTC, comment #5: 

Thank you!

I've also tested the following now:


$ for i in $(seq 1 100); do ./run-octave -qfH --eval 'hf = figure ("visible", "off"); d = getframe (hf);'; done
error: getframe: unable to retrieve figure pixels
error: getframe: unable to retrieve figure pixels
$ for i in $(seq 1 100); do ./run-octave -qfH --eval 'pause (1); hf = figure ("visible", "off"); d = getframe (hf);'; done
error: getframe: unable to retrieve figure pixels
error: getframe: unable to retrieve figure pixels
error: getframe: unable to retrieve figure pixels
$ for i in $(seq 1 100); do ./run-octave -qfH --eval 'pause (3); hf = figure ("visible", "off"); d = getframe (hf);'; done


I get the same 2-3% failure rate with the first two as with my other tests. But with a three second pause before doing anything each time Octave starts, I get zero failures. This again seems to tie it to the elapsed time between Octave starting and the call to getframe.

Mike Miller <mtmiller>
Group Member
Sat 15 Dec 2018 12:42:45 AM UTC, comment #4: 

On Ubuntu 18.10, nvidia drivers, first code shows no failure (2 runs), while second one fails sporadically (~10 times, 2 runs).

Building default almost always shows the getframe failure.

Juan Pablo Carbajal <juanpi>
Group Member
Sat 15 Dec 2018 12:08:22 AM UTC, comment #3: 

On i5-4570 first test never fails. Second test never fails when run as a single process, but fails often if i run at least 2 jobs in parallel.

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 14 Dec 2018 11:30:09 PM UTC, comment #2: 

I've done some more testing of this bug and how it relates to calling the 'getframe' function directly. This bug now seems very closely related to bug #53328, because I can also still reproduce the same error message with test('getframe').

One important observation is that this may have to do with 'getframe' being called very early in the Octave process lifetime, while either Octave's graphics subsystem or Qt's internals are still being initialized.

The following loop, for example, gives me zero test failures, everything seems all good


$ ./run-octave -qfH --eval "for i = 1:1000; test('getframe'); endfor"


But rewriting this as multiple independent invocations of Octave from the command shell does give me test failures, 24 out of 1000 on the last run


$ for i in $(seq 1 1000); do ./run-octave -qfH --eval "test('getframe')"; done


Can someone else test and compare these two different loop syntaxes on their system to see if they can reproduce my problem?

This is exactly how our build system works to generate the figures for the user manual, launching a new Octave instance to plot and save the figure as an image file each time.

If this can be confirmed, it might also be interesting to see if some time delay after starting Octave prevents this error from happening, and if this was also the cause of bug #53328, and if anyone has any insights into what might be causing this or what might not be initialized fully yet.

Mike Miller <mtmiller>
Group Member
Fri 14 Dec 2018 11:28:15 PM UTC, comment #1: 

See also this bug report about getframe failing under high CPU load (bug #53328).  It is marked as resolved, but maybe it really just moved from "often" to "occasionally".

Rik <rik5>
Group administrator
Fri 14 Dec 2018 09:23:45 PM UTC, original submission:  

On my system, building the figures for the user manual in the .png format occasionally fails with the error


error: getframe: unable to retrieve figure pixels


At this point, Octave exits with an error and the build fails.

This is very similar to bug #55225, but not exactly the same. In both reports, I can reproduce the error by repeatedly running the same command in serial and eventually seeing a failure. However, when printing to the .eps and .pdf formats, there is no error message and Octave appears to exit successfully. In the .png format, there is a clear error message and Octave exits with an error.

After tracking down the "FIXME" workaround where this error message is reported (see bug #53328), I was able to add a fourth iteration to the code with a much longer delay:


--- a/libgui/graphics/ObjectProxy.cc
+++ b/libgui/graphics/ObjectProxy.cc
@@ -24,6 +24,8 @@ along with Octave; see the file COPYING.
 #  include "config.h"
 #endif

+#include <iostream>
+
 #include <QCoreApplication>
 #include <QString>
 #include <QThread>
@@ -152,7 +154,13 @@ namespace QtHandles
             octave::sleep (0.2);
             if (! QMetaObject::invokeMethod (m_object, "slotGetPixels", t,
                                              Q_RETURN_ARG (uint8NDArray, retval)))
-              error ("getframe: unable to retrieve figure pixels");
+              {
+                std::cerr << "octave::ObjectProxy::get_pixels: slotGetPixels failed three times, sleeping longer" << std::endl;
+                octave::sleep (3.0);
+                if (! QMetaObject::invokeMethod (m_object, "slotGetPixels", t,
+                                                 Q_RETURN_ARG (uint8NDArray, retval)))
+                  error ("getframe: unable to retrieve figure pixels");
+              }
           }
       }


With this change, the "failed three times" message is printed occasionally, where it would have failed, but now with the additional 3 second delay it never fails for me. An initial attempt with a 0.5 second delay was not enough.

This hack does not affect bug #55225, probably because the .eps and .pdf formats are not using the same 'getframe' code path.

Mike Miller <mtmiller>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #45707:  synchronous_qt3.patch added by pantxo (7KiB - text/x-patch)
file #45681:  synchronous_qt_edit.patch added by mtmiller (5KiB - text/x-patch)
file #45671:  synchronous_qt.patch added by pantxo (6KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by haawda (Posted a comment)
  • -email is unavailable- added by pantxo (Updated the item)
  • -email is unavailable- added by juanpi (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-21 mtmiller StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2018-12-21 pantxo StatusPatch Reviewed Ready For Test
    2018-12-20 pantxo Attached File- Added synchronous_qt3.patch, #45707
    2018-12-17 mtmiller Attached File- Added synchronous_qt_edit.patch, #45681
        StatusConfirmed Patch Reviewed
    2018-12-15 pantxo Attached File- Added synchronous_qt.patch, #45671
    2018-12-15 mtmiller StatusNone Confirmed
    2018-12-14 mtmiller Carbon-CopyRemoved 80942 -

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code