bugGNU Octave - Bugs: bug #48195, missing getframe function in...

 
 

bug #48195: missing getframe function in gnuplot

Submitter:  Guillaume <gyom>
Submitted:  Fri 10 Jun 2016 02:02:25 PM UTC
   
 
Category:  Plotting with gnuplot Severity:  1 - Wish
Priority:  1 - Later Item Group:  Feature Request
Status:  Confirmed Assigned to:  None
Originator Name:  Guillaume Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 20 Jan 2024 10:51:39 PM UTC, comment #31: 

The gnuplot graphics backend is unsupported.  Lowering priority.

Rik <rik5>
Group administrator
Thu 01 Jun 2017 08:08:08 AM UTC, comment #30: 

@Phillip: thanks. I added the original bug number in the test to make it clear the failure may come from this known bug.

Pantxo Diribarne <pantxo>
Group Member
Tue 30 May 2017 06:06:49 PM UTC, comment #29: 

Just FTR, on Windows 7 64bit with a recently self-built mxe installer (64-bit) I get:

>> test getframe
Mesa warning: couldn't open dxtn.dll, software DXTn compression/decompression unavailable
warning: opengl_renderer: Error 'invalid operation' (1282) occurred in init_gl_context
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'figure' object
Mesa warning: couldn't open dxtn.dll, software DXTn compression/decompression unavailable
warning: opengl_renderer: Error 'invalid operation' (1282) occurred in init_gl_context
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'text' object
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'patch' object
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'axes' object
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'figure' object
***** testif HAVE_OSMESA
 hf = figure ("visible", "off");
 unwind_protect
   hax = axes ("visible", "off", "position", [0 0 1 1]);
   verts = [0 0; .5 0; 1 0; ...
            0 .5; .5 .5; 1 .5; ...
            0 1; .5 1; 1 1];
   faces = [1 2 5 4; 2 3 6 5; 4 5 8 7; 5 6 9 8];
   fvc = [1 0 0; 0 1 0; 0 0 1; 1 0 1];
   patch ("vertices", verts, "faces", faces, "facevertexcdata", fvc, ...
          "facecolor", "flat");

   kk = 1;
   pos = get (hf, "position");

   for jj = [0.05 0.55]
     for ii = [0.05 0.55]
       rect = [ii jj .4 .4].*[pos(3:4) pos(3:4)];
       im = getframe (hax, rect).cdata;
       assert (im(:,:,1) == fvc(kk,1)*255)
       assert (im(:,:,2) == fvc(kk,2)*255)
       assert (im(:,:,3) == fvc(kk,3)*255)
       kk++;
     endfor
   endfor
 unwind_protect_cleanup
   close (hf)
 end_unwind_protect
!!!!! test failed
assert (im (:, :, 1) == fvc (kk, 1) * 255) failed
>>


Perhaps also the condition "if (! ispc) ..." could be added?

However, getframe works beautifully, I'm very glad you've implemented it Pantxo! Thanks very much!

Philip Nienhuis <philipnienhuis>
Group Member
Sun 28 May 2017 09:57:51 AM UTC, comment #28: 

@Avinoam: thanks for testing. Good to know that demos work on windows.
The fact that tests don't work is expected: tests are run with invisible figures and thus need a working osmesa, which is known to not be the case on Windows (see bug #44338). The "testif HAVE_OSMESA" is not enough to prevent the test from being run (the mxe builf does have osmesa support) but I left it as is and added a second test that should fail (unlike the first one which actually passes after the warnings).

I pushed the patch here:

http://hg.savannah.gnu.org/hgweb/octave/rev/2aab625b502c  

I changed the title of the report to make it clear that the remaining issue is gnuplot specific. Changed Category and Status in accordance.

Pantxo Diribarne <pantxo>
Group Member
Fri 26 May 2017 06:01:25 AM UTC, comment #27: 


@Pantxo:

testing getframe8.patch (and fixing manually module.mk)
gave the same results. Somehow testif HAVE_OSMESA does not
work.

The demo works.

Do you want me to check more things in Windows?
Can you write a check script, so I will upload the results?

Avinoam Kalma <avinoam>
Group Member
Wed 24 May 2017 09:39:10 PM UTC, comment #26: 

@Avinoam: thanks for testing. The test makes use of OSMESA which is known to not work on Windows but is present (so that "testif HAVE_OSMESA" is ineffective): I'll mark the test as xfail. Does "demo getframe" work?

As for getframe not being distributed, I just saw that the patch did not apply cleanly on the current dev and scripts/image/module.mk is not updated. I attached an updated version that should fix this.


(file #40784)

Pantxo Diribarne <pantxo>
Group Member
Wed 24 May 2017 09:07:14 PM UTC, comment #25: 

Testing previous patch on Win-10, I get:


>> test getframe
Mesa warning: couldn't open dxtn.dll, software DXTn compression/decompression unavailable
warning: opengl_renderer: Error 'invalid operation' (1282) occurred in init_gl_context
warning: opengl_renderer: Error 'invalid operation' (1282) occurred drawing 'figure' object
PASSES 1 out of 1 test


The error is related to osmesa print, but since I have problem with this on Ubuntu, maybe it affects windows version.

Do you have more test scripts to check?

P.S.
I had to copy getframe.m manually to Windows Octave directory.

Avinoam Kalma <avinoam>
Group Member
Wed 24 May 2017 06:50:07 PM UTC, comment #24: 

@Guillaume: thanks for testing.

I attached an updated patch with a cleaner (but equivalent) approach, and the changelog.

@Philip: I'll have some time in the coming days to work on Octave  (and the movie function that is almost useless getframe). Do you think you'll have a chance to test the patch on Windows soon or should I ask for testing on the maintainers list?

(file #40782)

Pantxo Diribarne <pantxo>
Group Member
Thu 18 May 2017 02:42:12 PM UTC, comment #23: 

@Pantxo: thank you for looking back into this. Just tried your patch on openSUSE with nvidia drivers and it seems to work fine: the sombrero plot from a comment below and the demo. The content of the figure is also captured if it is set to "Visible" "off" or if the window is masked by another one or is outside the screen area. I will try and test on other linux platforms but it might take me more time.

Thanks!

Guillaume <gyom>
Thu 18 May 2017 01:11:57 PM UTC, comment #22: 

Oh, and since I don't have osmesa (bug on my ubuntu version) I was not able to test getframe on invisible figures. I would like some feedback about this also.

Pantxo Diribarne <pantxo>
Group Member
Thu 18 May 2017 01:02:55 PM UTC, comment #21: 

After some month I was able to come back to this issue. Please find attached a new take. Note:

  • getframe will render the figure which IIUC is not equivalent to Matlab's behavior (in ML examples I see one needs to call drawnow prior to getframe)
  • The issue with resizing is fixed but I don't fully understand the reason ...


I'd be interested in a feedback, especially on how it works on Windows and other linux flavors (mine is ubuntu 14.04)

(file #40732)

Pantxo Diribarne <pantxo>
Group Member
Sun 03 Jul 2016 12:31:53 PM UTC, comment #20: 

@Panxto:
Yes, _osmesa_print_() also returns solid black figures.
Alas.....

It seems Qt isn't coöperative on Windows; "native" OpenGL functions may be more robust.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 01 Jul 2016 05:26:51 PM UTC, comment #19: 

@Guillaume: right, I can trigger a segfault when resizing the figure. I'll work on this in the coming weeks I hope.
 have a properly working _osmesa_print_ are very few, windows
@Phillip: unfortunately, systems that have a properly working _osmesa_print_ are very few, windows is not among them.


Pantxo Diribarne <pantxo>
Group Member
Fri 01 Jul 2016 04:36:33 PM UTC, comment #18: 

Ah but now I see what you mean.
Well if _osmesa_print_ would work properly, getframe could be a simple wrapper. But on my Mageia Linux box, trying with sombrero(), I only see text and no graphics.

I wonder if it would work on Windows at all.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 01 Jul 2016 04:23:47 PM UTC, comment #17: 

@Panxto:
_osmesa_print_ probably won't save 15 sec., as it also writes to a disk file that imread needs to read into an image "array".

IMO copying some pixel array from graphics memory into another memory piece should always be faster. Problem of course is finding out which piece of memory to read from.
In the DOS era my proggies simply read graphics memory itself, i.e., various bit planes in succession, using int10 interrupts or mem copy variants, but those days are gone...

In spite of the side effects reported by Guillaume I'll try patch #3 anyway to check if it works at all in Windows.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 01 Jul 2016 03:42:14 PM UTC, comment #16: 

While the patch did work with the demos, it has side effects as described in bug #48361...

Guillaume <gyom>
Thu 30 Jun 2016 04:34:16 PM UTC, comment #15: 

Thanks Pantxo - I've just tried the new patch and this time it seems to work fine for me: all the demos worked as expected, which is brilliant!

Guillaume <gyom>
Tue 28 Jun 2016 08:29:47 PM UTC, comment #14: 

I attached another patch for testing. It is using a similar approach as _osmesa_print_ when you request an rgb image.

This is not ideal as the rgb array is stored after each redraw of figures (didn't time this), but it ensures the opengl context exists when trying to retrieve pixels.

@Philip: if you work on a machine that has osmesa capabilities this is good news, you can already save the 15 s a frame using _osmesa_print_ :-)



(file #37617)

Pantxo Diribarne <pantxo>
Group Member
Wed 15 Jun 2016 12:32:52 PM UTC, comment #13: 

@Guillaume:
Currently getframe() only works on figures. The root handle doesn't seem to be a figure but otherwise looks to be a good way to try for a "screen grabber".

As to Matlab compatibility, in Matlab getframe returns a struct with fields 'cdata' (the actual image) and 'colormap' (empty on my system).

Philip Nienhuis <philipnienhuis>
Group Member
Wed 15 Jun 2016 10:11:08 AM UTC, comment #12: 

I've had other issues with openGL in the past on my machine (with the nvidia drivers) so it might not be too surprising if it doesn't work straight away (I tried the demos as well). I don't really know how to investigate further - I also notice that Octave becomes unstable afterwards and segfault.

Also, I was not suggesting to modify the behavior of getframe - it would still work with figures or axes handles like in Matlab. My suggestion was to have an extra function, capturescreen, that getframe would rely on, and with which you could grab any part of the screen. Not essential but just thought it might be a great addition. An alternative would be to allow to pass the graphics root handle (0) to getframe to enable this scenario.


Guillaume <gyom>
Wed 15 Jun 2016 06:27:24 AM UTC, comment #11: 

Panxto, I'd be very satisfied with a getframe that works on just graphics windows.

I used to have a clumsy .m file substitute (printing to file, reading it and then adding it to a movie. 15 sec per frame, but yes it worked).

Philip Nienhuis <philipnienhuis>
Group Member
Tue 14 Jun 2016 09:28:45 PM UTC, comment #10: 

So it looks like "grabFrameBuffer", the Qt wrapper around "glReadPixels", is not very portable. I'll give "glReadPixels" a try.

As for capturing the whole screen it can't do the same job as retrieving pixels from opengl: suppose the figure window is partially masked behind another window then what you'll get is crap.

Pantxo Diribarne <pantxo>
Group Member
Tue 14 Jun 2016 06:57:34 PM UTC, comment #9: 

Hmm, does not work on Windows (original patch):

>> hh = figure ();
>> hg = plot ([1:9], 'm')
hg = -6.4752
>> grid
>> ishandle (hg)
ans = 1
>> isfigure (hh)
ans = 1
>> im = __get_frame__ (hh);
>> imshow (im)


shows a completely black plot save for a few colored pixels in the range x=[104:115], y=103  (attached)




Philip Nienhuis <philipnienhuis>
Group Member
Tue 14 Jun 2016 05:49:00 PM UTC, comment #8: 

I tried the original patch from Panxto and that works like a charm on Mageia-5. Thanks very much so far!
I've yet to try in Windows (made the build but had no time to install it yet) but I suppose it'll run fine there as well.

Several years back I looked into it but as my C++ proficiency is lacking and Octave's graphics system is a bit opaque to me I never got any further than reading up on OpenGL's glReadpixels :-)

Regarding Panxto's comments: as I understand Matlab's docs, getframe() is meant to grab (part of) a graphics window, not an arbitrary part of the users's screen.  Now that wouldn't be an argument to limit it to a graphics window. Maybe some special Octave-only input arg could help to switch between graphics window (axis measures) and screen (pixels).
The next challenge would be capturing remote screens; or maybe a specific screen in a dual/triple/multiple screens setup.

Let's first make getframe work reliably on a graphics window, then go for more options ?

Philip Nienhuis <philipnienhuis>
Group Member
Tue 14 Jun 2016 05:31:27 PM UTC, comment #7: 

@Guillaume: just to be sure, did you try the demos? You need to call "drawnow" between any plotting operation and "getframe" if you run it in a script.

Pantxo Diribarne <pantxo>
Group Member
Tue 14 Jun 2016 03:23:53 PM UTC, comment #6: 

I must say I don't like the idea of capturing the whole screen. The reason is that the definition of the "rect" argument would then have to be different in Octave and Matlab.
Implementing a screen capture function using Qt or Java, as you did, is trivial and should be part of another function IMHO.

As for the 0s, indeed it would be helpful if others tested the patch on their system (Mine is linux Mint 17.3, based on ubuntu 14.04). As I said, I could try to use a low level opengl function instead of the corresponding Qt wrapper. This could solve the issue (or not), and make the function work for fltk also.

Pantxo Diribarne <pantxo>
Group Member
Tue 14 Jun 2016 03:03:36 PM UTC, comment #5: 

Thanks Pantxo!

I applied your patch - it compiles fine but it doesn't seem to work on my system (openSUSE 13.2). The variable returned by getframe only contains 0s. it would be useful if others could try the patch on their system and report whether it works or not.

Also, what do you think of going beyond Matlab compatibility and let the low level function capturing any part of the screen and not only the content of a figure? The Java code allowed that and i guess that the same can be done with Qt. Then the low level function used by getframe could be exposed to the user, eg:


function retval = capturescreen (xmin, ymin, xmax, ymax)


Guillaume <gyom>
Tue 14 Jun 2016 01:59:32 PM UTC, comment #4: 

I attached an updated patch containing the getframe.m function.
It still has to be polished, and we could eventually use the same method to grab pixels for fltk and qt, using directly opengl function. Tests are welcome.

(file #37470)

Pantxo Diribarne <pantxo>
Group Member
Mon 13 Jun 2016 02:15:06 PM UTC, comment #3: 

I had once started to work on this function. I attached a very preliminary patch (with many debug/include leftovers etc) that I updated so that it applies to the current default branch.

The low level function _get_frame_ catches the pixels of the whole figure (in Qt toolkit only). Then a higher level getframe.m can be written to implement the ML compatible interface.

Let me know if it fits the need.

To test you can run e.g.:


figure (56);
sombrero ();
im = __get_frame (56);
imshow (im)


(file #37465)

Pantxo Diribarne <pantxo>
Group Member
Fri 10 Jun 2016 04:02:30 PM UTC, comment #2: 

Implementing it with Qt does make much more sense than with Java.

This will probably involve adding some functions to the octave_link interface and passing a large screen capture array over this interface between the GUI thread and the interpreter.

Mike Miller <mtmiller>
Group Member
Fri 10 Jun 2016 03:56:46 PM UTC, comment #1: 

Certainly Octave is moving towards having Qt be the principal window and plotting system.  It would be best to implement getframe without a Java dependency.  However, I'm not a GUI programmer so I don't know much about the Qt API and how easy this would be.

Rik <rik5>
Group administrator
Fri 10 Jun 2016 02:02:25 PM UTC, original submission:  

I am trying to implement the missing getframe functionality:

http://www.mathworks.com/help/matlab/ref/getframe.html

I am able to capture the screen using java:


width = 256;
height = 128;

img = javaMethod ('createScreenCapture',...
  javaObject ('java.awt.Robot'),...
  javaObject ('java.awt.Rectangle', 0, 0, width, height));
img = typecast (javaMethod ('getRGB', img, 0, 0, width, height, [], 0, width), 'uint8');
img = reshape (img, 4, width, height);
img = cat (3, squeeze (img(3,:,:))', squeeze (img(2,:,:))', squeeze (img(1,:,:))');

figure, imagesc (img)


but I wonder whether it would be possible, at least for Qt, to have a wrapper around QScreen::grabWindow to work even in the absence of the Java runtime:

http://doc.qt.io/qt-5/qscreen.html#grabWindow

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #40784:  getframe9.patch added by pantxo (19KiB - text/x-patch)
file #40782:  getframe8.patch added by pantxo (19KiB - text/x-patch)
file #40732:  getframe5.patch added by pantxo (18KiB - text/x-patch)
file #37617:  getframe3.patch added by pantxo (11KiB - text/x-diff)
file #37475:  t.png added by philipnienhuis (5KiB - image/png)
file #37470:  get_frame2.patch added by pantxo (10KiB - text/x-diff)
file #37465:  get_frame.patch added by pantxo (5KiB - text/x-diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by pantxo (Updated the item)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-01-20 rik5 Severity3 - Normal 1 - Wish
        Priority5 - Normal 1 - Later
    2017-05-28 pantxo CategoryOctave Function Plotting with gnuplot
        StatusPatch Submitted Confirmed
        Summarymissing getframe function missing getframe function in gnuplot
    2017-05-24 pantxo Attached File- Added getframe9.patch, #40784
    2017-05-24 pantxo Attached File- Added getframe8.patch, #40782
        StatusIn Progress Patch Submitted
    2017-05-18 pantxo Attached File- Added getframe5.patch, #40732
    2016-07-01 rik5 Carbon-CopyRemoved 72865 -
    2016-06-28 pantxo Attached File- Added getframe3.patch, #37617
    2016-06-14 philipnienhuis Attached File- Added t.png, #37475
    2016-06-14 pantxo StatusConfirmed In Progress
    2016-06-14 pantxo Attached File- Added get_frame2.patch, #37470
    2016-06-13 pantxo Attached File- Added get_frame.patch, #37465
    2016-06-10 rik5 StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code