bugGNU Octave - Bugs: bug #55648, [MXE-Octave] cannot print with...

 
 

bug #55648: [MXE-Octave] cannot print with gnuplot graphics toolkit

Submitter:  Avinoam Kalma <avinoam>
Submitted:  Tue 05 Feb 2019 10:13:04 PM UTC
   
 
Category:  Plotting with gnuplot Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Regression
Status:  Fixed Assigned to:  None
Originator Name:  Avinoam Open/Closed:  * Closed
Release:  * 5.0.91 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 19 Feb 2019 12:02:00 PM UTC, comment #30: 

Thanks for confirming.
Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Mon 18 Feb 2019 07:30:59 PM UTC, comment #29: 

Using octave stable version (hg_id = fb920b730931)
on windows, gnuplot works fine.
Thanks for the fix
This report can be closed.

Avinoam Kalma <avinoam>
Group Member
Fri 15 Feb 2019 09:10:46 AM UTC, comment #28: 

Avinoam, thanks for clarifying.

I pushed the patch to stable here:
https://hg.savannah.gnu.org/hgweb/octave/rev/5e97c26bf0e5

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Fri 15 Feb 2019 06:35:37 AM UTC, comment #27: 

What I meant is that I couldn't reproduce bug #53091 because printing gnuplot figures was broken,  and now that printing with gnuplot is possible again, I can see that bug #53091 still isn't fixed.

Avinoam Kalma <avinoam>
Group Member
Thu 14 Feb 2019 08:16:14 PM UTC, comment #26: 

@Avinoam: I am not sure I understand correctly.
Are you saying that bug #53091 was fixed before and the patch here caused a regression that made it re-appear?
Or are you saying that you couldn't reproduce bug #53091 because printing was broken completely? And now that printing with gnuplot is possible again, you can see that bug #53091 still isn't fixed?

Markus Mützel <mmuetzel>
Group administrator
Wed 13 Feb 2019 09:07:34 PM UTC, comment #25: 

This patch solve the problem, thanks.
But when I ran compare_plot_demos, I got again bug #53091

Avinoam Kalma <avinoam>
Group Member
Tue 12 Feb 2019 10:22:29 PM UTC, comment #24: 

I have confirmed that the patch solve the issue including my noise.
Thanks!

Tatsuro MATSUOKA <tmacchant>
Tue 12 Feb 2019 07:50:43 PM UTC, comment #23: 

The attached patch let's me print with gnuplot on Windows again.

"fclear" is necessary to reset the error state of the stream. Admittedly the documentation of fclear isn't very clear (no pun intended). Essentially, it calls std::io::clear() on the underlying C++ stream. This resets the eof state of the stream and allows to read from it again.
I don't think it is necessary to read the stream char by char. I added a token to the end of the received string instead to make sure it is completely read.

The command from comment #12 also no longer throws an error.

(file #46243)

Markus Mützel <mmuetzel>
Group administrator
Tue 12 Feb 2019 11:46:16 AM UTC, comment #22: 

Since the error message and error stack are the same, I estimate that the reason for the error is the same. I'll have a closer look.

Markus Mützel <mmuetzel>
Group administrator
Tue 12 Feb 2019 09:46:40 AM UTC, comment #21: 

What do you think about comment #15.
Function drawnow for gnuplot tookits on windows apparently has a problem.

print of gnuplot graphics toolkit uses drawnow("TERM",  "FILE")
If this has strange behavior, we have correct it the first, I think.


Tatsuro MATSUOKA <tmacchant>
Tue 12 Feb 2019 08:23:24 AM UTC, comment #20: 

Afaik, Octave doesn't know a "sleep" (i.e. "pause" without implicit "drawnow"). Maybe this could be a nice extension to what Matlab offers.

I'll try to come up with a changeset within the next few days.

Markus Mützel <mmuetzel>
Group administrator
Tue 12 Feb 2019 08:18:25 AM UTC, comment #19: 

Yeah, "pause" isn't what I meant there, but more like "sleep" so as to not waste too much CPU polling.  If pause is doing something that reads from the same pipe or i/o, then of course that really doesn't work.

Is the pause() necessary to get characters to move through the stream?  If not, drop the pause() and fclear() and see if there are any problems.  If the CPU goes up momentarily, that's not as bad as stuck in an infinite loop.

Dan Sebald <sebald>
Tue 12 Feb 2019 07:55:04 AM UTC, comment #18: 

Thanks, Dan, for the suggestion. It probably is better than what we have at the moment. But I don't think it would solve the issue at hand.
I'll try to better describe what is going on:
1. _gnuplot_get_var_ is called and a command is sent to gnuplot via fputs. Let's call this "thread A" (it's not different threads but I don't know a better word).
2. It waits for an answer and eventually calls "pause" which causes the callback queue to be flushed (implicit "drawnow").
3. A callback from the queue ("thread B") calls _gnuplot_get_var_ and sends a command to gnuplot via fputs.
4. For the sake of simplicity assume that this thread isn't interrupted again. So at a certain point it receives an answer from gnuplot. But that answer corresponds to the command send by "thread A" earlier. Since there are no further checks, in continuation this answer will be interpreted as if it corresponded to the command from "thread B".
5. Eventually the queue returns to "thread A" where an answer from gnuplot will be received. But that answer is to the latter command send from "thread B".

Maybe we could use persistent variables to collect and assign answers from different "threads". But if it is possible that fread could return early, that could get messy quite fast.

It would be easier (and cleaner imho) if the "pause" between fputs and fread could be avoided altogether. However, I don't how often _gnuplot_get_var_ is called and if that could lead to some kind of blocking.

I don't think that the real culprit is in drawnow but it really is the use of "pause" in _gnuplot_getvar_. Maybe a change might the issue occur more frequently. But this was always waiting to happen at some point.

Markus Mützel <mmuetzel>
Group administrator
Tue 12 Feb 2019 07:05:25 AM UTC, comment #17: 

Or more like this:


      str = [];
      time0 = time();
      while (time() - time0 < 10)
        cval = fread (istream, 1, "*char")';
        if (! isempty (cval))
          if (cval == SOME_END_OF_STRING_SYMBOL)
            str = regexp (str, 'OCTAVE:.*', "match");
            str = str{end}(8:end);
            break;
          else
            str = strcat(str, cval);
          endif
        else
          pause (0.05);
        endif
      endwhile


Dan Sebald <sebald>
Tue 12 Feb 2019 07:01:06 AM UTC, comment #16: 

To me, the code hunk in Comment #10 looks poorly constructed, almost bound to be an infinite loop depending on one's system speed, CPU load, etc.  Calling fclear(istream) could easily clear out the next string of characters that would represent a gnuplot output or partial gnuplot output (depending on how fread() waits/returns).  If that stream gets cleared after the next valid  character string is available then str will be empty from then on:


      while (isempty (str))
        str = fread (istream, "*char")';  % NOT AVAILABLE FROM GNUPLOT IMMEDIATELY
        if (isempty (str))
          pause (0.05);                   % ARRIVES FROM GNUPLOT WITHIN THIS TIME FRAME
        else
          str = regexp (str, 'OCTAVE:.*', "match");
          str = str{end}(8:end);
        endif
        fclear (istream);                 % GETS CLEARED AND ISTREAM IS EMPTY FROM HERE OUT
      endwhile


I think the hunk of code at line 95 of __gnuplot_get_var__ is a slightly better approach in that it reads until adequate characters are present (like a small buffer) and then treats that small buffer of characters as the return result.  Markus, can you devise a changeset more along those lines?  (I don't know MXE very well.)  Also, there really should be some kind of timer/timeout tied to this loop so that there is no hang if gnuplot can't interpret the command or crashes when executing the command.  E.g.,


      str = [];
      time0 = time();
      while (time() - time0 < 10)
        cval = fread (istream, 1, "*char")';
        if (! isempty (cval))
          str = strcat(str, cval);
        elseif (cval == SOME_END_OF_STRING_SYMBOL)
          str = regexp (str, 'OCTAVE:.*', "match");
          str = str{end}(8:end);
          break;
        else
          pause (0.05);
        endif
      endwhile


Dan Sebald <sebald>
Tue 12 Feb 2019 05:32:45 AM UTC, comment #15: 

I suspect that change in drawnow function C++ code is main point of this issue.

 drawnow( "pngcairo", "aaa", "debug.gp")

does not work on 5.0.9x on windows while works on linux.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 10:53:54 PM UTC, comment #14: 

I looked a code of libinterp/corefcn/graphics.cc

The codes of drawnow is changed between 4.4.1 and 5.0.9x.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 10:26:47 PM UTC, comment #13: 

Sorry:
On 4.4.1, this does not happen.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 10:25:42 PM UTC, comment #12: 

I think that this is related to drawnow itself


>> graphics_toolkit gnuplot
>> plot([0 1])
>> drawnow( "pngcairo", "aaa", "debug.gp")
error: __gnuplot_drawnow__: the gnuplot terminal, "pngcairo", is not available
error: called from
    __gnuplot_drawnow__ at line 50 column 9


On 4.4.1, this might happen.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 10:15:43 PM UTC, comment #11: 

@Tatsuro: Does the debug stop point make it work?

Sorry I did not use it.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 08:01:08 PM UTC, comment #10: 

The problem is probably the use of "pause" at around line 120 in _gnuplot_get_var_:

      while (isempty (str))
        str = fread (istream, "*char")';
        if (isempty (str))
          pause (0.05);
        else
          str = regexp (str, 'OCTAVE:.*', "match");
          str = str{end}(8:end);
        endif
        fclear (istream);
      endwhile


That allows the event queue being flushed that at that point seems to contain another call to this function. If I delete the pause command, printing works. But that might block other things.

CC'ing Dan Sebald as the expert for the gnuplot graphics toolkit.

Markus Mützel <mmuetzel>
Group administrator
Mon 11 Feb 2019 07:39:42 PM UTC, comment #9: 

I made a few debug outputs in _gnuplot_get_var_ to see what is sent to gnuplot and what is received.
It looks like the commands and the results get out of sync. The function seems to be interrupted by itself:

>> print test -dtikz
gp_var_name = GPVAL_TERMINALS
str =
gp_var_name = GPVAL_TERMINALS
str =
str = OCTAVE:  cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex lua mf mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tikz tkcanvas tpic unknown windows
wxt

gp_var_name = GPVAL_TERM
str =
str =
str =
str =
str = OCTAVE: wxt

str = OCTAVE:  cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex lua mf mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tikz tkcanvas tpic unknown windows
wxt

gp_var_name = GPVAL_TERMINALS
str =
gp_var_name = GPVAL_TERMINALS
str =
str =
str = OCTAVE:  cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex lua mf mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tikz tkcanvas tpic unknown windows
wxt

gp_var_name = GPVAL_TERM
str =
str = OCTAVE:  cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex lua mf mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tikz tkcanvas tpic unknown windows
wxt

str =
str =
str = OCTAVE: wxt

gnuplot_supports_term = 0
error: __gnuplot_drawnow__: the gnuplot terminal, "tikz", is not available
error: called from
    __gnuplot_drawnow__ at line 51 column 9
    __gnuplot_print__>local_drawnow at line 232 column 5
    __gnuplot_print__ at line 114 column 9
    print at line 692 column 14
gp_var_name = GPVAL_TERMINALS
str =
str =
str =
str =
str =
str =
str =
str =
str = OCTAVE:  cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex lua mf mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tikz tkcanvas tpic unknown windows
wxt

gp_var_name = GPVAL_TERM
str =
str =
str =
str =
str = OCTAVE: wxt


A "drawnow" in line 40 of _gnuplot_drawnow_ seems to prevent the error. But the hole construct seems to be very fragile. At least the !use_mkfifo part of _gnuplot_get_var_ relies on not being interrupted by callbacks that might start the same function again. There is no guarantee that that will never happen.

Markus Mützel <mmuetzel>
Group administrator
Mon 11 Feb 2019 05:03:47 PM UTC, comment #8: 

Adding pause(0.1) in line 40 of __gnuplot_drawnow__


plot_stream = __gnuplot_open_stream__ (2, h);
pause (0.1);
gnuplot_supports_term = __gnuplot_has_terminal__ (term, plot_stream);


Solves the problem. I do not know why and if  this is the correct solution.

Avinoam Kalma <avinoam>
Group Member
Mon 11 Feb 2019 04:25:29 PM UTC, comment #7: 

On mine, sometimes it seems to get stuck in the print.

Doing the debug stop does work. I can also print out some additional things in _gnuplot_has_terminal_ and that seems to be enough for it to work, so definatly a timing issue


Likely similar to issues fixed in #55528 ?

John Donoghue <lostbard>
Group Member
Mon 11 Feb 2019 08:44:07 AM UTC, comment #6: 

@Tatsuro: Does the debug stop point make it work?

Markus Mützel <mmuetzel>
Group administrator
Mon 11 Feb 2019 08:09:14 AM UTC, comment #5: 

I tried gnuplot 5.2.6 from official version.
 

gnuplot_binary C:\Programs\gp526-64\bin\gnuplot.exe
plot([0 1])
print ('-dps', 'a.ps')

does not give error messages but print freezes.

Tatsuro MATSUOKA <tmacchant>
Mon 11 Feb 2019 07:57:50 AM UTC, comment #4: 

I am seeing the same with 5.0.91 and the current development branch (see also bug #48474).
If I set a stop point in line 40 of gnuplot_drawnow and wait for the additional plot window to appear before I continue, it seems to work:

>> graphics_toolkit gnuplot
>> plot([0 1])
>> dbstop in __gnuplot_drawnow__ at 40
ans =  40
>> print test -dps
debug> dbcont
>>


So this might be a timing issue.

IMHO, this is a blocker for the Octave 5 release because it prevents printing with the gnuplot graphics toolkit.

Markus Mützel <mmuetzel>
Group administrator
Sun 10 Feb 2019 07:58:58 PM UTC, comment #3: 


Checking again version with hg id 496e7abe1662:


>> graphics_toolkit gnuplot
>> plot (rand(10))
>> print ('-dps', 'a.ps')
error: __gnuplot_drawnow__: the gnuplot terminal, "postscript", is not available
error: called from
    __gnuplot_drawnow__ at line 50 column 9
    __gnuplot_print__>local_drawnow at line 232 column 5
    __gnuplot_print__>eps_drawnow at line 220 column 5
    __gnuplot_print__ at line 165 column 7
    print at line 692 column 14
>>  system('gnuplot -e "print GPVAL_TERMINALS"')
cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex mf
mp pcl5 pdfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tkcanvas tpic un
known windows wxt



Avinoam Kalma <avinoam>
Group Member
Wed 06 Feb 2019 03:00:09 PM UTC, comment #2: 

comment #1 done with system('gnuplot -e "print GPVAL_TERMINALS"')

John Donoghue <lostbard>
Group Member
Wed 06 Feb 2019 02:57:02 PM UTC, comment #1: 

Not sure why the terminal error as querying gnuplot GPVAL_TERMINALS gives cairolatex canvas cgm context domterm dumb dxf eepic emf emtex epscairo epslatex fig hpgl latex mf mp pcl5 p
dfcairo pngcairo postscript pslatex pstex pstricks qms qt svg texdraw tgif tkcanvas tpic unknown windows wxt


John Donoghue <lostbard>
Group Member
Tue 05 Feb 2019 10:13:04 PM UTC, original submission:  

Using the script


cd  D:\Octave\Octave-5.0.91\mingw64\share\octave\5.0.91\m\testfun\private
addpath (pwd)
compare_plot_demos ('toolkits', 'gnuplot');


I get a lot of errors:


Printing "annotation_01.png" ...
(gnuplot.exe:4204): Pango-WARNING **: couldn't load font "Helvetic
a Not-Rotated 200", falling back to "Sans Not-Rotated 200", expect
 ugly output.

(gnuplot.exe:4204): Pango-WARNING **: couldn't load font "Helvetic
a Not-Rotated 400", falling back to "Sans Not-Rotated 400", expect
 ugly output.
ERROR in annotation_01: __gnuplot_drawnow__: the gnuplot terminal,
 "postscript", is not available


And there are no png files, only err files with "__gnuplot_drawnow__: the gnuplot terminal, "postscript", is not available"

Avinoam Kalma <avinoam>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #46243:  bug55648_gnuplot_get_var.patch added by mmuetzel (2KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by sebald (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by tmacchant (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by avinoam (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-02-19 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2019-02-15 mmuetzel StatusPatch Submitted Ready For Test
    2019-02-12 mmuetzel Attached File- Added bug55648_gnuplot_get_var.patch, #46243
        StatusNone Patch Submitted
    2019-02-11 mmuetzel Carbon-Copy- Added sebald
    2019-02-10 avinoam Summary[MXE-Octave] problems with gnuplot [MXE-Octave] cannot print with gnuplot graphics toolkit

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code