bugGNU Octave - Bugs: bug #59546, Missing latex interpreter

 
 

bug #59546: Missing latex interpreter

Submitter:  Pantxo Diribarne <pantxo>
Submitted:  Wed 25 Nov 2020 11:27:58 PM UTC
   
 
Category:  Plotting with OpenGL Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Feature Request
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 25 Nov 2021 04:24:48 PM UTC, comment #79: 

Seems to be working. Closing as fixed.

Pantxo, thanks again for this nice new feature.

Markus Mützel <mmuetzel>
Group administrator
Thu 18 Nov 2021 09:39:43 PM UTC, comment #78: 

Thanks Markus for reviewing and pushing the patch.

Pantxo Diribarne <pantxo>
Group Member
Thu 18 Nov 2021 07:46:37 PM UTC, comment #77: 

Sorry. Forgot about this report.

@Pantxo: Your patch looks good to me. I pushed it here:
http://hg.savannah.gnu.org/hgweb/octave/rev/f306fe9bfa0d

Markus Mützel <mmuetzel>
Group administrator
Thu 27 May 2021 09:14:30 PM UTC, comment #76: 

Ther is the remaining problem of ever growing and global cache. I attached a patch in which the cache is owned by the gh_manager and the set accessor takes care of limiting the cache size. I arbitrarily limited the number of cache entries to a fixed value of 500.

(file #51485)

Pantxo Diribarne <pantxo>
Group Member
Thu 27 May 2021 03:13:08 PM UTC, comment #75: 

This has been marked Ready for Test since April.  I think the new Latex interpreter works pretty well.  Is it okay to close?

Rik <rik5>
Group administrator
Fri 09 Apr 2021 08:08:43 PM UTC, comment #74: 

Thanks Markus for the continuous integration work! I pushed the patch here:

  http://hg.savannah.gnu.org/hgweb/octave/rev/78f42413e20f

I also fixed the copyright dates here:

  http://hg.savannah.gnu.org/hgweb/octave/rev/ddd25345734d


Pantxo Diribarne <pantxo>
Group Member
Fri 09 Apr 2021 06:53:01 PM UTC, comment #73: 

I compiled with your patch from comment #72 on top of 9f0649b4e912.
The results of the simple LaTeX test look good to me. The scaling seems to match for the three file types.


(file #51220)

Markus Mützel <mmuetzel>
Group administrator
Wed 07 Apr 2021 08:44:08 PM UTC, comment #72: 

@Markus: Great! Now that we use the logical coordinate system (through setWindow), I realize there is not need to scale our drawings based on the device's resolution. The attached patch simplifies the previous one by removing any need for the original screen's dpi. Could you test it before I push ?




(file #51210)

Pantxo Diribarne <pantxo>
Group Member
Wed 07 Apr 2021 01:43:00 PM UTC, comment #71: 

The change from comment #70 seems to have made the difference.
All three files (.png, .svg, and .pdf) look good to me.


(file #51205)

Markus Mützel <mmuetzel>
Group administrator
Tue 06 Apr 2021 05:23:19 PM UTC, comment #70: 

Thanks, so the problem is not the size of the output PDF (the properties show correct size) but rather the base transform that is applied when drawing. In the files you sent, I do see notable differences in the PDF for 100 and 125% scallings.
Does the following change (setViewport -> setWindow) have an effect on the output PDF?


diff -r d9b3402d52bd src/octave-svgconvert.cc
--- a/src/octave-svgconvert.cc  Mon Apr 05 19:34:04 2021 +0200
+++ b/src/octave-svgconvert.cc  Tue Apr 06 19:15:04 2021 +0200
@@ -66,7 +66,7 @@

     // Painter settings
     begin (&m_printer);
-    setViewport (QRect (0, 0, sz.width () * scl, sz.height () * scl));
+    setWindow (QRect (0, 0, sz.width () * scl, sz.height () * scl));
     scale (scl, scl);
   }


On linux this has no effect but it is expected since IIUC logical and device coordinates are the same on that platform.


Pantxo Diribarne <pantxo>
Group Member
Tue 06 Apr 2021 04:39:07 PM UTC, comment #69: 

I do have one monitor with DPI scaling (or Hi-DPI or whatever it is called) set to 125%. That is the monitor I usually have the Octave GUI running (and I used for the tests here). The second monitor is set to 100% scaling.

Running the example from comment #68 with my original settings (folder `125` in the .zip file) or 100% DPI scaling on both monitors (folder `100` in the .zip file) leads to similar results afaict.
The cropping of the .pdf file is off in both cases. If anything, the cropping seems to be worse if both monitors are set to 100% scaling.

By "default graphics settings", do you mean if I set any default values for graphics properties (e.g. in one of Octave's startup files)? Afaict, I don't set any of those. The startup files are empty atm (apart from some comments).

(file #51203)

Markus Mützel <mmuetzel>
Group administrator
Tue 06 Apr 2021 04:06:27 PM UTC, comment #68: 

@Markus: Thanks for testing. Do you have some default graphics settings related to figure and font size? I ask because in the png, the axes tick labels looks big as compared to what I obtain.

I expect that the simple figure below should have pretty much the same problems (and much lower output size):


figure ("units", "centimeters", "position", [2 2 12 9]);
plot (1:10);
title ("Some LaTeX", "interpreter", "latex")
print simple_latex.png
print simple_latex.svg
print -svgconvert -dpdfcrop simple_latex.pdf


Could you check that you have no default graphics setting and attach the three files generated by the above script?

Could you play with the display scaling on the Window OS (I'm talsking about something I heard exist...) and see if it has an influence on the incorrect pdf cropping? In which case this could be related to bug #49627.

Pantxo Diribarne <pantxo>
Group Member
Tue 06 Apr 2021 03:15:27 PM UTC, comment #67: 

I applied the patch from comment #63 locally on top of hg id be4b78fb4411 and tried to cross-compile for Windows.

It failed with "error: invalid use of incomplete type 'class QPainterPath'" and related errors.

Adding "#include <QPainterPath>" to the new file fixed the errors for me. See the attached modified version of your patch.

With that, I ran the following commands in Octave (after adding the path to the MikTeX binaries):

sombrero()
xlabel ("$x$", "interpreter", "latex")
ylabel ("$y$", "interpreter", "latex")
zlabel ('$\frac{\sqrt{x^2+y^2}}{x^2+y^2}$', "interpreter", "latex");
print -dpdfcrop -svgconvert sombrero.pdf
print sombrero.png


I'm attaching the two files.
The .pdf seems to be incorrectly cropped.


(file #51200, file #51202)

Markus Mützel <mmuetzel>
Group administrator
Tue 06 Apr 2021 12:37:23 PM UTC, comment #66: 

OK, I thought I was up to date but apparently I wasn't.  After updating to


@  changeset:   29500:be4b78fb4411
|  bookmark:    @
|  tag:         tip
|  user:        Markus Mützel <markus.muetzel@gmx.de>
|  date:        Fri Apr 02 19:28:07 2021 +0200
|  summary:     Replace Unicode conversion functions from gnulib with STL functions.


I don't see the dvipng message when executing the tests for graphics.cc.

Thanks, and sorry for the noise.

John W. Eaton <jwe>
Group administrator
Tue 06 Apr 2021 12:02:45 PM UTC, comment #65: 

@jwe: I have dvipng installed so in order to mimic the situation where it would'nt be installed I set OCTAVE_DVIPNG_BINARY="dvipngtst" prior to running tests, but cannot reproduce the issue in comment #64. After cset af41ebf3d1b3, this message should only appear (because of bug #44712) when the "latex" interpreter is actually requested, which should not happen in graphics.cc tests. Can you run `test graphics.cc verbose` to see in what specific test the message is shown? 


As for copyright dates, I will fix them:

  • base-text-renderer.cc contains mostly code extracted from ft-text-renderer.cc so those files should share the same copyright dates: 2013-2021
  • Others are new files which should only have 2021.



Pantxo Diribarne <pantxo>
Group Member
Tue 06 Apr 2021 11:38:44 AM UTC, comment #64: 

I noticed that if I don't have dvipng installed, there is a message about it missing when running "make check" but the tests pass:


  libinterp/corefcn/graphics.cc-tst ..............................sh: 1: dvipng: not found
 pass   53/53


Is that the behavior we want or should the test be failing because of an error here?

I also noticed that the new base-text-renderer.cc file has a copyright date of 2021 (only) even though it appears to be derived from ft-text-renderer.cc, so maybe it should have the same range of dates as the original file?  Similarly, the new latex-text-renderer.cc file has a copyright range of 2013-2021.  Is that all new code or derived from some other file(s)?  If it is all new, it should probably have 2021 only in the copyright year.

John W. Eaton <jwe>
Group administrator
Mon 05 Apr 2021 05:04:12 PM UTC, comment #63: 

@Markus, comment #56:

>> Is that borrowed code very dependent on the rest of Qt? How likely is it that this code depends on specific versions of Qt? Is it possible it'll break in combination with a different version of Qt?


Those are a bunch of static functions defined in a cpp file. AFAICS they depend mostly on base Qt types (QByteArray, QChar) and other base graphics classes that I would not expect to change much. But we never know.

The attached patch is an update of the one from comment #51 that fixes compilation against the current revision and parks Qt code in its own header file with the original license header.

(file #51194)

Pantxo Diribarne <pantxo>
Group Member
Fri 02 Apr 2021 09:55:39 PM UTC, comment #62: 

@leonardo: Unfortunately 3D scenes are not rendered very well in vector formats (see e.g. bug #45494) whether or not using "-svgconvert". This is especially true when lines z-fight with surfaces and gl2ps sorting algorithm fails badly.
What I was talking about in comment #52 is latex text (path) rendering: it seams to me that latex glyphs are rendered "fatter" in eps than in the original pdf.

Pantxo Diribarne <pantxo>
Group Member
Fri 02 Apr 2021 03:44:00 PM UTC, comment #61: 

@Pantxo: I'm trying some options with gs as you suggested in #52.

However, after this:


sombrero
print -svgconvert -dpdfcrop sombrero.pdf


I realize that even the pdf is not as good as the png version. The pdf looks just like file #51060.
If you compare the "peak" of the sombrero, it is easy to identify that the "lines" over the surface looks ugly.

Therefore, any attempt to convert pdf to eps will not produce a good looking sombrero.

Even using

print -dsvg

does not produces a good svg file.

Instead of this, I tryed to change the graph toolkit to gnuplot and the sombrero.pdf looks very good. Very similar to png (I'm considering only the surface in this comparison)

May it be an issue/limitation with gl2ps library?

Kind regards,
Leonardo

P.S. If it is not clear what I'm trying to say, I can upload a figure with both plots


Leonardo <pocker>
Fri 02 Apr 2021 11:43:31 AM UTC, comment #60: 

I pushed the patch from comment #58 here:

http://hg.savannah.gnu.org/hgweb/octave/rev/af41ebf3d1b3

It does not completely silence error messages, but it is due to what I think is a bug (see bug #60326).

Pantxo Diribarne <pantxo>
Group Member
Fri 02 Apr 2021 05:27:45 AM UTC, comment #59: 

Without dvipng, I see this message:


octave:1> sombrero
sh: 1: dvipng: not found
warning: latex_renderer: unable to compile "?"
warning: called from
    figure at line 96 column 7
    newplot at line 120 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5


Installing dvipng avoids the warning.

The patch from comment #58 also eliminates the warning for me.


John W. Eaton <jwe>
Group administrator
Thu 01 Apr 2021 09:38:53 PM UTC, comment #58: 

Can you test the attached patch? It differs the test for usability of the latex tool chain until needed.

(file #51175)

Pantxo Diribarne <pantxo>
Group Member
Thu 01 Apr 2021 08:48:28 PM UTC, comment #57: 

I second Markus' remark from comment #55.  Starting Octave and running "sombrero" now produces an annoying warning documented in bug #60316.


warning: latex_renderer: unable to compile "?"
warning: called from
    figure at line 96 column 7
    newplot at line 120 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5


This warning is issued the first time any figure is created, and I'm not using the latex renderer, as far as I know.

Rik <rik5>
Group administrator
Thu 01 Apr 2021 09:20:43 AM UTC, comment #56: 

@jwe: Thanks.


Re comment #51:

> Or should I put the borrowed code in a separate file (with the original license text) and #include it in the main program file?


@Pantxo: Having that code in a separate file might indeed be cleaner and make it easier to possibly merge changes from upstream in the future.
Is that borrowed code very dependent on the rest of Qt? How likely is it that this code depends on specific versions of Qt? Is it possible it'll break in combination with a different version of Qt?
But I guess we'd just have to use some configure tests and "#if defined"s like we do for the GUI in that case...

Markus Mützel <mmuetzel>
Group administrator
Thu 01 Apr 2021 06:43:59 AM UTC, comment #55: 

Re comment #53:

> >> Is it possible that creating the plots for the manual somehow fails if there is no complete latex toolchain installed on the builder?
>
> That should not happen. I'll look into it.


I no longer think that an incomplete LaTeX toolchain was the issue (see comment #50).

Checking for a working toolchain delays the first plot for me notably even if it doesn't contain any graphics objects that have their "interpreter" property set to "latex".

Would it be possible to delay the check for a working toolchain until it is actually needed? That would also avoid the torrent of warnings that appears at the moment when building the manual on a system with incomplete LaTeX toolchain (see also bug #60316).

Markus Mützel <mmuetzel>
Group administrator
Thu 01 Apr 2021 06:43:43 AM UTC, comment #54: 

Markus: I installed that package on my buildbot systems.

John W. Eaton <jwe>
Group administrator
Wed 31 Mar 2021 08:42:02 PM UTC, comment #53: 

@Markus:

>> Is it possible that creating the plots for the manual somehow fails if there is no complete latex toolchain installed on the builder?


That should not happen. I'll look into it.

Pantxo Diribarne <pantxo>
Group Member
Wed 31 Mar 2021 08:40:04 PM UTC, comment #52: 

@leonardo: The warning cannot be too specific since eps may just be an intermediate format when not using the -svgconvert option. For example:


octave:1> plot (1:10);
octave:2> title ("Some LaTeX", "interpreter", "latex");
octave:3> print -painters toto.png
warning: print: unhandled LaTeX strings. Use -svgconvert option or -d*latex* output device.


Now for the poor eps output when using -svgconvert I guess someone could investigate to see if he can find some "gs" option that improves things. If you are interested to do the detective work, the current tool chain is equivalent to:


plot (1:10);
title ("Some LaTeX", "interpreter", "latex");

print -svgconvert -dpdfcrop toto.pdf
cmd = "gs -dQUIET -dNOPAUSE -dBATCH -dSAFER -dAutoRotatePages=/None -sDEVICE=ps2write -dLanguageLevel=2 -dEmbedAllFonts=true -dEPSCrop -sOutputFile=toto.eps toto.pdf";
system (cmd)




Pantxo Diribarne <pantxo>
Group Member
Wed 31 Mar 2021 08:09:16 PM UTC, comment #51: 

@Markus: About license issues (comment #48): I checked and yes it is explicitly stated that we can use the code, see excerpt from the original file below. Now the question is just how to do it in practice.


...
**
** GNU General Public License Usage
** Alternatively, this file may be used under the terms of the GNU
** General Public License version 2.0 or (at your option) the GNU General
** Public license version 3 or any later version approved by the KDE Free
** Qt Foundation. The licenses are as published by the Free Software
** Foundation and appearing in the file LICENSE.GPL2 and LICENSE.GPL3
** included in the packaging of this file. Please review the following
** information to ensure the GNU General Public License requirements will
** be met: https://www.gnu.org/licenses/gpl-2.0.html and
** https://www.gnu.org/licenses/gpl-3.0.html.


The patch is attached.
Or should I put the borrowed code in a separate file (with the original license text) and #include it in the main program file?


(file #51168)

Pantxo Diribarne <pantxo>
Group Member
Wed 31 Mar 2021 04:15:51 PM UTC, comment #50: 

Inspecting the logs a little better, it turns out that Octave was built without GUI. IIRC, it must be compiled with GUI to build the tarball.
It looks like it didn't detect the Qt5Svg module.

CC'ing jwe: Could you please install `libqt5svg5-dev` on your buildbots? We currently need that for building the GUI.
IIUC, Pantxo is working on a change that will remove that requirement again. But it might be good to have the buildbots running again sooner.

Markus Mützel <mmuetzel>
Group administrator
Wed 31 Mar 2021 04:06:00 PM UTC, comment #49: 

The buildbots seem to be having trouble building the tarball after this change:
http://buildbot.octave.org:8010/#/builders/14/builds/1055/steps/7/logs/stdio

> make[3]: * No rule to make target 'doc/interpreter/octave_interpreter.qhc', needed by 'distdir-am'.  Stop.


There are a lot of blocks like the following in the logs:

  GEN      doc/interpreter/interpderiv1.eps
warning: latex_renderer: unable to compile "?"
warning: called from
    axes at line 107 column 8
    newplot at line 161 column 10
    plot at line 230 column 9
    geometryimages at line 108 column 5


Is it possible that creating the plots for the manual somehow fails if there is no complete latex toolchain installed on the builder?

I'd assume that having a complete latex toolchain shouldn't be necessary because we don't use the LaTeX renderer for those plots anyway.

Anyway, would it be possible to only warn if the renderer is actually set to "latex" and skip the toolchain check entirely otherwise?

Markus Mützel <mmuetzel>
Group administrator
Wed 31 Mar 2021 03:46:57 PM UTC, comment #48: 

Re comment #44:

> I have a fix that consists in reverting most of the changes and adding support for path elements in our hand-rolled svg renderer. For this the simplest way was to borrow some chunks of code in Qt's source code. Is it OK to do so?

Afaics, Qt is licensed under LGPLv3. IIUC, that license is compatible with the GPLv3 used by Octave.
To my understanding, it is ok to use code that you find in Qt (as long as it is covered by the LGPLv3) and use it in Octave (as long as it is covered by the GPLv3). I'm not sure if we'd need to assign the copyright of that part of the code separately though.
Maybe you should ask on the maintainers mailing list or the discourse forum.

Markus Mützel <mmuetzel>
Group administrator
Wed 31 Mar 2021 03:13:35 PM UTC, comment #47: 

Indeed.

I read the warning message and created the eps in both ways and concluded the same as you that


print -depsc2 -svgconvert sombrero.eps


does not look as good as


print -depslatexstandalone sombrero.tex


My question was more in the direction of if another alternative would be implemented. Just curiosity.

Although the warning message is pretty clear, what about changing it to look like this:


warning: print: unhandled LaTeX strings. Use -svgconvert option or -d*latex*output device, e.g, 'print -svgconvert -depsc2 output.eps' or 'print -depslatexstandalone output.tex'


It may be a bit too much, but it could help.

kind regards,
Leonardo

Leonardo <pocker>
Wed 31 Mar 2021 02:47:14 PM UTC, comment #46: 

For eps there are two options as explained in the warning text. Either use the -svgconvert of -dlatexstandalone:


clf;
sombrero ();
xlabel ("$x$", "interpreter", "latex")
ylabel ("$y$", "interpreter", "latex")
zlabel ('$\frac{\sin\left(\sqrt{x^2+y^2}\right)}{\sqrt{x^2+y^2}}$', "interpreter", "latex");
print -svgconvert -depsc2 sombrero_svg.eps
print -depslatexstandalone sombrero_latex.tex
system "latex sombrero_latex.tex"
system "dvips sombrero_latex.dvi"


The second option gives better results.

Pantxo Diribarne <pantxo>
Group Member
Wed 31 Mar 2021 01:21:51 PM UTC, comment #45: 

Hi.

if I run this


clf;
sombrero ();
xlabel ("$x$", "interpreter", "latex")
ylabel ("$y$", "interpreter", "latex")
zlabel ('$\frac{\sqrt{x^2+y^2}}{x^2+y^2}$', "interpreter", "latex");
print -depsc2 sombrero.eps


this warning appears:


warning: print: unhandled LaTeX strings. Use -svgconvert option or -d*latex*output device.
warning: called from
    __opengl_print__ at line 206 column 9
    print at line 771 column 16


the sombrero.eps file is generated without labels

Will be possible to implement this interpreter to create .eps files correctly ?

HG-ID 2ae4764180c6

Leonardo <pocker>
Tue 30 Mar 2021 07:11:48 PM UTC, comment #44: 

I finally pushed the patch after adding some documentation:

http://hg.savannah.gnu.org/hgweb/octave/rev/2ae4764180c6

Apart from the already known issue with the cache, I just noticed another problem: the QSvgRenderer renders text as path, which means not only LaTeX strings but also normal ones loose textual font info.
This is a regression and it looks like using QSvgRenderer was a bad option since there is not much you can do to control what it does in the background.

I have a fix that consists in reverting most of the changes and adding support for path elements in our hand-rolled svg renderer. For this the simplest way was to borrow some chunks of code in Qt's source code. Is it OK to do so?

Pantxo Diribarne <pantxo>
Group Member
Sat 27 Mar 2021 02:03:25 PM UTC, comment #43: 

Ok, let's do it that way, I'll look into the cache issue next week. I won't be able to push until monday though.

Pantxo Diribarne <pantxo>
Group Member
Fri 26 Mar 2021 06:44:18 PM UTC, comment #42: 

Thanks for explaining.

I pushed the patch from comment #40 for MXE Octave here:
https://hg.octave.org/mxe-octave/rev/1855ad40685a

And a patch that lets Octave look for the octave-svgconvert executable in the "bin" directory on Windows here:
https://hg.savannah.gnu.org/hgweb/octave/rev/7aa1994c0ca2

I made a few very minor changes to your patch (typos, copyright year update and alike). See the attachment.

I'd say it is ready to be pushed as an initial version.

We could follow up with changes (like a replacement for the static, ever growing, non thread-safe cache) here or in new reports.

What do you think?


(file #51146)

Markus Mützel <mmuetzel>
Group administrator
Thu 25 Mar 2021 08:20:18 PM UTC, comment #41: 

Good news! Thanks for digging into this.

I'd vote for option 2, i.e. keep the change specific to Windows.

I don't know much about packaging rules, but I think we don't  want to expose octave-svgconvert in the puclic (/usr/)bin directory on linux. This is an Octave specific program with many shortcomings, far from a general purpose svg converter as the name could suggest, and it should be treated as an internal if possible.

IIRC on Windows the bin directory is private (not on the Windows PATH by default) and contains pretty much everything including the dll libraries, so why not add octave-svgconvert.exe there if it fixes the linking issue.

Pantxo Diribarne <pantxo>
Group Member
Thu 25 Mar 2021 05:36:45 PM UTC, comment #40: 

Attached is a patch that changes the installation directory of "octave-svgconvert" to the "bin" directory and adjusts the `__svgconv_binary__` function.
With that change to `__svgconv_binary__` and the executable manually moved to the "bin" directory, I'm able to print to pdf using `-svgconvert` on Windows.

I'm not sure this is a good change though.
Why do we install some binaries in the "bin" directory and others in "archlib"?

Alternatively, we could copy the file to the bin directory in MXE Octave. We already do something similar for "octave-gui.exe".
But that would make finding the binary in `__svgconv_binary__` slightly more awkward.
I'm attaching a patch for the part that would need to be done in MXE Octave. We won't need this patch if we used the other patch in core Octave.
I don't have a patch yet that would need to accompany this one in core Octave.

I'll wait for feedback on which path we should take.


(file #51137, file #51138)

Markus Mützel <mmuetzel>
Group administrator
Thu 25 Mar 2021 09:58:35 AM UTC, comment #39: 

I was wondering why "octave-svgconvert.exe" wasn't installed with all the other binaries in "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\bin".
I moved it there and - it is working!

So a fix might be to change the build rule to install that binary in the "default" binary location. But I don't know why it is installed in the other location in the first place.
Any idea why that is?

Markus Mützel <mmuetzel>
Group administrator
Thu 25 Mar 2021 09:45:50 AM UTC, comment #38: 

That seems to fail. At least, I can't see the pdf file:

>> system('"D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\libexec\octave\7.0.0\exec\x86_64-w64-mingw32\octave-svgconvert.exe" "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\sombrero.svg" pdf 96.13 "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\share\octave\7.0.0\fonts\FreeSans.otf" 1 "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\output.pdf"')
ans = -1.0737e+09
>> typecast(ans, "uint64")
ans = 13965662441850273792
>> typecast(ans, "int64")
ans = -4481081631859277824


The cast of the return value to uint64 and int64 was in the hope that might be helpful. But I'm not sure what it means.
It looks like a random number. But it is always the same.

I've ran the same command in the msys2 shell that is part of the MXE Octave dist bundle and it worked.
I had to `set QT_PLUGIN_PATH=%OCTAVE_HOME%qt5\plugins` because that wasn't done automatically. After that:

$ echo $QT_PLUGIN_PATH
D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\qt5\plugins


This variable is set in Octave though:

>> getenv('QT_PLUGIN_PATH')
ans = D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\qt5\plugins


If that variable isn't set, I see the following:

Markus@Markus-PC MINGW64 ~
$ "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\libexec\octave\7.0.0\exec\x86_64-w64-mingw32\octave-svgconvert.exe" "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\sombrero.svg" pdf 96.13 "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\share\octave\7.0.0\fonts\FreeSans.otf" 1 "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\output.pdf"
qt.qpa.plugin: Could not find the Qt platform plugin "windows" in ""
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.


Markus@Markus-PC MINGW64 ~
$ echo $?
127


But that is probably to be expected and unrelated to the issue here.

I tried the same at the CLI (all previous tests were done using the GUI).
On executing the command, a window with the following message popped up:

> octave-svgconvert.exe - Einsprungpunkt nicht gefunden
> Der Prozedureinsprungpunkt "_Z11qt_assert_xPKcS0_S0_i" wurde in der DLL "D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\libexec\octave\7.0.0\exec\x86_64-w64-mingw32\octave-svgconvert.exe" nicht gefunden.


IIUC, that means there is a missing symbol (demangled: "qt_assert_x(char const*, char const*, char const*, int)").
The same message box repeated for "_Z5qHashRK7QStringj" (demangled: "qHash(QString const&, unsigned int)").
Those symbols look like Qt library functions. I don't know why it isn't looking for them in a library though. Maybe a linking error?

It is strange that it is working when called from the msys2 shell but not when it's called from the Octave prompt.
That makes me suspicious that something might be wrong with how we run a system command in Octave.

It's not the environment variables though afaict:

>> system('echo %QT_PLUGIN_PATH%')
D:\SVN\Octave\test\octave-2021-03-23-17-34-w64-latex\octave-2021-03-23-17-34-w64\mingw64\qt5\plugins
ans = 0


Any ideas what to do next?

Markus Mützel <mmuetzel>
Group administrator
Thu 25 Mar 2021 08:23:46 AM UTC, comment #37: 

What happens if you try to run octave-svgconvert on the file from comment #34? something along


cmd = 'C:\path\to\octave-svgconvert C:\path\to\sombrero.svg pdf 96.13 C:\path\to\install_dir\etc\fonts\FreeSans.otf 1 C:\path\to\output.pdf'
system (cmd)


I'll try to review the changes I made in octave-svgconvert but the "internal pipe error" is supposed to happen when Octave doesn't manage to write to the pipe (fwrite doesn't return the expected number of written characters), which in turn I don't expect to depend on the program(s) at the other end of the pipe.

Pantxo Diribarne <pantxo>
Group Member
Wed 24 Mar 2021 03:19:07 PM UTC, comment #36: 

Similar error:

>> close all
>> plot (1:10)
>> title ("This is a LaTeK formatted string", "interpreter", "latex")
* Caching This is a LaTeK formatted string:11.000000:1:0:0:0 (numel: 5)
>> print -dpdfcrop -svgconvert simple.pdf
Error: /undefinedfilename in (C:\\Users\\Markus\\AppData\\Local\\Temp\\oct-Q3U9IY)
Operand stack:

Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostri
ngval--   --nostringval--   false   1   %stopped_push
Dictionary stack:
   --dict:733/1123(ro)(G)--   --dict:0/20(G)--   --dict:75/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.50: Unrecoverable error, exit code 1
warning: print: No such file or directory, 'C:\Users\Markus\AppData\Local\Temp\oct-Q3U9IY'
warning: called from
    print at line 794 column 9

>> error: gl2ps_renderer::draw: internal pipe error
error: called from
    __opengl_print__ at line 204 column 9
    print at line 766 column 16
>>


This was with those lines still commented out. (But I'd guess that's not important.)

I noticed that the patch also contains changes to the octave-svgconvert executable.
Do those have to be done at the same time? Would it be possible to untangle those changes and the latex-renderer to better pinpoint what might be causing the issue?

Markus Mützel <mmuetzel>
Group administrator
Wed 24 Mar 2021 03:07:36 PM UTC, comment #35: 

The untouched SVG looks good to me, at least as good as on linux. Maybe the issue is with large files? Did you try to print a simpler figure, e.g.:


plot (1:10)
title ("This is a LaTeK formatted string", "interpreter", "latex")
print -dpdfcrop -svgconvert simple.pdf


Pantxo Diribarne <pantxo>
Group Member
Wed 24 Mar 2021 02:38:03 PM UTC, comment #34: 

I commented out those lines and printing to svg seems to work.
No errors or warnings.
The resulting file is quite large. Let's see if savannah will allow to attach it.


(file #51130)

Markus Mützel <mmuetzel>
Group administrator
Wed 24 Mar 2021 02:25:42 PM UTC, comment #33: 

Could you comment out the following block in scripts/plot/util/private/__print_parse_opts__.m and see what happens when printing to SVG without using the -svgconvert step?


...
  ## By default, postprocess svg files using svgconvert.
  if (strcmp (arg_st.devopt, "svg"))
    arg_st.svgconvert = true;
  endif
...


Pantxo Diribarne <pantxo>
Group Member
Wed 24 Mar 2021 02:16:15 PM UTC, comment #32: 


>> setenv ("OCTAVE_LATEX_DEBUG_FLAG", "1")

>> sombrero ()

* Caching ?:10.000000:0:0:0:0 (numel: 1)
>> xlabel ("$x$", "interpreter", "latex")

* Caching $x$:11.000000:0:38:38:38 (numel: 2)
>> ylabel ("$y$", "interpreter", "latex")

* Caching $y$:11.000000:2:38:38:38 (numel: 3)
>> zlabel ('$\frac{\sqrt{x^2+y^2}}{x^2+y^2}$', "interpreter", "latex");

* Caching $\frac{\sqrt{x^2+y^2}}{x^2+y^2}$:11.000000:1:38:38:38 (numel: 4)
>> print sombrero.svg
>> error: gl2ps_renderer::draw: internal pipe error
error: called from
    __opengl_print__ at line 204 column 9
    print at line 766 column 16


No file.

Fwiw, I tried to print a figure without LaTeX to svg, and it failed with the same error:

>> close all
>> sombrero ()

>> print sombrero.svg

>> error: gl2ps_renderer::draw: internal pipe error
error: called from
    __opengl_print__ at line 204 column 9
    print at line 766 column 16
>>


I repeated these steps in a new Octave session with the same results.

I still had a build from approx. 3 weeks ago lying around (hg id 3f45f5b6fa7f). With that, printing to svg and pdf still worked.

PS: I probably hit "Preview" in comment #30 after attaching the files. Next try.

file #51129)

Markus Mützel <mmuetzel>
Group administrator
Wed 24 Mar 2021 01:58:13 PM UTC, comment #31: 

Good news! Can you try to print to SVG and see what happens?

Pantxo Diribarne <pantxo>
Group Member
Wed 24 Mar 2021 01:35:42 PM UTC, comment #30: 

Thanks. That seems to have done the trick for on screen LaTeX on Windows:

>> setenv ("OCTAVE_LATEX_DEBUG_FLAG", "1")

>> sombrero ()

* Caching ?:10.000000:0:0:0:0 (numel: 1)
>> xlabel ("$x$", "interpreter", "latex")
* Caching $x$:11.000000:0:38:38:38 (numel: 2)
>> ylabel ("$y$", "interpreter", "latex")
* Caching $y$:11.000000:2:38:38:38 (numel: 3)
>> zlabel ('$\frac{\sqrt{x^2+y^2}}{x^2+y^2}$', "interpreter", "latex");
* Caching $\frac{\sqrt{x^2+y^2}}{x^2+y^2}$:11.000000:1:38:38:38 (numel: 4)
>> print sombrero.png
* Caching $\frac{\sqrt{x^2+y^2}}{x^2+y^2}$:22.916667:1:38:38:38 (numel: 5)
* Caching $y$:22.916667:2:38:38:38 (numel: 6)
* Caching $x$:22.916667:0:38:38:38 (numel: 7)
>> print -dpdfcrop -svgconvert sombrero.pdf
Error: /undefinedfilename in (C:\\Users\\Markus\\AppData\\Local\\Temp\\oct-1RMqKS)
Operand stack:

Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostri
ngval--   --nostringval--   false   1   %stopped_push
Dictionary stack:
   --dict:733/1123(ro)(G)--   --dict:0/20(G)--   --dict:75/200(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.50: Unrecoverable error, exit code 1
warning: print: No such file or directory, 'C:\Users\Markus\AppData\Local\Temp\oct-1RMqKS'
warning: called from
    print at line 794 column 9

>> error: gl2ps_renderer::draw: internal pipe error
error: called from
    __opengl_print__ at line 204 column 9
    print at line 766 column 16
>>


It looks like it printed the .png file successfully, too. Very good news.

But the .pdf is empty afaict.
Fwiw, there is no folder (or file) 'C:\Users\Markus\AppData\Local\Temp\oct-1RMqKS' after the last command.

Both files are attached.

Markus Mützel <mmuetzel>
Group administrator
Mon 22 Mar 2021 08:38:04 PM UTC, comment #29: 

Indeed, according to this post [1], it should be enough (at least with miktex and texlive, the most used latex distributions) to set the -output-directory option in order to direct both dvi and log files. So here is yet another patch, which uses this option instead of redirection.
The output of the compilation is now shown in case of failure together with the command and the error message if any.

[1] https://tex.stackexchange.com/a/7272

(file #51117)

Pantxo Diribarne <pantxo>
Group Member
Mon 22 Mar 2021 06:57:11 PM UTC, comment #28: 

Fwiw, I ran the following three commands in that directory:

"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\latex" -interaction=nonstopmode -output-directory=. default.tex
"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\dvisvgm" -n -TS1 -o default.svg default.dvi
"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\dvipng" default.dvi -o default.png -bg Transparent -D 72


After that the following files exist:

C:\Users\Markus\AppData\Local\Temp\latexl38Hbg>dir /b
default.aux
default.dvi
default.log
default.png
default.svg
default.tex


Markus Mützel <mmuetzel>
Group administrator
Mon 22 Mar 2021 06:39:47 PM UTC, comment #27: 

I cross-compiled with the patch from comment #25, made the same modifications to the PATH in the startup script and ran the commands from comment #26:

>> setenv ("OCTAVE_LATEX_DEBUG_FLAG", "1")
>> sombrero ()
warning: latex_renderer: unable to compile "?"
* Command: "latex -interaction=nonstopmode -output-directory="C:\Users\Markus\AppData\Local\Temp\latexl38Hbg" "C:\Users\Markus\AppData\Local\Temp\la
texl38Hbg\default.tex" > "C:\Users\Markus\AppData\Local\Temp\latexl38Hbg\default.log""
warning: called from
    figure at line 96 column 7
    newplot at line 120 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5

warning: latex_renderer: unusable latex tool-chain
warning: called from
    axes at line 107 column 8
    newplot at line 161 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5


I opened a command prompt and inspected that directory:

C:\Users\Markus\AppData\Local\Temp\latexl38Hbg>dir /b
default.log
default.tex

C:\Users\Markus\AppData\Local\Temp\latexl38Hbg>type default.tex
\documentclass[10pt, varwidth]{standalone}
\usepackage{amsmath}
\usepackage[utf8]{inputenc}
\begin{document}
\begin{flushleft}
?
\end{flushleft}

\end{document}
C:\Users\Markus\AppData\Local\Temp\latexl38Hbg>type default.log
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (MiKTeX 21.3)
entering extended mode
! I can't write on file `default.log'.
Please type another transcript file name
! Emergency stop
No pages of output.


So, at least it looks like it picked up the correct executable.

Maybe we can't redirect stdout to "default.log" because pdfTeX will try (and fail) to write its own log to a file with the same name? Maybe we could redirect to "default.out" or something else instead?

Markus Mützel <mmuetzel>
Group administrator
Mon 22 Mar 2021 04:16:50 PM UTC, comment #26: 

Oh, and with this patch, you need to set an environment variable to have debug outputs and preserved temporary directories:


setenv ("OCTAVE_LATEX_DEBUG_FLAG", "1")
sombrero ()


Pantxo Diribarne <pantxo>
Group Member
Mon 22 Mar 2021 02:42:25 PM UTC, comment #25: 

Sorry, the latest patch did not include the mentioned changes (forgot to `qreffresh`). You are going to regret what you said about cross-compiling not being exceedingly long :-).

I attached the updated version.

(file #51113)

Pantxo Diribarne <pantxo>
Group Member
Mon 22 Mar 2021 12:57:29 PM UTC, comment #24: 

The attached updated patch implements modification proposed in comment #23 plus fixes Windows specific output redirection (>NUL instead of >/dev/null).

(file #51112)

Pantxo Diribarne <pantxo>
Group Member
Mon 22 Mar 2021 12:10:31 PM UTC, comment #23: 

@Markus: You are right, as compared to using Fsystem, like you did in comment #18, the latex interpreter misses the Windows specific block from comment #22 before the call to "run_command_and_return_output". I'll add that.
This also makes me think that we should surround file names in the command with quotes since the temporary directory in which they reside can potentially contain white spaces and other problematic characters (parenthesis, non-ascii, ...). I'll update the patch to do that as well.

Pantxo Diribarne <pantxo>
Group Member
Mon 22 Mar 2021 10:18:42 AM UTC, comment #22: 

We have this code block in `Fsystem` in toplev.cc before `run_command_and_return_output` is called:

#if defined (OCTAVE_USE_WINDOWS_API)
  // Work around weird double-quote handling on Windows systems.
  if (type == et_sync)
    cmd_str = '"' + cmd_str + '"';
#endif


Maybe we'd need something similar in the `latex_renderer::render` function.

Don't worry. I'm happy if I can be helpful.
You did all the heavy lifting, and I'm thankful for that.
If the changes are in Octave only and I have a "half-baked" MXE Octave tree ready, cross-compiling doesn't take exceedingly long anyway.

Markus Mützel <mmuetzel>
Group administrator
Mon 22 Mar 2021 10:09:49 AM UTC, comment #21: 

@Markus: Those files look correct indeed. I attached a patch in which I commented out the cleanup of temporary directories so that you can inspect them after a failure. The compile failure warning should also show the command that failed. With this it should be easier to pinpoint the origin of the failure.

It is a pity that for such simple changes, you have to go through the whole process of cross compiling Octave. Many thanks for your time!

(file #51109)

Pantxo Diribarne <pantxo>
Group Member
Sun 21 Mar 2021 07:59:10 PM UTC, comment #20: 

Unfortunately, I don't know where I can find which packages were missing now that they have been installed. I think I remember "standalone", "varwidth", "luainputenc", "amscls", "amsmath". But there was a bunch more that I can't remember.

I'm attaching the resulting .png and .svg files. Afaict, they look ok.

file #51104)

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Mar 2021 07:32:27 PM UTC, comment #19: 


>> Maybe I need to let it install more packages?

If the commands now run without errors, I don't think you need to install additional packages. Before the cleanup, can you inspect the resulting png and svg files?


...
  open (png);
  open (svg);
unwind_protect_cleanup
...


>> Is the default.tex file in comment #17 equivalent to the test that is run when initializing the latex_renderer?


Yes, default.tex is the actual file that is generated for the purpose of testing the latex tool chain.


PS: Can you note the packages you had to install to get the default.tex file to compile? This maybe help full to know in the future.

Pantxo Diribarne <pantxo>
Group Member
Sun 21 Mar 2021 07:10:58 PM UTC, comment #18: 

Thanks. That was very helpful.

I tried to run the script line by line. But it failed when I tried to execute the latex command:

>>   cmd = [latex_binary, " -interaction=nonstopmode -output-directory=", d, ...
         " ", tex];
>>   [status, msg] = system (cmd);
qt.qpa.plugin: Could not load the Qt platform plugin "windows" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: direct2d, minimal, offscreen, windows.


When I tried to execute the following similar command at the Windows shell, several dialogs popped up that asked to install a bunch of additional MiKTeX packages:

D:\SVN\Octave\bug59546_latex>"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\latex" -interaction=nonstopmode -output-directory=. default.tex
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (MiKTeX 21.3)
entering extended mode
(default.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18>
(C:\Program Files\MiKTeX 2.9\tex/latex/standalone\standalone.cls
Document Class: standalone 2018/03/26 v1.3a Class to compile TeX sub-files stan
dalone
(C:\Program Files\MiKTeX 2.9\tex/latex/latex-tools\shellesc.sty)
(C:\Program Files\MiKTeX 2.9\tex/generic/iftex\ifluatex.sty
(C:\Program Files\MiKTeX 2.9\tex/generic/iftex\iftex.sty))
(C:\Program Files\MiKTeX 2.9\tex/latex/xkeyval\xkeyval.sty
(C:\Program Files\MiKTeX 2.9\tex/generic/xkeyval\xkeyval.tex
(C:\Program Files\MiKTeX 2.9\tex/generic/xkeyval\xkvutils.tex
(C:\Program Files\MiKTeX 2.9\tex/generic/xkeyval\keyval.tex))))
(C:\Program Files\MiKTeX 2.9\tex/latex/standalone\standalone.cfg)
(C:\Program Files\MiKTeX 2.9\tex/latex/base\article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(C:\Program Files\MiKTeX 2.9\tex/latex/base\size10.clo))
(C:\Program Files\MiKTeX 2.9\tex/latex/varwidth\varwidth.sty))
(C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsmath.sty
For additional information on amsmath, use the `?' option.
(C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amstext.sty
(C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsgen.sty))
(C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsbsy.sty)
(C:\Program Files\MiKTeX 2.9\tex/latex/amsmath\amsopn.sty))
(C:\Program Files\MiKTeX 2.9\tex/latex/base\inputenc.sty)
(C:\Program Files\MiKTeX 2.9\tex/latex/l3backend\l3backend-dvips.def)
(default.aux) [1] (default.aux) )
Output written on D:\SVN\Octave\bug59546_latex\default.dvi (1 page, 356 bytes).
Transcript written on D:\SVN\Octave\bug59546_latex\default.log.


After that the original command from the script executed successfully in Octave.

I'm guessing that the Qt that MiKTeX seems to be expecting for the package installation dialog is incompatible with the Qt libraries that are already loaded by Octave.

The same happened for the other two commands.
Basically, I executed these three commands in a Windows shell and allowed MiKTeX's package manager to install all packages it asked for:

"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\latex" -interaction=nonstopmode -output-directory=. default.tex
"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\dvisvgm" -n -TS1 -o default.svg default.dvi
"C:\Program Files\MiKTeX 2.9\miktex\bin\x64\dvipng" default.dvi -o default.png -bg Transparent -D 72


After that, a slightly changed `tstwin` script executed without errors (see attachment).

However, `sombrero` still gives the same warnings.
Maybe I need to let it install more packages? Is the default.tex file in comment #17 equivalent to the test that is run when initializing the latex_renderer?


(file #51101)

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Mar 2021 06:19:22 PM UTC, comment #17: 


>> Can I somehow intercept what the latex_renderer tries to compile and try to execute the respective commands manually?


No, sorry, as I said in comment #10, the patch is not currently debug friendly. I will update the patch to add some useful terminal outputs. In the mean time, can you try to reproduce the tool chain with the attached .tex file. It should go (approximately) as follows (attached as tstwin.m):


## Create a temporary directory
d = tempname ();
mkdir (d)
copyfile ("./default.tex", d);

## Run latex toolchain
base_path = 'C:\Program Files\MiKTeX 2.9\miktex\bin\x64';
#base_path = '/usr/bin';

tex = fullfile (d, "default.tex");
dvi = fullfile (d, "default.dvi");
latex_binary = sprintf ('"%s"', fullfile (base_path, "latex"));
png = fullfile (d, "default.png");
png_binary = sprintf ('"%s"', fullfile (base_path, "dvipng"));
svg = fullfile (d, "default.svg");
svg_binary = sprintf ('"%s"', fullfile (base_path, "dvisvgm"));

unwind_protect
  cmd = [latex_binary, " -interaction=nonstopmode -output-directory=", d, ...
         " ", tex];
  [status, msg] = system (cmd);
  if (status)
    error ("Running latex: %s", msg);
  endif

  cmd = [svg_binary, " -n -TS1 -o ", svg, " ", dvi];
  [status, msg] = system (cmd);
  if (status)
    error ("Running dvisvg: %s", msg);
  endif

  cmd = [png_binary, " ", dvi, " -o ", png, " -bg Transparent -D 72"];
  [status, msg] = system (cmd);
  if (status)
    error ("Running dvipng: %s", msg);
  endif
unwind_protect_cleanup
  rmdir (d)
end_unwind_protect


(file #51099, file #51100)

Pantxo Diribarne <pantxo>
Group Member
Sun 21 Mar 2021 05:31:58 PM UTC, comment #16: 

I believe Windows automatically tries to find executables with a set of file extensions (PATHEXT) when trying to execute a file without specifying its extension. On my system, that is:

>> getenv("PATHEXT")
ans = .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC


Anyway, I opened Octave and tried the commands from comment #15 before running anything else. I still see the same when trying to plot:

>> base_path = 'C:\Program Files\MiKTeX 2.9\miktex\bin\x64';
>> setenv ("OCTAVE_LATEX_BINARY", sprintf ('"%s"', fullfile (base_path, "latex.exe")))
>> setenv ("OCTAVE_DVIPNG_BINARY", sprintf ('"%s"', fullfile (base_path, "dvipng.exe")))
>> setenv ("OCTAVE_DVISVG_BINARY", sprintf ('"%s"', fullfile (base_path, "dvisvgm.exe")))
>> sombrero ();
warning: latex_renderer: unable to compile "?"
warning: called from
    figure at line 96 column 7
    newplot at line 120 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5

warning: latex_renderer: unusable latex tool-chain
warning: called from
    axes at line 107 column 8
    newplot at line 161 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5


The three executables installed by MiKTeX:

C:\Program Files\MiKTeX 2.9\miktex\bin\x64>latex --version
MiKTeX-pdfTeX 4.6 (MiKTeX 21.3)
© 1982 D. E. Knuth, © 1996-2021 Hàn Thế Thành
TeX is a trademark of the American Mathematical Society.
using bzip2 version 1.0.8, 13-Jul-2019
compiled with curl version 7.72.0; using libcurl/7.72.0 Schannel
compiled with expat version 2.2.10; using expat_2.2.10
compiled with jpeg version 9.4
compiled with liblzma version 50020052; using 50020052
compiled with libpng version 1.6.37; using 1.6.37
compiled with libressl version LibreSSL 3.1.4; using LibreSSL 3.1.4
compiled with MiKTeX Application Framework version 4.1; using 4.1
compiled with MiKTeX Core version 4.5; using 4.5
compiled with MiKTeX Archive Extractor version 4.0; using 4.0
compiled with MiKTeX Package Manager version 4.2.1; using 4.2.1
compiled with uriparser version 0.9.4
compiled with xpdf version 4.02
compiled with zlib version 1.2.11; using 1.2.11

C:\Program Files\MiKTeX 2.9\miktex\bin\x64>dvipng --version
This is dvipng 1.16 Copyright 2002-2015, 2019 Jan-Ake Larsson
dvipng 1.16
MiKTeX 21.3
Compiled with Freetype 2.10.3
Using libft 2.10.3
Copyright (C) 2002-2015, 2019 Jan-Ake Larsson.
There is NO warranty.  You may redistribute this software
under the terms of the GNU Lesser General Public License
version 3, see the COPYING file in the dvipng distribution
or <http://www.gnu.org/licenses/>.

C:\Program Files\MiKTeX 2.9\miktex\bin\x64>dvisvgm --version
dvisvgm 2.11.1


Can I somehow intercept what the latex_renderer tries to compile and try to execute the respective commands manually?

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Mar 2021 04:34:11 PM UTC, comment #15: 

I realize that even with latex binaries in the path, Octave will try to invoke "latex", not "latex.exe". Maybe the following could work:


base_path = 'C:\Program Files\MiKTeX 2.9\miktex\bin\x64';
setenv ("OCTAVE_LATEX_BINARY", sprintf ('"%s"', fullfile (base_path, "latex.exe")))
setenv ("OCTAVE_DVIPNG_BINARY", sprintf ('"%s"', fullfile (base_path, "dvipng.exe")))
setenv ("OCTAVE_DVISVG_BINARY", sprintf ('"%s"', fullfile (base_path, "dvisvgm.exe")))
sombrero ();


Note that I surrounded the binaries with quotes, hopping that it will allow for running binaries with white space in the path.

Pantxo Diribarne <pantxo>
Group Member
Sun 21 Mar 2021 11:57:39 AM UTC, comment #14: 

I compiled for Windows with the last patch from comment #8 using MXE Octave.

Cross-building with that change worked without issues.

On Windows, I installed MikTeX and added the following line to Octave's startup script to add the path to "latex.exe", "dvipng.exe", and "dvisvgm.exe" to the PATH variable:

wshSystemEnv("PATH") = "C:\Program Files\MiKTeX 2.9\miktex\bin\x64;" & wshSystemEnv("PATH")


With that change, I see the following when trying to plot something:

>> sombrero ();

warning: latex_renderer: unable to compile "?"
warning: called from
    figure at line 96 column 7
    newplot at line 120 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5

warning: latex_renderer: unusable latex tool-chain
warning: called from
    axes at line 107 column 8
    newplot at line 161 column 10
    surf at line 76 column 9
    sombrero at line 68 column 5


Is there something I can do to track down why it is failing?

Markus Mützel <mmuetzel>
Group administrator
Mon 15 Mar 2021 07:48:05 PM UTC, comment #13: 

@Markus: Indeed the gh_manager looks like a good candidate for storing cached data, but I still didn't get a chance to try and tackle this issue properly and currently left it as a FIXME.
Note that the ft_manager singleton object, which manages already loaded font faces, is equivalently unsafe (global and non-thread safe) and will have to be taken care as well. Just to say that this is a pretty big change... Could we envisage this as a second step, after the new interpreter has been integrated and eventually tested on other platforms?

As for the ever growing cache, I'll have a look, but this could take some time. Limiting the cache to a fixed maximum number of elements by using a specialized class is probably enough.

Pantxo Diribarne <pantxo>
Group Member
Mon 15 Mar 2021 04:56:11 PM UTC, comment #12: 

@Pantxo: Thanks for the example. I knew I was missing something basic -- like entering math mode with `$` (d'oh).
When used correctly, your patch works like a charm.

I remember you asked on the discourse forum about how to manage the cache:
https://octave.discourse.group/t/cache-manager-class/519

IIUC, jwe prefers to not have a static singleton for that.
Do you think the `gh_manager` could manage the cache instead? IIUC, this is what you suggested in that thread. (But it's quite possible I misunderstood.)

If I read the patch correctly, the cache uses a map that is ever growing until Octave is closed.
Afaict, these questions haven't been answered yet in the other thread: Do we need to limit the amount of memory the cache occupies? Do we need functions that purge the cache once in a while?

The closest I can think of that we have to a cache manager might be the `load_path` class and its function map. But I'm not sure if much of it could be useful here...

Maybe the interpreter or parser caches parsed code somewhere. But I don't know that part of the code base very well...


Markus Mützel <mmuetzel>
Group administrator
Mon 15 Mar 2021 04:14:31 PM UTC, comment #11: 

@pantxo: I tried your patch with HG-ID 4a516dce916c+ and it is working.

Both png and pdf (with you code from comment #8) are just fine.

Congrats on this!

Leonardo <pocker>
Mon 15 Mar 2021 01:12:19 PM UTC, comment #10: 

@Markus: In the current state of the patch, if one of the latex steps fails, the temporary directory in which the tex and log files are stored is deleted. This makes it hard to debug ... If needed I can provide an option to allow those directories to not be deleted for the purpose of debugging.

Pantxo Diribarne <pantxo>
Group Member
Sun 14 Mar 2021 03:47:19 PM UTC, comment #9: 

I messed the sombrero formula, shame on me, here it is:


Pantxo Diribarne <pantxo>
Group Member
Sun 14 Mar 2021 03:42:27 PM UTC, comment #8: 

@Markus: Yes, the Qt5Svg module is required. I have no strong opinion on whether it should be possible to build without it, but anyway, this can be taken care last.

You'll also need three binaries: latex, dvipng (for onscreen rendering), and dvisvgm (for printing). If thos binaries are not on the path, you can provide their location throught three environment variables: OCTAVE_LATEX_BINARY, OCTAVE_DVIPNG_BINARY OCTAVE_DVISVG_BINARY.

I attached an updated version of the patch that fixes a few issues. For me, with this patch (but also with the previous one), using the latex interpreter is then as easy as

clf;
sombrero ();
xlabel ("$x$", "interpreter", "latex")
ylabel ("$y$", "interpreter", "latex")
zlabel ('$\frac{\sqrt{x^2+y^2}}{x^2+y^2}$', "interpreter", "latex");
print -dpdfcrop -svgconvert sombrero.pdf
print sombrero.png


This produces the following image (png and pdf attached).



(file #51059, file #51060,

Pantxo Diribarne <pantxo>
Group Member
Sun 14 Mar 2021 01:30:56 PM UTC, comment #7: 

To make use of the new feature, I had to in install `libqt5svg5-dev` on my Ubuntu 20.10. Without that package, building the GUI was completely disabled.
Should we require the Qt5Svg library for the GUI from now on? Or would it make sense to selectively disable only the LaTeX feature without that package? The former is probably good enough imho.

But even with that package installed, I failed to produce a plot with LaTeX labels or texts.

Could you please provide an example to make sure I'm using the correct commands?

Markus Mützel <mmuetzel>
Group administrator
Fri 27 Nov 2020 04:47:52 PM UTC, comment #6: 

Probably this won't help, but I have been using a program called Xournall++ and it has LaTeX input box. I don't know how it works under the hood, but it uses texlive

https://xournalpp.github.io/guide/tools/#latex-tool

But, more important:

For this preliminary implementation of a latex interpreter, I would like to say Thank You Pantxo!!

Kind regards,
Leonardo

Leonardo <pocker>
Fri 27 Nov 2020 04:19:13 PM UTC, comment #5: 

This sounds like a great addition to Octave.  Also, I'm not particularly bothered by speed (within reason).  The core bits of Octave, such as computing the inverse of a matrix, need to be very fast because it happens so frequently.  But for code paths which are used infrequently, some slowness is acceptable.

An example already in the Octave code base is our use of print_usage() when input validation has failed.  The function goes and reads the m-file as text, finds and extracts just the Texinfo, calls out to the shell to run makeinfo on the Texinfo source, and then presents it to the screen.  That is very slow indeed, but it doesn't matter because it is only used when an error has occurred which isn't frequent.

The same thing applies here.  Latex plots aren't made all the time and if they take a little longer it is acceptable.

Regarding discrepancies between screen and printout, how bad will it be if the programmer uses one of the standalone formats such as "pdflatexstandalone"?  I would think the consistency would be very high in this case.

Rik <rik5>
Group administrator
Thu 26 Nov 2020 09:46:31 AM UTC, comment #4: 

@Kai: Here is his repository, on Jordi's site

http://inversethought.com/hg/octave-lojdl

@Dmitri: The only limitation I can see is that gl2ps has its own standard template for latex files and font names are ignored. So there would be a discrepancy between the on-screen and printout results when using -dxxlatex formats (not when using -svgconvert to produce pdf or svg directly).

Pantxo Diribarne <pantxo>
Group Member
Thu 26 Nov 2020 02:12:39 AM UTC, comment #3: 

I wonder if using xetex would be better (wrt font suppost) than
straight latex.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 26 Nov 2020 02:06:34 AM UTC, comment #2: 

Great teaser! I want it, even if it is slow 👍  Expecting high-quality graphics to be generated in less than a second is quite impossible, also MathJax on websites takes "ages" if the number of formulas increases.

In the commit message you wrote that it is started from a GSoC 2013 project by "Andrej Lojdl" https://wiki.octave.org/Summer_of_Code Where is his legacy to find?

Kai Torben Ohlhus <siko1056>
Group Member
Wed 25 Nov 2020 11:36:21 PM UTC, comment #1: 

I attached a teaser:






Pantxo Diribarne <pantxo>
Group Member
Wed 25 Nov 2020 11:27:58 PM UTC, original submission:  

The attached patch is a preliminary implementation of a latex interpreter for graphics.

The new latex_renderer searches for latex, dvipng and dvisvgm executables on the path and eventually uses them to render pixels on-screen, or path (not characters) in SVG and PDF formats.

The process is quite slow, even after caching already parsed strings (see debug messages appearing each time a new string is parsed), and I don't think we can do much better without implementing our own latex engine.

The implementation has to be polished, in particular it uses an ugly global variable for caching (which is not thread safe among others), but I'd like to have feedback and eventually advises before proceeding.

Pantxo Diribarne <pantxo>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #51485:  bug59546_cache.patch added by pantxo (5KiB - text/x-patch)
file #51220:  simple_latex.zip added by mmuetzel (25KiB - application/x-zip-compressed)
file #51210:  bug59546_svgconvert_4.patch added by pantxo (46KiB - text/x-patch)
file #51205:  simple_latex-setWindow.zip added by mmuetzel (25KiB - application/x-zip-compressed)
file #51203:  simple-latex.zip added by mmuetzel (46KiB - application/x-zip-compressed)
file #51200:  bug59546_svgconvert_v3.patch added by mmuetzel (46KiB - application/octet-stream)
file #51201:  sombrero.png added by mmuetzel (71KiB - image/png)
file #51202:  sombrero.pdf added by mmuetzel (59KiB - application/pdf)
file #51194:  bug59546_svgconvert_2.patch added by pantxo (46KiB - text/x-patch)
file #51168:  bug59546_svgconvert_1.patch added by pantxo (46KiB - text/x-patch)
file #51146:  bug59546_6.patch added by mmuetzel (61KiB - text/plain)
file #51130:  sombrero.svg added by mmuetzel (2MiB - image/svg+xml)
file #51128:  sombrero.png added by mmuetzel (71KiB - image/png)
file #51129:  sombrero.pdf added by mmuetzel (2KiB - application/pdf)
file #51117:  bug59546_5.patch added by pantxo (61KiB - text/x-patch)
file #51113:  bug59546_4.patch added by pantxo (62KiB - text/x-patch)
file #51112:  bug59546_3.patch added by pantxo (61KiB - text/x-patch)
file #51109:  bug59546_2.patch added by pantxo (61KiB - text/x-patch)
file #51103:  default.png added by mmuetzel (155B - image/png)
file #51104:  default.svg added by mmuetzel (1KiB - image/svg+xml)
file #51101:  tstwin.m added by mmuetzel (1KiB - text/plain)
file #51099:  tstwin.m added by pantxo (1KiB - text/x-objcsrc)
file #51100:  default.tex added by pantxo (160B - text/x-tex)
file #51062:  sombrero.png added by pantxo (136KiB - image/png)
file #51059:  bug59546.patch added by pantxo (60KiB - text/x-patch)
file #51060:  sombrero.pdf added by pantxo (247KiB - application/pdf)
file #51061:  sombrero.png added by pantxo (136KiB - image/png)
file #50346:  latex_tst.png added by pantxo (278KiB - image/png)
file #50345:  latex_support_2.patch added by pantxo (56KiB - text/x-patch)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by pocker (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by pantxo (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-11-25 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2021-05-27 pantxo Attached File- Added bug59546_cache.patch, #51485
    2021-04-09 mmuetzel Attached File- Added simple_latex.zip, #51220
    2021-04-07 pantxo Attached File- Added bug59546_svgconvert_4.patch, #51210
    2021-04-07 mmuetzel Attached File- Added simple_latex-setWindow.zip, #51205
    2021-04-06 mmuetzel Attached File- Added simple-latex.zip, #51203
    2021-04-06 mmuetzel Attached File- Added bug59546_svgconvert_v3.patch, #51200
        Attached File- Added sombrero.png, #51201
        Attached File- Added sombrero.pdf, #51202
    2021-04-05 pantxo Attached File- Added bug59546_svgconvert_2.patch, #51194
    2021-04-01 pantxo Attached File- Added bug59546_differ_test_1.patch, #51175
    2021-04-01 mmuetzel Dependencies- bugs #60320 is dependent
    2021-03-31 pantxo Attached File- Added bug59546_svgconvert_1.patch, #51168
        StatusPatch Submitted Ready For Test
    2021-03-31 mmuetzel Carbon-Copy- Added jwe
    2021-03-26 mmuetzel Attached File- Added bug59546_6.patch, #51146
    2021-03-25 mmuetzel Attached File- Added bug59546-svgconvert-bin.patch, #51137
        Attached File- Added bug59546-mxe-svgconvert-bin.patch, #51138
    2021-03-24 mmuetzel Attached File- Added sombrero.svg, #51130
    2021-03-24 mmuetzel Attached File- Added sombrero.png, #51128
        Attached File- Added sombrero.pdf, #51129
    2021-03-22 pantxo Attached File- Added bug59546_5.patch, #51117
    2021-03-22 pantxo Attached File- Added bug59546_4.patch, #51113
    2021-03-22 pantxo Attached File- Added bug59546_3.patch, #51112

    Back to the top

    Powered by Savane 3.13-caa5.
    Corresponding source code