bugGNU Octave - Bugs: bug #39828, Missing symbols from new TeX...

 
 

bug #39828: Missing symbols from new TeX interpreter

Submitter:  Rik <rik5>
Submitted:  Tue 20 Aug 2013 10:24:33 PM UTC
   
 
Category:  Plotting Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  goffioul
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 05 Dec 2014 06:57:58 PM UTC, comment #45: 

I made the change here http://hg.savannah.gnu.org/hgweb/octave/rev/e72052d96a13.  '\0' now produces the emptyset symbol.  Closing report again.

Rik <rik5>
Group administrator
Tue 02 Dec 2014 10:36:38 PM UTC, comment #44: 

Rik: Sounds right to me.


Michael Godfrey <godfrey>
Group Member
Tue 02 Dec 2014 06:43:13 PM UTC, comment #43: 

Oops, in my last post it should be '\O', a slash followed by a zero, which I think should be mapped to the empty set.

Rik <rik5>
Group administrator
Tue 02 Dec 2014 06:39:50 PM UTC, comment #42: 

Re-opening this bug about the TeX interpreter to ask a question about two symbols


\O : mapped to 0x2298 (Circled Division Slash)
\oslash : mapped to 0x2298


In Matlab, both of these symbols are mapped to 0x2205 (Empty Set).

In TeX


\O : mapped to 0xD8 (Scandinavian Captial O with slash)
\oslash : mapped to 0x2298 (Circled Division Slash)


Finally, TeX also has


\emptyset : mapped to 0x2205 (Empty Set)


It seems to me that Octave should map the capital '\O' to the Empty Set.  This is compatible with Matlab and gives user's a way to actually get a circle where the slash line extends outside the diameter of the circle, unlike '\oslash' where the slash line is within the diameter of the circle.

As an addition, since Octave is a superset of Matlab, we could also offer the expansion '\emptyset' which would also expand to the emptyset.

Is this a good solution?

Rik <rik5>
Group administrator
Tue 01 Oct 2013 05:18:42 AM UTC, comment #41: 
Rik <rik5>
Group administrator
Tue 01 Oct 2013 04:27:18 AM UTC, comment #40: 

Attached fixes \deg display in Manual.

Actually, I think that I knew about this
change some time ago, but forgot.  Will
check at CTAN.


(file #29278)

Michael Godfrey <godfrey>
Group Member
Tue 01 Oct 2013 04:13:53 AM UTC, comment #39: 

OK.  Sometime fairly recently someone decided to
replace \deg by ^{\circ}

I will look into how this happened, but in the meantime
I will submit a patch soon.

Michael Godfrey <godfrey>
Group Member
Tue 01 Oct 2013 04:04:29 AM UTC, comment #38: 

Actually, the plot.txi file is correct according
to the TeX Manual.  But $\deg$ fails to show the
degree symbol in the current TeX.  I will check on this.

Michael Godfrey <godfrey>
Group Member
Tue 01 Oct 2013 03:51:02 AM UTC, comment #37: 

Rik,

Sorry about that. This is an error in the plot.txi
source.  \deg works correctly when used in Octave.

I will fix this soon.

Michael

Michael Godfrey <godfrey>
Group Member
Tue 01 Oct 2013 03:30:38 AM UTC, comment #36: 

Michael,

I applied your patch here (http://hg.savannah.gnu.org/hgweb/octave/rev/3670ce8da1a0).  One more thing to review would be the '\deg' symbol at the very end of table 15.1.  In the PDF this is showing up as 'deg' rather than the symbol.

Rik <rik5>
Group administrator
Mon 30 Sep 2013 04:13:03 PM UTC, comment #35: 

Rik,

The patch which is attached simply
provides a constant width to all the
subtables in Table 15.1.  This improves
the appearance quite a bit.

Michael


(file #29271)

Michael Godfrey <godfrey>
Group Member
Tue 24 Sep 2013 12:43:20 AM UTC, comment #34: 

I applied your changeset here (http://hg.savannah.gnu.org/hgweb/octave/rev/710b3d5fe966).  I changed the multicolumn table for Info to have only 4 columns, since that is indeed what it had.

Rik <rik5>
Group administrator
Mon 23 Sep 2013 10:06:27 PM UTC, comment #33: 

I think that this is a reasonably final set
of corrections to plot.txi and
oct-tex-symbols.in

My previous patch files are obsolete.

The output od the "doc" command looks OK.
The result in octave.pdf could look a bit
nicer, but it is, I believe, all correct.

My test code writes all the symbols in a plot
and they appear to be all correct.

The attached changeset is: tex_table_update2.diff

So, this is done, I hope.



(file #29205)

Michael Godfrey <godfrey>
Group Member
Mon 23 Sep 2013 08:31:20 PM UTC, comment #32: 

I am just completing the texinfo part
of the plot.txi code.

I will post a changeset which replaces the
current one soon.

Michael Godfrey <godfrey>
Group Member
Sun 22 Sep 2013 03:49:08 AM UTC, comment #31: 

And, specifically, the change to
oct-tex-symbols.in was to change
the unicode value of 0 to 0x2298 to
make it match the MS symbol code and Matlab.


Michael Godfrey <godfrey>
Group Member
Sun 22 Sep 2013 12:44:13 AM UTC, comment #30: 

I should also have said that this only updates
the @tex  part of plot.txi, so the results
are only for octave.pdf.

Michael Godfrey <godfrey>
Group Member
Sun 22 Sep 2013 12:20:31 AM UTC, comment #29: 

The patch which is attached updates the Table in the
Manual to match the new symbols.  It groups the
symbols as in the TeX Manual.

The integration of this into plot.txi is a bit
crude.  The Table now extends over 2 pages and I
did not really know how to make that happen smoothly.
The best I could do causes a Tabele 15.1 for the
first part and Table 15.2 for the second.

Also, each sub-table has its width set for the symbols
in it.  So, the subtables are not all the same width.
I think that this is OK, and probably better than making
the all the width of the widest.  But, others may have
other views.

In any case all the symbols are there and in the right
subtable and in the right order.

Finally, one unicode error was fixed in oct-tex-symbols.in

(file #29193)

Michael Godfrey <godfrey>
Group Member
Fri 20 Sep 2013 01:02:58 AM UTC, comment #28: 

I should have added that I used the attached script
to test that all symbols display correctly.


(file #29181)

Michael Godfrey <godfrey>
Group Member
Fri 20 Sep 2013 12:58:38 AM UTC, comment #27: 

A patch is attached which simply sorts the
current set of symbols in the order in which they
appear in The TeXbook.  This will make it
much easier to maintain this file as/if additional
symbols are added.

The way in which this file is parsed made it difficult
(for me) to annotate the file as clearly as I had intended

An annotated oct-tex-symbols.in is attached for the
record, or it could be used by someone who knows how
the annotations must be formatted.

Next should be to organize the Manual Table to show the
symols in the same sort order...


(file #29179, file #29180)

Michael Godfrey <godfrey>
Group Member
Fri 30 Aug 2013 12:23:50 AM UTC, comment #26: 
Michael Goffioul <goffioul>
Thu 29 Aug 2013 11:35:17 PM UTC, comment #25: 

I've applied the patch. I'll submit a changeset for the degree symbol shortly.

Michael Goffioul <goffioul>
Thu 29 Aug 2013 10:35:57 PM UTC, comment #24: 

Global settings that don't affect program results (like what the prompt looks like) are OK, but global settings that affect program behavior are generally bad because it makes getting reproducible results harder.  If I really want \circ to do the TeX thing no matter who runs it, then my script has to ensure that the global variable is set properly while displaying \circ.

We used to have many settings like this that affected program behavior.  It was a lot of work to remove them and I don't want to repeat the mistake.

Yes, we should have \deg to get the degrees symbol.

John W. Eaton <jwe>
Group administrator
Thu 29 Aug 2013 10:26:46 PM UTC, comment #23: 

What about '\deg' for the degree symbol since that would now be missing. 

Also, you suggested adding the remaining TeX symbols.  Do you want to do that or is it too much work for too little gain?

Another more global way to handle these repeated inconsistencies would be to have a "Quirks" mode which would consult a global configuration variable before deciding whether to do things the Octave way or the Matlab way.

Rik <rik5>
Group administrator
Thu 29 Aug 2013 08:53:05 PM UTC, comment #22: 

Based on John's recommendation (below) I am attaching
a changeset to provide the correct unicodes for
circ and bullet.

This should close this report.



(file #28957)

Michael Godfrey <godfrey>
Group Member
Thu 29 Aug 2013 03:09:24 AM UTC, comment #21: 

Could we please keep the discussion on the tracker?  If you send mail directly to the mailing list it won't be archived in the tracker.  Or, if you want to discuss on a list, please use the maintainers list.

I suspect that we will get bug reports about the wong placement of \circ no matter what we do.  My inclination is to do what TeX does and tell people that this is one of those things about Matlab that we don't want to copy.

John W. Eaton <jwe>
Group administrator
Sun 25 Aug 2013 04:41:27 PM UTC, comment #20: 

Michael,

I looked up the Unicode Table at:
http://unicode-table.com/en/#mathematical-operators

and found:

circ            0x2218
bullet          0x2219

These look right to me as the attached jpg
will, I hope, show.  It is also appropriate
that the Unicode folks think that they are
math operators just as Knuth does.

I have not gone through Metafont to find exactly
how Knuth does it.

The symbols look a bit small, but they match the
size shown on page 436 (Appendix F: font tables 7.
Binary operators) in the TeX book.  So, I think
they are the right choice.

So now, I think these should be used, and also
include deg using  00B0.




Michael Godfrey <godfrey>
Group Member
Sun 25 Aug 2013 02:50:41 AM UTC, comment #19: 

No, I don't. My main sources of information are:

http://www.utf8-chartable.de/unicode-utf8-table.pl
http://www.kostis.net/charsets/symbol.htm

If you can figure out a better character, please let me know.

Michael Goffioul <goffioul>
Sun 25 Aug 2013 01:35:20 AM UTC, comment #18: 


I also just noticed that the code you had used for
the "real" \circ is too big.  The
text(.5,.5, char(176))
produces a \circ which looks like it is the right size
but is not centered vertically. It is too high. Do you
know the correct code for a \circ which is the same
size and position as as \bullet?


Michael Godfrey <godfrey>
Group Member
Sun 25 Aug 2013 01:19:44 AM UTC, comment #17: 

Oh.  I probably knew that once.  Sorry.
In most places in the Manual the text part
of arguments to plotting functions are shown in
double quotes.  In the section on use of
"interpreter" the example does use single quotes,
but this is not mentioned in the text.

I will think about a documentation update.

Michael Godfrey <godfrey>
Group Member
Sat 24 Aug 2013 11:51:23 PM UTC, comment #16: 

This is due to the use of double-quote instead of single quotes. In that case, the \b character is interpreted by octave as the backspace character. This then triggers the warning, as ascii character code 8 (backspace) doesn't have any corresponding glyph.

Michael Goffioul <goffioul>
Sat 24 Aug 2013 09:41:06 PM UTC, comment #15: 

The latest patchs seem to have broken "tex" mode:
octave:9> gh=title("\circ \bullet 123 \mu","interpreter","tex")
warning: ft_render: skipping missing glyph for character '8'
warning: ft_render: skipping missing glyph for character '8'
gh = -5.1478
octave:10>            get(gh,'interpreter')
ans = tex

Michael Godfrey <godfrey>
Group Member
Sat 24 Aug 2013 07:00:03 PM UTC, comment #14: 


>>Although my preference was to keep a real \circ,


I definitely think that your preference is the correct choice.
No one benefits from blindly copying Matlab errors.  If the
logic of foloowing Matlab errors is to be followed I should
file a substantial number of bug reports so that more errors
can be introduced into Octave.

If others, including John and Jordi and Rik agree that more errors
should be introduced, I will go ahead.  Other wise,  \circ
should be  \circ as defined in TeX and as expected.

Michael Godfrey <godfrey>
Group Member
Sat 24 Aug 2013 06:49:31 PM UTC, comment #13: 

Although my preference was to keep a real \circ, I opted for the degree symbol for compatibility. As explained in comment #8, it seems the use of the degree symbol is intentional.

This is implemented with this changeset:

http://hg.savannah.gnu.org/hgweb/octave/rev/fe6518a1d87c

Michael Goffioul <goffioul>
Sat 24 Aug 2013 12:06:20 AM UTC, comment #12: 

I don't care much either way.  I'm just concerned that if a user already has an m-file that produces some pretty graph and labels it, then they should be able to run that in Octave.  Asking users to search and replace "\\circ" with "\\deg" isn't very convenient.

But, if we're keeping \circ then we should definitely add \deg so that users have an easy way of accessing that symbol.

Rik <rik5>
Group administrator
Fri 23 Aug 2013 11:08:57 PM UTC, comment #11: 

I definitely prefer circ is circ
And, it would be friendly to add deg
for degree.

Separately, some or all of theTeX defined
symbols as given in the TeX Book should be
considered.  But, this is not for this
bug report.

Michael Godfrey <godfrey>
Group Member
Fri 23 Aug 2013 10:51:07 PM UTC, comment #10: 

The problem with "char(176)" should be fixed with the following changeset:

http://hg.savannah.gnu.org/hgweb/octave/rev/c8809de193a3

One then only need to decide what to do with "\circ". I don't have a strong opinion on the matter. I'm ok with any solution (real \circ or degree symbol).

Michael Goffioul <goffioul>
Fri 23 Aug 2013 09:16:33 PM UTC, comment #9: 

This is very tacky. TeX provides deg
for the degree symbol.

I suggest that circ be implemented as circ
and that we add deg for the degree symbol.

Matlab users who are that concerned about the
difference should be using Octave anyhow!

Michael Godfrey <godfrey>
Group Member
Fri 23 Aug 2013 08:59:58 PM UTC, comment #8: 

I'm pretty sure Matlab intended this to be a degree symbol.  See this posting on how to get a degree symbol (http://www.mathworks.com/support/solutions/en/data/1-2CH660/index.html).

This is important for Octave user's because they can't get a degree symbol the other way by using the popularly mapped ASCII code 176.  See this bug report (https://savannah.gnu.org/bugs/?func=detailitem&item_id=33155).

Example Code:


text (0.5, 0.5, char (176))
warning: ft_render: skipping missing glyph for character 'ffffffb0'



Rik <rik5>
Group administrator
Fri 23 Aug 2013 08:08:05 PM UTC, comment #7: 

I think a real \circ makes more sense than a degree symbol. It can be easily changed, but I'm not sure this is something we want to do.

Michael Goffioul <goffioul>
Fri 23 Aug 2013 07:53:23 PM UTC, comment #6: 

attached is a PDF of:
text (0.5, 0.5, '123 \circ 456 \bullet 789')

It also looks to me closer to a degree symbol, but
I think that they intended it to be a \circ.  It is
raised above where \circ belongs, and is not obviously
a circle.  This may be a resolution issue.


(file #28916)

Michael Godfrey <godfrey>
Group Member
Fri 23 Aug 2013 06:16:42 PM UTC, comment #5: 

Awesome.  Thanks for the missing symbols.  I did a visual re-check and now the only difference I see is for '\circ'.  In Octave this produces a circle, in Matlab's documentation it produces what looks like a degree symbol (small circle slightly above centerline of glyph).

Michael Godfrey, could you verify what glyph Matlab actually produces for "text (0.5, 0.5, '\circ')"?  I don't always trust their documentation.


Rik <rik5>
Group administrator
Fri 23 Aug 2013 03:48:14 PM UTC, comment #4: 
Michael Goffioul <goffioul>
Wed 21 Aug 2013 07:53:46 PM UTC, comment #3: 

I suspected that it was from Matlab, and not surprised
that they choose a kind of random subset.

Right, doing the full TeX set would be good.
In the past there were some font issues with respect to
the full TeX set, but this should be better now.  Still,
there will likely be font differences between platforms...


Michael Godfrey <godfrey>
Group Member
Wed 21 Aug 2013 07:30:25 PM UTC, comment #2: 

My list of "missing" symbols is based on the list that Matlab documents, rather than what Octave or TeX documents.  In keeping with Octave being a superset of Matlab we should definitely implement whatever additional symbols TeX can do, but we also should at least try to match Matlab.

Rik <rik5>
Group administrator
Wed 21 Aug 2013 06:39:03 PM UTC, comment #1: 

Of the five symbols below four are shown in the TeX
Mode table in Section 15.2.7 in the Manual, but \ast
is not.  So, when implementing it the Table should
also be updated.

Rik,
I am not sure what list you worked from to decide that
these 5 symbols are missing.  And, I cannot remember
what decided on the entries in the Table in the Manual
(even though I updated the Table -- long ago).

Would it possibly make sense to implement all the symbols
that are shown in Appendix F of the TeX Manual?

I also noticed that while \pm is implemented \mp
is not,  Seems odd.

Michael Godfrey <godfrey>
Group Member
Tue 20 Aug 2013 10:24:33 PM UTC, original submission:  

The complete list of symbols that the Matlab TeX interpreter accepts is located here (http://www.mathworks.com/help/matlab/ref/text_props.html) where you have to click on the down arrow next to "TeX Character Sequence Table".

I ran the list through the new Octave TeX interpreter and found the following missing symbols.


warning: ignoring unknown symbol: sim
warning: ignoring unknown symbol: angle
warning: ignoring unknown symbol: ast
warning: ignoring unknown symbol: Leftarrow
warning: ignoring unknown symbol: Rightarrow


These two should be easy, just substitute the actual character for the symbol name.


\sim = '~'
\ast = '*'


The others might be harder.

For the empty set symbol ('\0') I get a Tex Parse Error.


text (0.5, 0.5, '\0')
TeX parse error: syntax error


Probably this has to do with the fact that the null character \0 is also an end of string marker for C.

Finally, I get a render error with the right angle symbol.  I tried a few fonts, but I still always get it.


text (0.5,1, '\rangle', 'fontname', 'arial')
warning: ft_render: skipping missing glyph for character '232a'



Rik <rik5>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #29278:  plot_upd4.diff added by godfrey (761B - text/x-patch)
file #29271:  tex_table_update3.diff added by godfrey (6KiB - text/x-patch)
file #29205:  tex_table_update2.diff added by godfrey (27KiB - text/x-patch)
file #29193:  tex_table_update1.diff added by godfrey (22KiB - text/x-patch)
file #29181:  test_tex_new.m added by godfrey (1KiB - text/x-objcsrc)
file #29179:  tex_symbols.diff added by godfrey (5KiB - text/x-patch)
file #29180:  oct-tex-symbols.in added by godfrey (4KiB - application/octet-stream)
file #28957:  circ_bullet.diff added by godfrey (969B - text/x-patch)
file #28934:  circ.jpg added by godfrey (20KiB - image/jpeg)
file #28916:  circ.pdf added by godfrey (3KiB - application/pdf)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by godfrey (Posted a comment)
  • -email is unavailable- added by rik5 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2014-12-05 rik5 StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2014-12-02 rik5 StatusFixed In Progress
        Open/ClosedClosed Open
    2013-10-01 godfrey Attached File- Added plot_upd4.diff, #29278
    2013-09-30 godfrey Attached File- Added tex_table_update3.diff, #29271
    2013-09-23 godfrey Attached File- Added tex_table_update2.diff, #29205
    2013-09-22 godfrey Attached File- Added tex_table_update1.diff, #29193
    2013-09-20 godfrey Attached File- Added test_tex_new.m, #29181
    2013-09-20 godfrey Attached File- Added tex_symbols.diff, #29179
        Attached File- Added oct-tex-symbols.in, #29180
    2013-08-29 godfrey Attached File- Added circ_bullet.diff, #28957
    2013-08-25 godfrey Attached File- Added circ.jpg, #28934
    2013-08-24 goffioul StatusNone Fixed
    2013-08-23 godfrey Attached File- Added circ.pdf, #28916
    2013-08-23 goffioul Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code