bugGNU roff - Bugs: bug #50989, incorrect generation of...

 
 

bug #50989: incorrect generation of typesetting.pdf

Submitter:  Bertrand Garrigues <bgarrigues>
Submitted:  Tue 09 May 2017 09:26:59 PM UTC
   
 
Category:  Driver gropdf Severity:  5 - Blocker
Item Group:  Incorrect behaviour Status:  Fixed
Privacy:  Public Assigned to:  bgarrigues
Open/Closed:  Closed Planned Release:  1.22.4
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 21 Aug 2017 03:02:51 PM UTC, comment #15: 

I have committed the patch. I have not committed my CheckType1.pl script since it is unlikely to be of use to anyone, apart from testing the new font loading code. If gropdf fails to load a particular font the utility is not going to help find out why.

To check that all the postscript fonts are "working" I usually do:-

groff -Tpdf -mandoc src/devices/gropdf/gropdf.1.man |okular -

And the URW fonts:-

groff -Tpdf -mandoc src/devices/gropdf/gropdf.1.man -P-yU |okular -

All the fonts are shown from page 2. So I'm not sure if we need a separate check script.

Deri James <deri>
Group Member
Fri 18 Aug 2017 04:07:58 PM UTC, comment #14: 

Deri,

I've made a second version of your patch `bug-50989-v2.patch' as the one you posted can no longer be applied on master.  All the tests are OK on my side.  File `typesetting.pdf' is now correctly generated.  I've also used two files (see attachments) `basic_fonts.tr' and `urw_fonts.tr' that use all available fonts; without your patch lots of the fonts of basic_fonts.tr cannot be used, and the problem is well fixed with your patch. I've tested with ghostscript 9.21 and also made non-regression tests with the old fonts installed by older version of ghostscript.

So you can commit on master the patch, unless you saw some other problems.

I think you should also commit CheckType1.pl as it's a usefull tool.  I would also like to commit `basic_fonts.tr' and `urw_fonts.tr' (in font/devpdf) if you agree.  It would be better to have a way to automatically check the output of these files and add the test into `make check' but I know no easy way to check a pdf.


(file #41568, file #41569, file #41570)

Bertrand Garrigues <bgarrigues>
Group administrator
Wed 02 Aug 2017 08:38:32 PM UTC, comment #13: 

I have rewritten the font loading code, it seems to handle both old and new ghostscript supplied fonts. However, before I commit the change I have supplied a patch for testing, since it is significantly different from the old version! I have also written a little script which can be used to test the algorithm I am using without installing all your fonts into groff!

This won't be added to the groff git, it is just a tool to give some confidence that the algorithm will work if the font was installed. It is called CheckType1.pl and is used like this:-


CheckType1.pl /usr/share/ghostscript/9.20/Resource/Font/*|less


or


CheckType1.pl `locate \.pfa`|less


This will produce a report on each file similar to this:-


/usr/share/ghostscript/9.20/Resource/Font/C059-BdIta
Segment 1 (ascii) =     898
Segment 2 (ascii) =     151141
Segment 3 (ascii) =     532
File=152571, Font=152571

/usr/share/ghostscript/9.20/Resource/Font/C059-Bold
Segment 1 (ascii) =     887
Segment 2 (ascii) =     144662
Segment 3 (ascii) =     532
File=146081, Font=146081

/usr/share/ghostscript/9.20/Resource/Font/C059-Italic
Segment 1 (ascii) =     896
Segment 2 (ascii) =     147446
Segment 3 (ascii) =     532
File=148874, Font=148874

/usr/share/ghostscript/9.20/Resource/Font/C059-Roman
Segment 1 (binary) =    3591
Segment 2 (binary) =    135386
Segment 3 (binary) =    533
File=139530, Font=139528

/usr/share/ghostscript/9.20/Resource/Font/D050000L
Segment 1 (ascii) =     4391
Segment 2 (ascii) =     38420
Segment 3 (ascii) =     532
File=43343, Font=43343

/usr/share/ghostscript/9.20/Resource/Font/NimbusMonoPS-Bold
Segment 1 (ascii) =     898
Segment 2 (ascii) =     154493
Segment 3 (ascii) =     532
File=155923, Font=155923


A type 1 font has three segments which I have to identify as separate length for the font to work in a pdf. So the report specifies the size and type of each segment found. The type can be:-

ascii: for segment 2 this is actually raw bytes. The "File" and "Font" totals should be the same, all the font has been recognised.

ascii hex : this is format where segment 2 rather than being raw bytes ls coded as ascii text and I have to convert to raw bytes before embedding. In this case the two sizes will be different since the size of segment 2 is about half what it was because every two hex characters result in 1 raw byte. It is not exactly half because there are also linefeeds in the ascii hex which get dropped.

binary : this a form where each segment is introduced by a six byte header which specifies the length of the segment. The old ghostscript fonts were of these type (normally called .pfb), now they are ascii, .pfa type. The difference between the two sizes should always be two. The reason is because these binary versions have a terminating two byte header which must be dropped. Also the total of the individual segments will not equal the font total because the font total takes into account the 3 six byte headers which I remove.

Note that you may find some fonts which just have segment 1, these are not full type 1 fonts, even if they are called ".pfa" they are likely to be postscript programs which generate a font from another font, an example is the font ZDR (Dingbats Reversed) which is available with grops. This is not available for gropdf, but any glyph can be reversed by using X’pdf:xrev’ extension. The left hand dingbats glyph is pre-defined as:-


.char \[lh] \X'pdf: xrev'\[rh]\X'pdf: xrev'


Any testing you can do will be appreciated. I am not able to test on a windows groff version, but I believe setting the filehandle to binary mode previously solved a problem there, so this should be Ok.


(file #41395, file #41396)

Deri James <deri>
Group Member
Tue 27 Jun 2017 10:55:09 PM UTC, comment #12: 

I will be looking at this soon, I hope. Things seem to pile up!

Deri James <deri>
Group Member
Tue 27 Jun 2017 10:20:35 PM UTC, comment #11: 

Hi Deri,

I don't know if you found some time to rewrite the font loading in gropdf.  I've worked a bit on another issue on this file generation: as said in my comment #1, using LC_ALL=C also causes some problems (preconv falls back to "latin1" encoding).  So I'm assigning you this bug and I'll open another bug for the other problem.

Bertrand Garrigues <bgarrigues>
Group administrator
Sun 14 May 2017 04:05:41 PM UTC, comment #10: 

The specification of the PFB format is in Adobe Technical Note #5040:

http://www.adobe.com/content/dam/Adobe/en/devnet/font/pdfs/5040.Download_Fonts.pdf

Note that there could theoretically be more than three chunks in a PFB (which I've never seen).  Note also that what we call `PFA' today actually is not what was originally defined in #5040 (and which I've never seen either).  Instead, PFAs today are simply Type 1 PS resources with an ASCII eexec part.

And yes, supporting Type 1 PS resources with a binary eexec part (which is what gs now distributes) would be good.

Werner LEMBERG <wl>
Group administrator
Sun 14 May 2017 12:16:51 PM UTC, comment #9: 

I am intending to rewrite the font loading code in gropdf to handle the situation where the 3 "chunks" of a type 1 font are treated separately for binary/ascii loading. It would help if I could find where the actual official file format is defined, I have the type 1 spec pdf from Adobe but I don't think this specifies the actual file format, just the font format.

I am temporarily very busy, not just work but I have to get a planning objection in the next couple of days, they want to build 215 new houses in a field 50 metres from my house!

Deri James <deri>
Group Member
Sun 14 May 2017 11:54:17 AM UTC, comment #8: 

On my system all the fonts in `/usr/share/ghostscript/9.21/Resource/Font' have the same problem (there are not .pfb, 6-bytes header missing).  Should we try to contact people from ghostscript to ask them why they won't install these fonts in .pfb format anymore, or implement what Deri suggests to support Type 1 font in non-pfb format? The problem is that there are already 2 official versions of ghostscript (9.20, 9.21) that install these fonts.

Bertrand Garrigues <bgarrigues>
Group administrator
Thu 11 May 2017 10:38:11 PM UTC, comment #7: 

Thanks.  `C059-Roman' is not a PFB.  It is a Type 1 font, though, with a binary eexec part (which is valid according to the specification).


Werner LEMBERG <wl>
Group administrator
Thu 11 May 2017 10:16:25 PM UTC, comment #6: 

Here's an example of the new font (C059-Roman), and the old version pre gs 9.20 (CenturySchL-Roma).

(file #40669, file #40670)

Deri James <deri>
Group Member
Thu 11 May 2017 09:40:45 PM UTC, comment #5: 

Are you sure it's a problem if gropdf?  What about the possibility that the PFB files are invalid? As far as I know, a PFB must start with binary segment header of 6 bytes.  Can you contact the gs people for a definite answer?

Werner LEMBERG <wl>
Group administrator
Thu 11 May 2017 09:27:58 PM UTC, comment #4: 

It looks like only the first block header is missing, there is one immediately after the "eexec" which gives the length of the main block which is then followed by another block which is the trailing "0"s. So it looks likely I can do a sort of hybrid type which parses the first block as though it is a .pfa and then check for a six byte chunk header (starts with \x80 which is not a valid ascii hex character), and if found switch to .pfb type processing.

Deri James <deri>
Group Member
Thu 11 May 2017 09:07:21 PM UTC, comment #3: 

This is a strange one. Type 1 fonts come in two flavours which handle the binary part of the font differently. .pfa uses hex encoded ascii, and .pfb has raw binary. The way to differentiate is that a .pfb file is meant to have six bytes of binary data on the front of the file which define the length of the first block. The new CenturySchoolbook fonts from ghostscript are raw binary but do not have this binary header.

So the problem is in gropdf, I'm working on a solution. :-)

Cheers

Deri

Deri James <deri>
Group Member
Wed 10 May 2017 11:04:30 PM UTC, comment #2: 

Font family N are also not properly rendered (nothing is displayed), maybe 2b5a438e0c205a5f118b456db7438e35dab50e09 was not sufficient.

Bertrand Garrigues <bgarrigues>
Group administrator
Wed 10 May 2017 10:40:56 PM UTC, comment #1: 

It's the inclusion of LC_ALL=C that causes the problem (contrib/mom/mom.am):

# Rule to generated .pdf files from .mom files
SUFFIXES += .mom .pdf
.mom.pdf:
@$(MKDIR_P) `dirname $@`
@echo LC_ALL=C $(MOMPDFMOM) $< >$@
LC_ALL=C $(MOMPDFMOM) $< >$@

Bertrand Garrigues <bgarrigues>
Group administrator
Tue 09 May 2017 09:26:59 PM UTC, original submission:  

When building groff, typesetting.pdf (from contrib/mom/examples/typesetting.mom) is incorrectly generated (french accent not rendered) in the build tree (and thus the doc installed in /usr/local/share/doc/groff-1.22.3/examples/mom/typesetting.pdf is KO).  However, after installing the newly-compiled groff, calling `pdfmom -k typesetting.mom' does correctly generated the pdf.
Seen on revision 2b5a438e0c205a5f118b456db7438e35dab50e09 with devpdf fonts fully generated.

Bertrand Garrigues <bgarrigues>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #41568:  bug-50989-v2.patch added by bgarrigues (4KiB - text/x-patch)
file #41569:  basic_fonts.tr added by bgarrigues (1KiB - text/troff)
file #41570:  urw_fonts.tr added by bgarrigues (1KiB - text/troff)
file #41395:  bug-50989.patch added by deri (5KiB - text/x-patch)
file #41396:  CheckType1.pl added by deri (2KiB - application/x-perl)
file #40669:  C059-Roman added by deri (142KiB - application/octet-stream - New and old ghostscript CenturySchoolbook)
file #40670:  CenturySchL-Roma added by deri (44KiB - application/octet-stream - New and old ghostscript CenturySchoolbook)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Updated the item)
  • -email is unavailable- added by wl (Posted a comment)
  • -email is unavailable- added by deri (Posted a comment)
  • -email is unavailable- added by bgarrigues (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 17 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-05-02 gbranden Summaryuncorrect generation of typesetting.pdf incorrect generation of typesetting.pdf
    2022-05-02 gbranden CategoryNone Driver gropdf
        Planned ReleaseNone 1.22.4
        SummaryIncorrect generation of typesetting.pdf uncorrect generation of typesetting.pdf
    2017-08-21 bgarrigues StatusReady for Merge Fixed
        Open/ClosedOpen Closed
    2017-08-18 bgarrigues Attached File- Added bug-50989-v2.patch, #41568
        Attached File- Added basic_fonts.tr, #41569
        Attached File- Added urw_fonts.tr, #41570
    2017-08-15 bgarrigues StatusIn Progress Ready for Merge
        Assigned toderi bgarrigues
    2017-08-02 deri Attached File- Added bug-50989.patch, #41395
        Attached File- Added CheckType1.pl, #41396
    2017-06-27 bgarrigues Assigned tobgarrigues deri
    2017-05-11 deri Attached File- Added C059-Roman, #40669
        Attached File- Added CenturySchL-Roma, #40670
    2017-05-10 bgarrigues Severity4 - Important 5 - Blocker

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code