bugGNU roff - Bugs: bug #62860, [gropdf] trying to access URW...

 
 

bug #62860: [gropdf] trying to access URW fonts from wrong directory

Submitter:  G. Branden Robinson <gbranden>
Submitted:  Thu 04 Aug 2022 12:23:37 AM UTC
   
 
Category:  Driver gropdf Severity:  4 - Important
Item Group:  Build/Installation Status:  Invalid
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 23 Oct 2022 08:04:03 PM UTC, comment #22: 


comment #21:

> As mentioned in comment #17, the `/usr/local/share/groff/site-font` directory doesn't exist.
>
> However, I only just noticed that `/usr/local/share/groff` contained other directories, which are vestiges of the last successful `make && make install` attempt. I nuked the entire `/usr/local/share/groff` directory, and reattempted a fresh build of Groff. And—wouldn't you know it—it now builds fine (tests still fail, though…). Even with the bumped GhostScript version (10.00.0_1), it built fine.
>
> I've updated the Gist containing my build run:
> https://gist.github.com/Alhadis/67fccae66e5e636e71fe68ee6991a2d6/2fd04b1d102abe8de3b61b12f0e2adf864f1ce06
>
> I feel like shit, because god only knows how much of your time I've wasted because I never thought to nuke vestiges of a partially overwritten installation… :(


Don't worry about it.  There are some usability challenges with foundries and download files and what not, and there will be even if we get our magic font installer tool written and shipped with the distribution, because simply accessing and embedding real font files opens a new axis of things that can go wrong.

I note two things:

1. The "make uninstall" target should help with this sort of issue; it is touted in the INSTALL.extra file.


Uninstalling
============

If you are dissatisfied with groff, or to prepare for a new installation
from source, you can uninstall it to ensure that no stale files persist
on the system.  Run the command 'make uninstall'.  (If you used 'sudo
make install', run 'sudo make uninstall'.)  At a minimum, some
directories not particular to groff, like 'bin' and (depending on
configuration) an X11 'app-defaults' directory will remain, as will one
plain file called 'dir', created by GNU Texinfo's 'install-info'
command.  (As of this writing, 'install-info' offers no provision for
removing an effectively empty 'dir' file, and groff does not attempt to
parse this file to determine whether it can be safely removed.)  All
other groff artifacts will be deleted from the installation hierarchy.


2.  At the same time, people can create "download" files of their own, and "make uninstall" won't help to remove things that weren't emplaced by the "install" target in the first place.

It sounds like this ticket can be resolved since your problem went away, but bug #62950 remains open.  What you tripped over, others will.  Maybe we can cushion the fall.

G. Branden Robinson <gbranden>
Group administrator
Sat 15 Oct 2022 12:44:29 PM UTC, comment #21: 

comment #20:

> The file I am interested in seeing is /usr/local/share/groff/site-font/devpdf/download.


As mentioned in comment #17, the `/usr/local/share/groff/site-font` directory doesn't exist.

However, I only just noticed that `/usr/local/share/groff` contained other directories, which are vestiges of the last successful `make && make install` attempt. I nuked the entire `/usr/local/share/groff` directory, and reattempted a fresh build of Groff. And—wouldn't you know it—it now builds fine (tests still fail, though…). Even with the bumped GhostScript version (10.00.0_1), it built fine.

I've updated the Gist containing my build run:
https://gist.github.com/Alhadis/67fccae66e5e636e71fe68ee6991a2d6/2fd04b1d102abe8de3b61b12f0e2adf864f1ce06

I feel like shit, because god only knows how much of your time I've wasted because I never thought to nuke vestiges of a partially overwritten installation… :(

John Gardner <alhadis>
Sat 15 Oct 2022 11:33:08 AM UTC, comment #20: 

Branden did ask for ./font/devpdf/download I did not realise you gave him this file instead. The dot was meant to refer to the directory in which you did the "make" rather than /usr/share/local/groff/site_font/devpdf. Also I missed out one directory level in my request (sorry). The file I am interested in seeing is /usr/local/share/groff/site-font/devpdf/download.


Deri James <deri>
Group Member
Sat 15 Oct 2022 03:38:25 AM UTC, comment #19: 

comment #18:

> Please could you attach the file /usr/local/share/groff/site-fonts/download.


I already did; see file #53823

John Gardner <alhadis>
Fri 14 Oct 2022 11:33:05 PM UTC, comment #18: 

Please could you attach the file /usr/local/share/groff/site-fonts/download.

Deri James <deri>
Group Member
Fri 14 Oct 2022 12:19:31 PM UTC, comment #17: 


> Please check your copy of download in the site-font directory to see if it contains the stale paths.


All of the paths referenced by `font/devpdf/download` point to font files.

Also, concerning environment variables: the directory referenced by GROFF_FONT_PATH doesn't exist (it used to, but I deleted it). For the record, here are the other variables I have set in my environment that might be relevant:


GROFF_FONT_PATH=/usr/local/share/groff/site-font
GROFF_TMAC_PATH=/Users/Alhadis/.files/var/tmac:/Users/Alhadis/Labs/Mono:/usr/local/share/groff/site-tmac
GS_OPTIONS=-dNOSAFER -dNOSHORTERRORS -dOSTACKPRINT -dESTACKPRINT -I.
TMAC=/usr/local/share/groff/current/tmac
TROFFONTS=/Users/Alhadis/Fonts:/Users/Alhadis/Library/Fonts:/usr/local/share/groff/site-font


(GS_OPTIONS is to facilitate general-purpose PostScript programming, and has nothing to do with Groff)

John Gardner <alhadis>
Wed 12 Oct 2022 08:09:11 PM UTC, comment #16: 


comment #15:

> comment #14:
> > What we know:-
> >
> > The filepaths in the built download file are correct.
> > So gropdf must be reading a different download.
> > The locations to search for the download file are hard-coded in line 59.
> > After the path in GROFF_FONTPATH
>
> I am uncertain of the wisdom of using GROFF_FONT_PATH, whose semantics are a search path for groff device and font description files, also as a search path for Type 1 or other font files.


Er, excuse me--that's not what's happening here.  GROFF_FONT_PATH is being used to locate the "download" file, which is a groffy thing, so no abuse is taking place.

And the semantics of practically every Unix environment variable with a name ending in _PATH are that the first match wins.

That's good.  If we can just get our resolution order straight, then, and maybe talk Peter into unwinding that copy-system-to-site change, then we'll have more robust operation.

G. Branden Robinson <gbranden>
Group administrator
Wed 12 Oct 2022 08:00:14 PM UTC, comment #15: 

comment #14:

> What we know:-
>
> The filepaths in the built download file are correct.
> So gropdf must be reading a different download.
> The locations to search for the download file are hard-coded in line 59.
> After the path in GROFF_FONTPATH


I am uncertain of the wisdom of using GROFF_FONT_PATH, whose semantics are a search path for groff device and font description files, also as a search path for Type 1 or other font files.

But I'm not in a hurry to burn it with my puritanical fire quite yet...see below.

> it looks in site-font.
> Currently gropdf uses the last entry when a duplicate discovered.


Right, I recall this discussion from August now.

> A recent change to install-fonts.sh now copies entries from the system download to the site download.


Hmmmm.

> This is a mistake since the site font directory is intended for added fonts.


Yes, that is indeed the convention and historical practice.
 

> I did suggest changing gropdf behaviour to use the first found entry but this simple change
> got caught up in discussions about what grops should do, so I did nothing.


Oops.

> So my diagnosis is that the download file in site-font contains the stale fontpaths (which were correct
> at the time install-fonts.sh was run, but are now stale). This diagnosis can be confirmed by looking
> in the site-font download file to see if it contains the stale entries.
>
> Changing the behaviour of gropdf as described above would solve the build issue,


Can you proceed with this?

> but once the built groff was installed in the system directory, and GROFF_FONTPATH is unset,
> the stale entries in the site-font directory would be used first.


I reckon it's not the worst thing in the world if people can use GROFF_FONT_PATH as a crutch to tide them over while we sort out grops/gropdf/install-font.sh font file resolution ordering.

I do point out that the groff project does not have control of install-font.sh--that's Peter Schaffter's baby, though I gather he'd prefer someone else adopted it.
 

> The change in install-fonts.sh was to paper over a deficiency in grops which stops looking when it has
> found the first download file, but logically the system fonts should only appear there if you are
> purposely over-riding a system font.


Let's get gropdf fixed and see if I have sufficient wits about me to ape the fix in grops.

> Please check your copy of download in the site-font directory to see if it contains the stale paths.


John, can you follow up to Deri's query, please?

Deri, it sounds like bug #62950, which I filed, is yet another duplicate of this.  Do you agree?

G. Branden Robinson <gbranden>
Group administrator
Mon 10 Oct 2022 05:49:31 PM UTC, comment #14: 

What we know:-

The filepaths in the built download file are correct.
So gropdf must be reading a different download.
The locations to search for the download file are hard-coded in line 59.
After the path in GROFF_FONTPATH it looks in site-font.
Currently gropdf uses the last entry when a duplicate discovered.
A recent change to install-fonts.sh now copies entries from the system download to the site download.

This is a mistake since the site font directory is intended for added fonts.

I did suggest changing gropdf behaviour to use the first found entry but this simple change
got caught up in discussions about what grops should do, so I did nothing.

So my diagnosis is that the download file in site-font contains the stale fontpaths (which were correct
at the time install-fonts.sh was run, but are now stale). This diagnosis can be confirmed by looking
in the site-font download file to see if it contains the stale entries.

Changing the behaviour of gropdf as described above would solve the build issue, but once the built groff
was installed in the system directory, and GROFF_FONTPATH is unset, the stale entries in the site-font
directory would be used first.

The change in install-fonts.sh was to paper over a deficiency in grops which stops looking when it has
found the first download file, but logically the system fonts should only appear there if you are
purposely over-riding a system font.

Please check your copy of download in the site-font directory to see if it contains the stale paths.

Deri James <deri>
Group Member
Mon 10 Oct 2022 02:58:03 PM UTC, comment #13: 

Thanks, John.  I've added Deri to the CC list because we have once again reached the limits of my knowledge and/or confident judgement.

comment #12:

> Yes, the build now ends with:
>


> gropdf:contrib/mom/examples/typesetting.mom: fatal error: unable to open font '/usr/local/Cellar/ghostscript/9.56.1/share/ghostscript/9.56.1/Resource/Font/C059-Roman' for embedding
>   GROFF    doc/meref.me
>   GROFF    doc/meref.ps
>   GROFF    doc/meintro.me
>   GROFF    doc/meintro.ps
>   GROFF    doc/meintro_fr.me
>   GROFF    doc/meintro_fr.ps
>   GROFF    doc/ms.ps
>   GROFF    doc/pic.ps
>   GROFF    doc/groff-man-pages.pdf
> gropdf:src/utils/addftinfo/addftinfo.1: fatal error: unable to open font '/usr/local/Cellar/ghostscript/9.56.1/share/ghostscript/9.56.1/Resource/Font/NimbusRoman-Italic' for embedding
> make[1]: *** [doc/groff-man-pages.pdf] Error 1
> make[1]: *** Deleting file `doc/groff-man-pages.pdf'
> make: *** [all] Error 2


Okay, this is really interesting.  Is this still from the build where you said ./configure --with-urw-fonts-dir=/usr/local/Cellar/ghostscript/10.0.0_1/share/ghostscript/fonts ?

The attached download file suggests that it is.

And yet, stubbornly, gropdf is demanding the file from /usr/local/Cellar/ghostscript/9.56.1/share/ghostscript/9.56.1/Resource/Font/.

Deri, do you have any idea what's going on here?  If I read the LoadDownload subroutine correctly, it is called once, unconditionally, and as long as it finds the download file, it robotically sets up a hash mapping (foundry, typeface) 2-tuples to file names read from the download file.  The download file has only one filespec per font so it seems like it should be impossible for this to go wrong.

https://git.savannah.gnu.org/cgit/groff.git/tree/src/devices/gropdf/gropdf.pl?id=dbb3a589698484c6b367df516167555ce51f053b#n614

What am I missing?

G. Branden Robinson <gbranden>
Group administrator
Mon 10 Oct 2022 02:41:01 PM UTC, comment #12: 

Yes, the build now ends with:


gropdf:contrib/mom/examples/typesetting.mom: fatal error: unable to open font '/usr/local/Cellar/ghostscript/9.56.1/share/ghostscript/9.56.1/Resource/Font/C059-Roman' for embedding
  GROFF    doc/meref.me
  GROFF    doc/meref.ps
  GROFF    doc/meintro.me
  GROFF    doc/meintro.ps
  GROFF    doc/meintro_fr.me
  GROFF    doc/meintro_fr.ps
  GROFF    doc/ms.ps
  GROFF    doc/pic.ps
  GROFF    doc/groff-man-pages.pdf
gropdf:src/utils/addftinfo/addftinfo.1: fatal error: unable to open font '/usr/local/Cellar/ghostscript/9.56.1/share/ghostscript/9.56.1/Resource/Font/NimbusRoman-Italic' for embedding
make[1]: *** [doc/groff-man-pages.pdf] Error 1
make[1]: *** Deleting file `doc/groff-man-pages.pdf'
make: *** [all] Error 2


John Gardner <alhadis>
Mon 10 Oct 2022 02:16:13 PM UTC, comment #11: 

Hi John,

Thanks for the follow-up.

Can you confirm that the error message changed after the patch?

G. Branden Robinson <gbranden>
Group administrator
Mon 10 Oct 2022 01:50:32 PM UTC, comment #10: 

Done. Attached are the files you requested (I forgot about the file-attachment feature…)

(file #53823, file #53824, file #53825, file #53826)

John Gardner <alhadis>
Mon 10 Oct 2022 12:14:09 PM UTC, comment #9: 

Hi John,

It appears I updated the text of the wrong diagnostic.  But, there was only one other with the exact same text (this is one reason I exhort against writing the exact same text for multiple diagnostics).

If you apply the following one-line patch to the gropdf.pl in your source tree and re-run make, I'll bet you see the error messages change.


diff --git a/src/devices/gropdf/gropdf.pl b/src/devices/gropdf/gropdf.pl
index 50d07ca27..37f6e5644 100644
--- a/src/devices/gropdf/gropdf.pl
+++ b/src/devices/gropdf/gropdf.pl
@@ -2571,7 +2571,7 @@ sub GetType1
     my $f;

     OpenFile(\$f,$fontdir,"$file");
-    Die("failed to open '$file'") if !defined($f);
+    Die("unable to open font '$file' for embedding") if !defined($f);

     $head=GetChunk($f,1,"currentfile eexec");
     $body=GetChunk($f,2,"00000000") if !eof($f);


Thanks for the comprehensive build logs.  I now think the problem is that your download file for the gropdf driver is getting generated incorrectly.

Can you attach the following files from your build tree to this ticket?


./font/devpdf/Foundry
./font/devpdf/download


G. Branden Robinson <gbranden>
Group administrator
Mon 10 Oct 2022 06:57:52 AM UTC, comment #8: 

Neither the presence nor absence of `--with-urw-fonts-dir` had any bearing on the build. I've nonetheless uploaded logs from both build attempts:



Homebrew's updated its Ghostscript formula since I filed this issue, so its version string no longer includes a version bump (i.e., "10.00.0_1" bumped from "10.00.0"). I had to tamper with the Homebrew's package description to force it to install to a path containing the problematic version-string. Logically, this has no bearing on what version GhostScript thinks it is: it's purely an administrative detail on Homebrew's part.


$ gs --version
10.00.0
$ realpath `which gs`
/usr/local/Cellar/ghostscript/10.0.0_1/bin/gs


Also, I'm not sure if this is relevant, but I noticed Homebrew is installing GhostScript fonts from an external source; specifically, one with a hardcoded URL:


  # https://sourceforge.net/projects/gs-fonts/
  resource "fonts" do
    url "https://downloads.sourceforge.net/project/gs-fonts/gs-fonts/8.11%20%28base%2035%2C%20GPL%29/ghostscript-fonts-std-8.11.tar.gz"
    sha256 "0eb6f356119f2e49b2563210852e17f57f9dcc5755f350a69a46a0d641a0c401"
  end


John Gardner <alhadis>
Mon 10 Oct 2022 03:59:55 AM UTC, comment #7: 

John, can you let me know if the following changes resolve your URW font location problem?

I would count it as "resolved" if you need to pass a --with-urw-fonts-dir= flag to the configure script to clue in the build...as long as gropdf works afterward.  It would require a lot of machinery for groff to enumerate multiple installations of URW fonts and select a "best" set itself.


commit 57cf998eae20fe7c886ec2fbda24248dcafa8209
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sun Oct 9 21:27:01 2022 -0500

    [m4]: Try to make URW font search deterministic.

    * m4/groff.m4: Try to make gropdf's search for URW font files more
      deterministic.  To disambiguate cases where multiple--not necessarily
      identical--copies may be found on the system, search _only_ in the
      directory given to the `--with-urw-fonts-dir` configure option, if
      present.  This populates the "Foundry" file generated by the build,
      causing gropdf to search there first.

      (GROFF_URW_FONTS_PATH): Stop `AC_SUBST`ing `urwfontsdir` here.

      (GROFF_URW_FONTS_CHECK): `AC_REQUIRE` `GROFF_URW_FONTS_CHECK` to be
      run first.  If `urwfontsdir` is not null, search only there for URW
      fonts.  Assign `urwfontsdir` the value of the directory in which the
      fonts are found.  Assign it an empty value if none are found.  Either
      way, `AC_SUBST` the variable.

    Attempts to fix <https://savannah.gnu.org/bugs/?62860> (1/2).

commit 0b93f550d8d686326cf6701953bb0f5f590beb1a
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sun Oct 9 21:29:09 2022 -0500

    configure.ac: Report where URW fonts were located.

    * configure.ac: If URW fonts were located, add the directory where they
      were found to the configuration report.

    Attempts to fix <https://savannah.gnu.org/bugs/?62860> (2/2).


G. Branden Robinson <gbranden>
Group administrator
Fri 07 Oct 2022 08:44:37 AM UTC, comment #6: 

See bug #62918 for a report of the same problem with full build logs.

G. Branden Robinson <gbranden>
Group administrator
Fri 07 Oct 2022 07:45:39 AM UTC, comment #5: 

Here's the originally reported problem.


gropdf:contrib/mom/examples/typesetting.mom: fatal error:
  failed to open '/usr/share/ghostscript/9.52/Resource/Font/C059-Roman'
gropdf:src/utils/addftinfo/addftinfo.1: fatal error:
  failed to open '/usr/share/ghostscript/9.52/Resource/Font/NimbusRoman-Italic'


G. Branden Robinson <gbranden>
Group administrator
Fri 07 Oct 2022 05:48:18 AM UTC, comment #4: 

I also attempted a build after doing a normal out-of-tree build and then, for every executable in the build tree, making a file in /usr/local/bin of the same name that was a copy of /bin/false, then cleaning and rebuilding, which should have caused the build to fail if recourse was ever made to a "partial" groff installation as Werner described in https://lists.gnu.org/archive/html/bug-groff/2022-07/msg00115.html .

But doing that didn't break my build either.

G. Branden Robinson <gbranden>
Group administrator
Fri 07 Oct 2022 05:37:49 AM UTC, comment #3: 

An in-tree build was clean in this respect for me, too, which means I can't reproduce the problem.

G. Branden Robinson <gbranden>
Group administrator
Fri 07 Oct 2022 05:05:06 AM UTC, comment #2: 

An out of tree build seems clean.


    ./bootstrap && \
    mkdir build && \
    cd build && \
    ../configure &&
    strace -f make > ../strace.log.$$ 2>&1 && \
    make check && \
    make distcheck
$ grep ' execve("/bin' strace.log.5071 | sed 's/,.*//' | grep -Ev '/bin/(bash|sh)' | grep . || echo EMPTY
EMPTY
$ grep ' execve("/usr/bin' strace.log.5071 | sed 's/,.*//'| grep -Ev '/(ar|as|awk|basename|byacc|cat|chmod|cmp|cut|diff|dirname|cp|egrep|etex|expr|g++|gcc|grep|gs|ld|ls|mkdir|mktemp|mv|pdfetex|perl|rm|m4|make|makeinfo|pnmdepth|pnmtops|ranlib|sed|sh|sort|tar|texi2dvi|texindex|touch|tr|uname|uniq|xpmtoppm)' | grep . || echo EMPTY
EMPTY


G. Branden Robinson <gbranden>
Group administrator
Wed 24 Aug 2022 10:05:13 AM UTC, comment #1: 

Kicking up to blocker severity because this needs to be done before 1.23.0 finalizes.

G. Branden Robinson <gbranden>
Group administrator
Thu 04 Aug 2022 12:23:37 AM UTC, original submission:  

Issue raised by Werner on bug-groff list.

https://lists.gnu.org/archive/html/bug-groff/2022-07/msg00115.html

See follow-up thread on main groff list.

https://lists.gnu.org/archive/html/groff/2022-07/msg00221.html

...and replies.

G. Branden Robinson <gbranden>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53823:  download added by alhadis (7KiB - application/octet-stream)
file #53824:  config.log added by alhadis (249KiB - application/octet-stream)
file #53825:  Foundry added by alhadis (8KiB - application/octet-stream)
file #53826:  config.status added by alhadis (68KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by deri (Posted a comment)
  • -email is unavailable- added by gbranden (gropdf expert)
  • -email is unavailable- added by alhadis (Posted a comment)
  • -email is unavailable- added by gbranden (filed similar report)
  • -email is unavailable- added by gbranden (original reporter)
  • -email is unavailable- added by gbranden (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 20 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-12-19 gbranden Planned Release1.23.0 None
    2022-10-23 gbranden StatusNeed Info Invalid
        Open/ClosedOpen Closed
    2022-10-10 gbranden Carbon-Copy- Added -email is unavailable-
    2022-10-10 alhadis Attached File- Added download, #53823
        Attached File- Added config.log, #53824
        Attached File- Added Foundry, #53825
        Attached File- Added config.status, #53826
    2022-10-10 gbranden StatusIn Progress Need Info
    2022-10-10 gbranden Carbon-Copy- Added -email is unavailable-
    2022-10-10 gbranden StatusNeed Info In Progress
    2022-10-07 gbranden CategoryGeneral Driver gropdf
        Summary[build] audit for inappropriate use of system groff resources [gropdf] trying to access URW fonts from wrong directory
    2022-10-07 gbranden StatusIn Progress Need Info
    2022-10-07 gbranden StatusNone In Progress
        Assigned toNone gbranden
    2022-09-26 gbranden Severity5 - Blocker 4 - Important
        Planned ReleaseNone 1.23.0
    2022-08-24 gbranden Severity3 - Normal 5 - Blocker
    2022-08-04 gbranden Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code