Keith,

Sorry, you will have to help me here, if you want me to understand what you meant.

> I am willing to pursue an extended lex/yacc implementation, (subject to Deri actually answering the question I've now asked twice, without a response...

This implies to me that your pursuit of an extended lex/yacc implementation which copes with pdf 1.5 extensions to the standard is subject to me answering your question, i.e. it cannot be done until you receive the answer. This is why it looks like you were claiming I was holding you up.

It is progress that you realised I was pointing you to Brendan's reference in comment #4, not my silly mistake in the previous comment, I hope you have time to read my answer to your cropbox question.

As to whether this ticket refers to PDFPIC or not: I think the answer is in the title of the ticket and also the fact that you textually linked bug #61324 to this ticket in your email:-

https://lists.gnu.org/archive/html/groff/2021-09/msg00057.html

I.E. That this original ticket would solve the problems with using pdfinfo with PDFPIC, the email chain is even referenced in bug #61324.

As to what upsets me is the impression you gave that I had been unhelpful in not answering your question, whereas it was lack of dilligence in not reading an email which actually starts with "Hi Keith". I do try to be a helpful chap because I have had so much help from so many people.

Cheers

Deri

Am I correct in guessing that a bounding box/MediaBox extractor would have a lot of shared logic for PS and PDF?  If so, one preprocessor could perform both tasks.  I guess we could call it "grobb" or something, and assign a -B flag to it in groff(1).  In fact, if we have a preprocessor and claim an option letter, it's probably best if it's as general-purpose as is reasonable.

No, unfortunately, there are significant differences. However, my option (D) does not envisage a .pdfbb, but rather in the manner of tbl which scans the source looking for .TS/.TE and replacing intervening lines with its own code, it would look for .PDFPIC and insert code to achieve what PDFPIC does today. The same is not true for .PSPIC, since we have to retain .psbb since someone may be using it outside of its use in PSPIC.

Branden,

comment #7:

> > but troff then incurs the overhead of setting up an IPC pipeline, to capture the preprocessor output, then fork the preprocessor,
>
> groff already handles all of this.

I don't think that it does...

> Some of the most important bits are:
>
> https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/groff.cpp#n54
> https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/groff.cpp#n576
> https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/pipeline.c
>
> ...or I am badly misunderstanding you.

Consider pre-grohtml; that isn't run within groff's normal pipeline; it is forked, with its own subsidiary pipeline, as and when required.  One of my earliest contributions to groff was to make that subsidiary pipeline setup, and the associated fork, MS-Windows compatible, and I would anticipate a similar overhead, if psbb were to be delegated to a preprocessor.

Hi, Keith,

comment #6:

> Sorry, I don't know; the choice was made, w.r.t. EPS files, before my association with the project began.  I can imagine, however, it may be a matter of convenience, because by having the psbb parsing code accessible as linked in functions, the assignment to troff's internal registers is straightforward,

I suppose, but putting

.nr llx 1234u
.nr lly 5678u
.nr urx 6789u
.nr ury 7890u

onto the output stream isn't too hard, either.  (Though while I'm here I'll note the unnecessary intrusion into the user's name space.  There's no reason these couldn't have been called groff*bbox*llx and similar, for instance.  If the user desires short names for these, the `aln` request is at hand.  Beyond that, isn't PSPIC pretty much the only invoker of the `psbb` request in the first place?)

> whereas, if the parsing is delegated to a preprocessor, not only does that preprocessor have to implement the same parsing logic,

I'm thinking more of "moving" than "reimplmenting" here, though I admit if my other spitball idea of writing the thing in Perl would indeed make it a question of reimplementation.

> but troff then incurs the overhead of setting up an IPC pipeline, to capture the preprocessor output, then fork the preprocessor,

groff already handles all of this.  Some of the most important bits are:

https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/groff.cpp#n54
https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/groff.cpp#n576
https://git.savannah.gnu.org/cgit/groff.git/tree/src/roff/groff/pipeline.c

...or I am badly misunderstanding you.

> and ultimately, reinterpret the preprocessor output to assign the requisite register values.

The contract of writing something called a groff preprocessor is that you will emit output that GNU troff can parse.

Every time I've looked at the psbb code in GNU troff I've become uneasy.  It feels strongly like something that should have a higher barrier of interfacing around it.  It's about 5% of the source lines of the file, which contains the implementations of about 100 requests and the troff program's command-line option parsing and diagnostic handling to boot.  The psbb part of the file defines its own bespoke classes, and and most of the psbb-related functions call only each other or standard C library functions (like sscanf, strlen, and fseek), not other troff functions.

I wonder if anyone else feels a similar unease.

I'll approximate the conclusion of Deri's mail of 13 October.

[Keith Marshall wrote:]

>> some further (non-trivial) development effort will be required, to support concealment of trailer dictionaries and cross reference tables within /XRefStm objects.

[Deri James wrote:]

> There are several options which would address this problem, i.e. non portability of grep and desirability of avoiding groff unsafe mode.

> A) Replace grep with sed/awk (still requires unsafe mode).

> B) Use psbb (requires "non-trivial development").

> C) Use pdfbb (requires hook in input.cpp to call pdfbb and return results).

> D) Convert pdfbb to be a pre-gropdf (i.e. a preprocessor like pre-grohtml) which would look for .PDFPIC and replace with the appropriate calls to \X'pdf: pdfpic’ and add vertical space with .sp.

> (A) is obviously the easiest and quickest, (C) and (D) are not too much work, since the parser required is already in use.

Okay, it's Branden again.  My inclination is (A) to get a short-run fix in place to get the splinter out of users' paws no matter when groff 1.23.0 gets released, and (D) for the longer run.

I would emphasize, lest the point be overlooked, that a preprocessor's interface to the rest of a troff system is a file stream.  Therefore we can write them in Perl, the shell, or yet another language if necessary.  Perl seems the most likely alternative since we already have a Perl v5.6.1 dependency, and none on any other popular scripting language except for the shell, which is a pretty gross language to write a parser in (though I admit I've done it).

Am I correct in guessing that a bounding box/MediaBox extractor would have a lot of shared logic for PS and PDF?  If so, one preprocessor could perform both tasks.  I guess we could call it "grobb" or something, and assign a -B flag to it in groff(1).  In fact, if we have a preprocessor and claim an option letter, it's probably best if it's as general-purpose as is reasonable.

This was my option (D) in this email (for PDFPIC, but also applies to .psbb):-

https://savannah.gnu.org/bugs/?55107

In this mailing-list message[1], Deri offered two PDF files, namely Picture.pdf[2] and croptest.pdf[3], from which the original prototype code[4], as referenced on this ticket, is unable to extract any valid MediaBox specification.

In this follow-up message[5], I explained that the failure to extract the MediaBox from Picture.pdf was caused by an omission from the groff-psbb lexer's pattern matching rules for the PDF dictionary scanning state, resulting in mishandling of nested dictionaries; this is readily resolved by the attached patch[6].

OTOH, croptest.pdf uses new PDF (post PDF-1.5) features, and lacks any trailer dictionary, or free-standing cross reference table, (both of which are required by the current groff-psbb prototype implementation); to support these new PDF features, substantial additions to the current implementation will be required.

[1]: https://lists.nongnu.org/archive/html/groff/2021-09/msg00064.html
[2]: https://lists.nongnu.org/archive/html/groff/2021-09/pdf7tyGN4NLTE.pdf
[3]: https://lists.nongnu.org/archive/html/groff/2021-09/pdfBjudbNbwI2.pdf
[4]: https://osdn.net/users/keith/pf/groff-psbb/scm/tree/e25e11c6770a3d7a2e98cbcfce66dbffd7d8b5a0/
[5]: https://lists.nongnu.org/archive/html/groff/2021-10/msg00043.html
[6]: patch file #52093