bugGNU roff - Bugs: bug #59402, discordance between preconv and...


bug #59402: discordance between preconv and groff

Submitter:  Dave <barx>
Submitted:  Tue 03 Nov 2020 01:18:45 AM UTC
Category:  Preprocessor preconv Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup

Tue 03 Nov 2020 01:18:45 AM UTC, original submission:  

This bug report is the spawn of bug #57618; the relevant discussion begins at comment 8 in that bug report.  This example illustrates the problem:

$ printf "co\xF6perate\n" | uconv -f latin1 -x Any-NFD | groff -Kutf8 > /dev/null
<standard input>:1: warning: can't find special character `u0308'

Unpacking the above: the "printf" hard-codes a Latin-1 character in its otherwise plain ASCII output; the "uconv" converts its Latin-1 input to UTF-8, specifically requesting Normalization Form D; groff's -K option implicitly runs preconv first and tells it to expect UTF-8 input; and finally groff processes preconv's output, whereupon it doesn't understand the non-ASCII part.

(No error is generated if the above is changed to have uconv emit Normalization Form C (NFC) instead of NFD.)

The problem, then, is that preconv emits a Unicode sequence that groff doesn't understand.  Whether that's a problem with preconv, with groff, or with both, is unclear (to me), but if both, then this bug can fork into two bugs to track each problem separately.

Branden's initial analysis was that this is "a bug in (1) preconv for not emitting composite Unicode glyph escapes and (2) troff for failing to accept them," though if the antecedent to "them" is "composite Unicode glyph escapes," there's a logic error there: troff can't be failing to accept something that it's not even being given.  I read the "them" as referring to the sequence that preconv does in fact emit from the above input:

  • coo\[u0308]perate

So this bug could be fixed by making groff as liberal as possible with the forms of Unicode escapes it recognizes.  It already knows three equivalent forms of this string:

  • co\[:o]perate (historical roff style)
  • co\[u00F6]perate (NFC)
  • co\[u006F_0308]perate (NFD, but "spelled" differently)

(A fix for bug #58796 would enable preconv to emit the first of the above.)

The advantages of making groff grok the "o\[u0308]" sequence are:

  1. it follows the “Be liberal in what you accept" principle
  2. preconv would not have to change
  3. groff could handle arbitrary combinations for which the other three forms do not exist, such as

* Spin\[u0308]al Tap

The drawback, as Branden elucidates it, is that it's significantly easier to change preconv to output something groff already recognizes.

In any case, the first step seems to be to define exactly what groff should--and, if relevant, should not--accept as valid forms of Unicode-escape input.

Dave <barx>
Group Member


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by barx (Submitted the item)

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.


    No changes have been made to this item

    Back to the top

    Powered by Savane 3.12.
    Corresponding source code