bugGNU roff - Bugs: bug #65710, [preconv] require disambiguation...

 
 

bug #65710: [preconv] require disambiguation of U+00A0 on input

Submitter:  G. Branden Robinson <gbranden>
Submitted:  Tue 07 May 2024 11:50:39 AM UTC
   
 
Category:  Utilities Severity:  3 - Normal
Item Group:  Feature change Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Wed 12 Jun 2024 02:35:04 AM UTC, comment #4: 

comment #3:

> Bjarni's prescription was much too strong.


Bjarni's "prescription" was an editorial recommendation to users that had no bearing on his proposed code change.  Users would be free to ignore it.

The current proposal--to upgrade his proposed warning to a fatal error--only intensifies the problems.  As I said over there, preconv should not be in the business of policing which parts of valid Unicode users use.  The new proposal kicks that policing from a written warning up to jail time.  It's wildly out of proportion with the offense.

> Maybe they mean `\ ` (an unadjustable space).  It's
> impossible to know, which is why they should disambiguate it.


It's technically ambiguous, in the same way that "We're going to spend the week in a cabin" is technically ambiguous: that person could be talking about either a house in the woods or the interior of an airplane.  But anyone hearing it will know exactly what they mean--crucially, because if they did mean the latter, they'd specifically note it.  When you say something mildly ambiguous, but with one meaning far more likely than another, it's only the exceptional meaning that tends to need to be disambiguated.

Even the roff language--hardly a paragon of DWIM design--understands this.  You need only say ".sp 4" to space down four lines; you don't have to specify "4v" because roff gives the request a sensible default unit.

Likewise, \~ is the sensible default meaning for U+00A0: in almost all normal situations, the user will want \~.  The documentation clearly spells out how to get different {units for .sp / types of nonbreaking spaces}, so users who want the rarer \space in certain places can explicitly say so.

Making the user edit a bunch of valid Unicode characters (or valid ISO 8859-1, or 8859-2, or any other encoding in the ISO 8859 family) only impedes preconv's ability to import text from another source and use it directly.  We should be making this easier for users, not putting up needless roadblocks in the name of semantic purity, certainly not without wider discussion.

> And if there are multiple U+00A0 characters in sequence, the
> author might be better off supplying a `\h` sequence to
> express what it is they want, precisely.


Sure.  But the formatter allows \~\~\~\~\~ without complaint, and adding a complaint here is beyond what this ticket is proposing, so this is tangential.

Dave <barx>
Group Member
Wed 05 Jun 2024 03:57:21 AM UTC, comment #3: 


comment #2:

> original submission:
> > In my opinion preconv should either:
> >
> > 1.  Fail and force the user to edit the input to make a real
> > choice, eliminating U+00A0 in the input.
>
> This option seems materially the same as the rejected bug #65654, except with that bug's proposed warning upgraded to a fatal error.  (Granted, #65654 was rejected a few days before bug #65693 (the current ticket's inciting report) was opened.)


Yes.

Bjarni's prescription was much too strong.


Subject: preconv.cpp: Issue a warning if code '0xA0' is used in
the input and thus changed to '\~'

File: "src/preproc/preconv/preconv.cpp"
Subroutine: unicode_entity(int u)

  The code '0xA0' is displayed as a normal space, and is thus
visually the same as it.

  So authors better use '\~' directly in their files.


No, authors has better not, necessarily, use `\~` directly in their files.  Maybe they mean `\ ` (an unadjustable space).  It's impossible to know, which is why they should disambiguate it.

And if there are multiple U+00A0 characters in sequence, the author might be better off supplying a `\h` sequence to express what it is they want, precisely.

A ticket grounded on sound principles often meets a better fate than one grounded on poor ones.

One could of course accuse me of authorial ("submittorial"?) bias...

At this point I'm leaning toward solution 1 from comment #0.

G. Branden Robinson <gbranden>
Group administrator
Mon 27 May 2024 07:31:42 PM UTC, comment #2: 

original submission:

> In my opinion preconv should either:
>
> 1.  Fail and force the user to edit the input to make a real
> choice, eliminating U+00A0 in the input.


This option seems materially the same as the rejected bug #65654, except with that bug's proposed warning upgraded to a fatal error.  (Granted, #65654 was rejected a few days before bug #65693 (the current ticket's inciting report) was opened.)

Dave <barx>
Group Member
Wed 08 May 2024 08:59:11 PM UTC, comment #1: 

original submission:

> an input U+00A0...: is it a fixed-width non-breakable space, or a
> variable-width non-breakable space?  Unicode does not distinguish.


Unicode intentionally provides some latitude to rendering engines to make the best typographical decisions they can.  That is, Unicode is not really a driving factor here; the only Unicode requirement of U+00A0 is that it be nonbreaking, and both "\ " and "\~" meet that requirement.

Since 2003 (commit 48615a44), groff_char(7) has said "the ISO Latin-1 no-break space is mapped to `\~', the stretchable space character."  While this claim was erroneous until bug #58962 addressed it (and remains erroneous with certain fonts, per bug #65693), it has always been the most typographically sound solution.

I think groff should strive to do by default what makes most typographic sense, and (as #58962 argued) there are few if any situations where "\ " is preferable to "\~".

> This is analogous to how we don't know whether a man page
> author means a hyphen or a minus sign when they type `-`.


Weakly analogous:

  • Hyphens and minus signs are different semantically.  The two types of nonbreaking spaces are different only presentationally.
  • Hyphens and minus signs must copy to the clipboard as different characters; both types of nonbreaking spaces will possibly copy as U+00A0, though more probably (and more usefully to the user in most cases) copy as an ordinary space.


> 2.  Add an option instructing preconv which escape sequence to
> transform U+00A0 into.


I don't oppose an option to override the default escape preconv uses, but its current default is sound.  (Still, I would question the need for such an option, since users can always specify either escape directly in the source, and preconv won't touch it.)

Dave <barx>
Group Member
Tue 07 May 2024 11:50:39 AM UTC, original submission:  

Considering bug #65693, I suggest that it may be worth treating an input U+00A0 as ambiguous: is it a fixed-width non-breakable space, or a variable-width non-breakable space?  Unicode does not distinguish.

This is analogous to how we don't know whether a man page author means a hyphen or a minus sign when they type `-`.

In my opinion preconv should either:

1.  Fail and force the user to edit the input to make a real choice, eliminating U+00A0 in the input.

2.  Add an option instructing preconv which escape sequence to transform U+00A0 into.

G. Branden Robinson <gbranden>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by barx (Posted a comment)
  • -email is unavailable- added by gbranden (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-06-05 gbranden StatusNeed Info None

    Back to the top

    Powered by Savane 3.13-58d6.
    Corresponding source code