bugGNU roff - Bugs: bug #65716, [afmtodit] afmtodit.tables doesn't...

 
 

bug #65716: [afmtodit] afmtodit.tables doesn't reflect movement of characters out of Unicode PUA

Submitter:  Deri James <deri>
Submitted:  Wed 08 May 2024 11:47:44 PM UTC
   
 
Category:  Utilities Severity:  3 - Normal
Item Group:  Build/Installation Status:  Need Info
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 22 Sep 2024 05:16:21 PM UTC, comment #10: 

Adobe have not updated symbol glyph list since unicode 2.0 but Apple, who used the same mapping as Adobe (except for one glyph) updated up to v4.0, see:-
 
http://unicode.org/Public/MAPPINGS/VENDORS/APPLE/SYMBOL.TXT

Is this authorative enough?

Deri James <deri>
Group Member
Sun 22 Sep 2024 02:02:48 PM UTC, comment #9: 

Retitling, descheduling, and updating status to indicate blockage on external dependency.

G. Branden Robinson <gbranden>
Group administrator
Sun 22 Sep 2024 01:45:23 PM UTC, comment #8: 

At 2024-09-22T07:20:13-0400, Deri James wrote:

> Re-opened since Branden has not dealt with the fact that some of the
> Adobe Glyphs which were originally in the PUA are now in the unicode
> standard, if our glyph list was corrected with the current code
> points, this change is not necessary. See comment #4 below.


Oh.  Well, I guess this one will stay open indefinitely, then.  Neither
of us seems to know of an authoritative source for the corrected code
points.  If nobody's maintaining glyphlist.txt in this respect, then
that resource will need a successor.  And maybe there isn't one.

Regards,
Branden

G. Branden Robinson <gbranden>
Group administrator
Sun 22 Sep 2024 11:20:07 AM UTC, comment #7: 

Re-opened since Branden has not dealt with the fact that some of the Adobe Glyphs which were originally in the PUA are now in the unicode standard, if our glyph list was corrected with the current code points, this change is not necessary. See comment #4 below.

Deri James <deri>
Group Member
Sun 22 Sep 2024 04:39:17 AM UTC, comment #6: 


commit fc2f0abfbe67ed6a7b46c9c68db706b6e627cfae
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sat Sep 21 21:35:50 2024 -0500

    [afmtodit]: Fix Savannah #65716 (use PUA) (1/2).

    * src/utils/afmtodit/make-afmtodit-tables: Alter maintainer-mode script
      to stop excluding code points from the Private Use Area in the Unicode
      Basic Multilingual Plane from glyph mappings for Adobe fonts (and
      workalikes).  Doing so expands coverage to many more glyphs, including
      groff's "braceex".

    Fixes <https://savannah.gnu.org/bugs/?65716>.  Thanks to Deri James for
    the report.

    Also alter script comment to emphasize its function, not its name.

commit ed7788df43535ee002717ca58a74c884fb3de38e
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sat Sep 21 21:43:58 2024 -0500

    [afmtodit]: Fix Savannah #65716 (use PUA) (2/2).

    * src/utils/afmtodit/afmtodit.tables: Update with freshly changed
      "make-afmtodit-tables" script, adding ~192 mappings.

    Fixes <https://savannah.gnu.org/bugs/?65716> (2/2).  Thanks to Deri
    James for the report.


G. Branden Robinson <gbranden>
Group administrator
Sun 22 Sep 2024 02:26:50 AM UTC, comment #5: 


comment #2:

> Werner put that line in when he first checked in the script, in 2005.
>
> Can/should we do anything about this?


I think as an experiment, I'm going to try removing this.  As you note, it should improve font coverage considerably, and the BMP Private Use Area (like the ones in the upper planes) is not defined as "never use".


Private-use characters are assigned Unicode code points whose interpretation is not specified by this standard and whose use may be determined by private agreement among cooperating users. These characters are designated for private use and do not have defined, interpretable semantics except by private agreement.


I reckon if we choose to cooperate with glyphlist.txt for the purpose of rendering fonts encoded compatibly with it, that's a perfectly cromulent use of the PUA.

G. Branden Robinson <gbranden>
Group administrator
Sun 08 Sep 2024 04:58:35 PM UTC, comment #4: 

I believe since Unicode 3.2 some of the glyphs Adobe had in the PUA got their own code points in the 0x23xx block. I did find an apple glyph list in the same format with the new mapping, but I can't find it now, sorry, but there is this discussion about it at mozilla:-

https://www.mail-archive.com/fonts@xfree86.org/msg00790.html

braceex is included.

Deri James <deri>
Group Member
Sat 07 Sep 2024 01:40:10 PM UTC, comment #3: 

Got my powers back!

Oh, the exaltation!

G. Branden Robinson <gbranden>
Group administrator
Sat 07 Sep 2024 01:19:38 PM UTC, comment #2: 

The reason is this line:

https://git.savannah.gnu.org/cgit/groff.git/tree/src/utils/afmtodit/make-afmtodit-tables?h=1.23.0#n118


      -e '/;\(E\|F[0-8]\)/d' \


That's a sed line.  So it's nuking all Unicode code points matching E000..F8FF.  That's the Unicode Private Use Area.  So it seems to be a correct call.  Except we're also matching E0000..EFFFFF and F0000..F8FFFF, which...I don't know, may not be assigned yet.  But they're less than 10FFFF so they might be some day.

Werner put that line in when he first checked in the script, in 2005.

Can/should we do anything about this?

G. Branden Robinson <gbranden>
Group administrator
Sat 07 Sep 2024 01:00:56 PM UTC, comment #1: 

I've lost most of my privileges on Savannah (except, oddly, group-level administrative ones like "editing field values"), but I can still add comments to tickets).

There would seem to be a bug in "make-afmtodit-tables".

I was going to ask you to look into it "since it's in Perl, surely"...

...but it's a shell script.  Hah.  That puts me on the hook.

I'm one of those damned souls who has more shell scripting experience than is healthy for any ten hackers.  (Thanks, Debian package maintainer scripts!)

I'll have a look.


G. Branden Robinson <gbranden>
Group administrator
Wed 08 May 2024 11:47:44 PM UTC, original submission:  

As an example current glyphlist.txt has:-

bqsquare;33C3
braceex;F8F4
braceleft;007B

but our afmtodit.tables file has:-

  "bqsquare", "33C3",
  "braceleft", "007B",

"braceex" is missing. (300+ lines missing). Is there a reason for this? It seems to have been created 2022-10-09 which is later than the last change to glyphlist.txt. I have been writing a little tool to document the glyph properties of the fonts we deliver with groff. The braceex is used in our S font (see the "Private Use Area" unicode block in S.pdf).

Deri James <deri>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #56028:  S.pdf added by deri (47KiB - application/pdf)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by deri (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-09-22 gbranden StatusNone Need Info
        Assigned togbranden None
        Planned Release1.24.0 None
        Summaryafmtodit.tables omits many code points from glyphlist.txt [afmtodit] afmtodit.tables doesn't reflect movement of characters out of Unicode PUA
    2024-09-22 deri StatusFixed None
        Open/ClosedClosed Open
    2024-09-22 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.24.0
    2024-09-22 gbranden StatusNone In Progress
    2024-09-22 gbranden StatusNeed Info None
    2024-09-08 deri Assigned toderi gbranden
    2024-09-07 gbranden SummaryWe don't seem to be using the &quot;latest&quot; glyphlist.txt to generate the tables for afmtodit. afmtodit.tables omits many code points from glyphlist.txt
    2024-09-07 gbranden StatusNone Need Info
        Assigned toNone deri
    2024-05-08 deri Attached File- Added S.pdf, #56028

    Back to the top

    Powered by Savane 3.15-26b0.
    Corresponding source code