bugGNU troff - Bugs: bug #60666, [grotty] add OSC 8 support for...

 
 

bug #60666: [grotty] add OSC 8 support for hyperlinks

Submitted by:  G. Branden Robinson <gbranden>
Submitted on:  Tue 25 May 2021 03:04:47 AM UTC
 
Category:  Device - others/general Severity:  1 - Wish
Item Group:  Feature change Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       No canned response available

 

( Jump to the original submission )

Fri 01 Oct 2021 12:00:06 PM UTC, comment #7: 

commit 5c250cc6e79c998135b5c30582d28d9565693846
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Fri Oct 1 21:43:31 2021 +1000

    [man]: Use OSC 8 hyperlinks.

    * tmac/an.tmac: Add registers `mY` (output driver is "grotty") and `U`
      (generate hyperlinks).  `U` is intended to apply to hyperlinking
      support in general, not just grotty's OSC 8 feature.  If the output
      driver is either grohtml or grotty, make `U` true (`man.local` is read
      subsequently and can override it).

      (UR, MT): Conditionalize diversion production on hyperlink support,
      not grohtml use per se.

      (UE, ME): Generate device control commands for `tty` device to emit
      hyperlinks.

    * tmac/groff_man.7.man.in (Options): Document -U.

      (Files) [style]: Add example of `U` register enablement.

    * tmac/man.local: Disable `U` register by default, except on the `html`
      device.

    * NEWS: Add item.

commit ab73e8189988d15bae12a82c6b3eb07948eda1d7
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Fri Oct 1 20:43:30 2021 +1000

    [grotty]: Add OSC 8 hyperlink support.

    * src/devices/grotty/tty.cpp: Do it.  Define `OSC` (Operating System
      Command) and `ST` (String Terminator) preprocessor symbols for these
      ECMA-48 (ISO 6429) character sequences.

      (tty_printer::simple_add_char): Add stripped-down alternative to
      `add_char()` member function for cases where we want to use many
      defaults because we're writing a device control command, not rendering
      a glyph.  (A function like `add_char()` that take 8 arguments of
      varying types is a code smell--phew!)

      (tty_printer::special): Call `special_link()` member function if the
      device control command is `link`.

      (tty_printer::special_link): Add new member function to generate OSC 8
      hyperlinks.

    * src/devices/grotty/grotty.1.man: Document it.  Observe in multiple
      places that disablement of SGR escape sequences disables OSC 8 too.

    * src/devices/grotty/tests/osc8_works.sh: Test it.
    * src/devices/grotty/grotty.am (TESTS, grotty_TESTS): Run test.
      (EXTRA_DIST): Ship test.

    * NEWS: Add item.

    Fixes <https://savannah.gnu.org/bugs/?60666>.  Thanks to Steffen
    Nurpmeso for supplying a proof-of-concept.  (I went with my own
    implementation, though, so blame me if it breaks.)

G. Branden Robinson <gbranden>
Project Administrator
Fri 01 Oct 2021 02:45:18 AM UTC, comment #6: 

Here's the current WIP.  Some work needs to be done in src/roff/troff/input.cpp:encode_char() to stop throwing warnings about some escape sequences.  Maybe that type of warning needs to be defeasible with a flag, or a new request that strips such things from strings developed.

(file #52008)

G. Branden Robinson <gbranden>
Project Administrator
Wed 02 Jun 2021 11:39:02 PM UTC, comment #5: 

Hello Branden.
Well, sure. I saw your SGR thing already by then. I do not like introducing more and more environment variables for each and every thing myself. However, .. end-users can only control via that.  More or less. There should be a well-known resource file for groff in a XDG path or in ~/ or wherever, maybe from the start. How many users know about groff resource control? Just a thought.

The idea of calling it "link", well, why not. OSC 8 may go away, hmhmm :)
Well why not do both and let link alias OSC 8, then maybe it could even be configurable.

A special request to turn it on and off like is available for sgr .. is redundant when sgr umbrellas OSC 8, otherwise, sure. Never used that myself, that on/off thing ;) But sure, having the switch is better than not having it. Like that everybody could if she or he would.

Necessary code changes seem trivial (as the code is as such).

Ciao from Germany. (I am not subscribed.)

Anonymous
Wed 26 May 2021 06:52:31 AM UTC, comment #4: 

Sorry, forgot to preview--that thing was riddled with typos.

Second try.

My design and implementation thoughts.

1. As suggested earlier, I think we want two device control keywords.  One, "osc8" to enable/disable this feature.  The other, to actually write the link to the output stream, I think we should call something more obvious, like "link".  Advantages of this include (a) not having to expose yet another environment variable for this and (b) making the device control for writing a hyperlink independent of its representation in the output.  Consider the possibility that OSC 8 is deprecated at some point in the future in favor of some alternative mechanism.  It will be misleading and confusing to tell the device 'X tty: osc8 id=foo url=bar' if the output sent to the terminal doesn't use OSC 8.  It will not be disruptive to change some per-device tmac files to switch out 'tty: osc8' for the new flavor.  For us, this change would be confined to tty.tmac, and people could get ahead of us in troffrc-end if they wanted to.

2. I'd cause SGR disablement (whether by -c option, "sgr" device command, or GROFF_NO_SGR environment variable) to also implicitly disable OSC 8; while they aren't fundamentally coupled, they do happen to both be ISO 6429/ECMA-48 escape sequences.  We don't have any way at all to support hyperlinks in the legacy output format.  If the many fans of backspaces in BSD world can come up with a statistically-unlikely sequence of backslashes and underscores with which to embed hyperlinks, then we can revisit this question, but I think it's unlikely to arise.  Moreover, if people share Ingo's concern that SGR itself constitutes a security vulnerability, then I can't imagine that URL embedding wouldn't arouse even more agitation.  So I'd lose the last chunk of the diff entirely.

3. While I personally prefer the "char const" form of declaration over "const char" because it is more regular, it is not idiomatic in the groff source tree and Ralph Corderoy's and my suggestion to migrate was ill met on the groff mailing list about three years ago[1][2].

Miscellaneous factors include:

90. I think it's habit that keeps people using the 7-bit forms for ISO 6429/ECMA-48.  CSI and OSC are both well-defined C1 controls and have been for decades.  I'd kind of like to make grotty(1) emit them.  This will probably turn up a lot of terminal emulators that don't support them. :-/  Maybe an "8bit" device control command is also in order.  But this is out of scope.

Comments and disputation welcome as always.

Regards,
Branden

[1] https://lists.gnu.org/archive/html/groff/2018-05/msg00028.html
[2] https://lists.gnu.org/archive/html/groff/2018-05/msg00042.html

G. Branden Robinson <gbranden>
Project Administrator
Wed 26 May 2021 06:44:34 AM UTC, comment #3: 

My design and implementation thoughts.

1. As suggested earlier, I think we want two device control keywords.  One, "osc8" to enable/disable this feature.  The other, to actual direct the output of the link, I think we should call something more obvious, like "link".  Advantages of this include (a) not having to expose yet another environment variable for this and (b) making the device control for writing a hyperlink independent if its representation in the output.  Consider the possibility that OSC 8 is deprecated at some point in the future in favor of some alternative mechanism.  It will be misleading and confusing to tell the device 'X tty: osc8 id=foo url=bar' if the output sent to the terminal doesn't use OSC 8.

2. I'd cause SGR disablement (whether by -c option, "sgr" device command, or GROFF_NO_SGR environment variable) to also implicitly disable OSC 8; we don't have any way to support these links in the legacy output format.  If the many fans of backspaces in BSD world can come up with a statistically-unlikely sequence of backslashes and underscores with which to embed hyperlinks, then we can revisit this question, but I think it's unlikely.  Moreover, if people share Ingo's concern that SGR itself constitutes a security vulnerability, then I can't imagine that URL embedding would arouse even more agitation.  So I'd lose the last chunk of the diff entirely.

3. While I personally prefer the "char const" form of declaration over "const char" because it is more regular, it is not idiomatic in the groff source tree and Ralph Corderoy's and my suggestion to migrate was ill met on the groff mailing list about three years ago[1][2].

Miscellaneous factors include:

90. I think it's habit that keeps people using the 7-bit forms for ISO 6429/ECMA-48.  CSI and OSC are both well-defined C1 controls and have been for decades.  I'd kind of like to make grotty(1) emit them.  This will probably turn up a lot of terminal emulators that don't support them. :-/  Maybe an "8bit" device control command is also in order.  But this is out of scope.

Comments and disputation welcome as always.

Regards,
Branden

[1] https://lists.gnu.org/archive/html/groff/2018-05/msg00028.html
[2] https://lists.gnu.org/archive/html/groff/2018-05/msg00042.html

G. Branden Robinson <gbranden>
Project Administrator
Wed 26 May 2021 06:17:53 AM UTC, comment #2: 

Thanks, Steffen!

I'm inlining your groff patch; I have comments on it.  I'd like to share my thinking so other groff contributors or observers can weigh in.


From ee1627a04640576719d45c6be0807aeb9dbdacec Mon Sep 17 00:00:00 2001
Message-Id: <ee1627a04640576719d45c6be0807aeb9dbdacec.1621901733.git.steffen@sdaoden.eu>
From: Steffen Nurpmeso <steffen@sdaoden.eu>
Date: Tue, 25 May 2021 02:07:15 +0200
Subject: [PATCH] src/devices/grotty/tty.cpp: add support for \X'tty: osc8 [id
 ID] [uri URI]'

---
 src/devices/grotty/tty.cpp | 79 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/src/devices/grotty/tty.cpp b/src/devices/grotty/tty.cpp
index 72f963e3de..550f0ef170 100644
--- a/src/devices/grotty/tty.cpp
+++ b/src/devices/grotty/tty.cpp
@@ -19,6 +19,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>. */

 #include "driver.h"
 #include "device.h"
+#include "cset.h"
 #include "ptable.h"

 typedef signed char schar;
@@ -56,6 +57,7 @@ static int italic_flag;
 static int reverse_flag_option = 0;
 static int reverse_flag;
 static int old_drawing_scheme = 0;
+static int no_osc8_refs = 0;

 static void update_options();
 static void usage(FILE *stream);
@@ -80,8 +82,12 @@ static unsigned char bold_underline_mode;

 #ifndef IS_EBCDIC_HOST
 #define CSI "\033["
+#define OSC8 "\033]8;"
+#define ST "\033\\"
 #else
 #define CSI "\047["
+#define OSC8 "\047]8;"
+#define ST "\047\\"
 #endif

 // SGR handling (ISO 6429)
@@ -189,6 +195,7 @@ class tty_printer : public printer {
   void line(int, int, int, int, color *, color *);
   void draw_line(int *, int, const environment *);
   void draw_polygon(int *, int, const environment *);
+  void special_osc8(char *, const environment *);
 public:
   tty_printer();
   ~tty_printer();
@@ -443,6 +450,76 @@ void tty_printer::special(char *arg, const environment *env, char type)
       old_drawing_scheme = 0;
     update_options();
   }
+  else if (strncmp(command, "osc8", p - command) == 0 && !no_osc8_refs)
+    special_osc8(p, env);
+}
+
+void
+tty_printer::special_osc8(char *ap, const environment *env){
+  size_t idl, uril, *lp;
+  char *ap_orig, *idp, *urip, *cp, **bpp;
+
+  ap_orig = ap;
+  idl = uril = 0;
+  idp = urip = NULL;
+
+  for(lp = NULL; *ap != '\0';){
+    for(; *ap == ' ' || *ap == '\n'; ++ap)
+      ;
+    cp = ap;
+    for(; *ap != '\0' && *ap != ' ' && *ap != '\n'; ++ap)
+      ;
+    if(cp == ap){
+      assert(*ap == '\0');
+      break;
+    }
+
+    if(lp != NULL){
+      *bpp = cp;
+      *lp = (size_t)(ap - cp);
+      lp = NULL;
+
+      for(; cp < ap; ++cp)
+        if(!csprint(*cp)){
+          error("invalid X osc8 parameter content, ignoring: %1", ap_orig);
+          goto jleave;
+        }
+    }else if(!strncmp(cp, "id", (size_t)(ap - cp))){
+      lp = &idl;
+      bpp = &idp;
+    }else if(!strncmp(cp, "uri", (size_t)(ap - cp))){
+      lp = &uril;
+      bpp = &urip;
+    }else{
+      error("unknown X osc8 parameter, ignoring: %1", ap_orig);
+      goto jleave;
+    }
+  }
+
+  if(lp != NULL){
+    error("missing argument of X osc8 parameter, ignoring: %1", ap_orig);
+    goto jleave;
+  }
+
+  for(char const *ccp = OSC8; *ccp != '\0'; ++ccp)
+    add_char(*ccp, 0, env->hpos, env->vpos, env->col, env->fill, 0);
+
+  if(idp != NULL){
+    for(char const *ccp = "id="; *ccp != '\0'; ++ccp)
+      add_char(*ccp, 0, env->hpos, env->vpos, env->col, env->fill, 0);
+    while(idl-- != 0)
+      add_char(*idp++, 0, env->hpos, env->vpos, env->col, env->fill, 0);
+  }
+
+  add_char(';', 0, env->hpos, env->vpos, env->col, env->fill, 0);
+
+  while(uril-- != 0)
+    add_char(*urip++, 0, env->hpos, env->vpos, env->col, env->fill, 0);
+
+  for(char const *ccp = ST; *ccp != '\0'; ++ccp)
+    add_char(*ccp, 0, env->hpos, env->vpos, env->col, env->fill, 0);
+
+jleave:;
 }

 void tty_printer::change_color(const environment * const env)
@@ -836,6 +913,8 @@ int main(int argc, char **argv)
   static char stderr_buf[BUFSIZ];
   if (getenv("GROFF_NO_SGR"))
     old_drawing_scheme = 1;
+  if(getenv("GROFF_NO_OSC8"))
+    no_osc8_refs = 1;
   setbuf(stderr, stderr_buf);
   setlocale(LC_CTYPE, "");
   int c;
--
2.31.1

G. Branden Robinson <gbranden>
Project Administrator
Tue 25 May 2021 06:56:18 PM UTC, comment #1: 

Great! Let me attach my final patches to grotty and less (your linked one does not yet securely quote content).
It would be great to see OSC 8 addition to grotty, not unlikely converters like docbook etc. would start using it.
(I am not subscribed.)

Ciao!

(file #51478, file #51479)

Anonymous
Tue 25 May 2021 03:04:47 AM UTC, original submission:  

OSC 8 (Operating System Command 8) is a *nix community-initiated standard for supporting hyperlinks in terminal emulators within the established syntax of ISO 6429/ECMA-48.

https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda

This isn't strictly a prerequisite for the new .MR (man page cross reference) feature addition to man(7) I've been rattling my saber about for months, but it sure would make it easier to test, since at heart I'm a terminal junkie.

That said I'll have to launch of the terminal emulators listed at the URL above, because my old standby xterm doesn't support this feature yet.

Here's what I think this would look like.

  • Add a device control in the 'tty:' space called 'osc8' in parallel with the current 'sgr' to enable or disable the production of OSC 8 terminal escape sequences.
  • GROFF_NO_SGR would also disable OSC 8 sequences, since its semantics are to use the legacy output format.  This precludes the emission of OSC 8 terminal escape sequences.

[...time passes...]

This report sat in a browser tab so long that Steffen Nurpsemo, whose message[1] to the groff mailing list alerted me to the existence of the OSC 8 initiative in the first place has now posted a patch.[2]

So I'll probably base any further design and development on that foundation.

[1] https://lists.gnu.org/archive/html/groff/2021-05/msg00062.html
[2] https://lists.gnu.org/archive/html/groff/2021-05/msg00073.html

G. Branden Robinson <gbranden>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #52008:  60666-WIP-2021-10-01.diff added by gbranden (8KiB - text/x-patch)
file #51478:  grotty-osc8.patch added by None (4KiB - application/octet-stream)
file #51479:  less-osc8-search.patch added by None (12KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

     

     

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-10-01 gbranden StatusIn Progress => Fixed
        Open/ClosedOpen => Closed
        Planned ReleaseNone => 1.23.0
    2021-10-01 gbranden Attached File- => Added 60666-WIP-2021-10-01.diff, #52008
    2021-05-26 gbranden StatusNone => In Progress
    2021-05-25 None Attached File- => Added grotty-osc8.patch, #51478
        Attached File- => Added less-osc8-search.patch, #51479

    Back to the top


    Powered by Savane 3.9