bugGNU roff - Bugs: bug #62909, [chem] does not restrict...

 
 

bug #62909: [chem] does not restrict whitespace/comment changes to its own regions

Submitter:  None
Submitted:  Thu 18 Aug 2022 04:07:00 AM UTC
   
 
Category:  Preprocessor - others/general Severity:  3 - Normal
Item Group:  Incorrect behaviour Status:  None
Privacy:  Public Assigned to:  None
Open/Closed:  Open Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 23 Aug 2022 05:51:00 AM UTC, comment #8: 

I have created a github project for the script.

https://github.com/rdindir/chem-na

Riza Dindir <rdindir>
Tue 23 Aug 2022 04:44:22 AM UTC, comment #7: 

Forgot to attach how I am using the script. It is a simple shell script that will invoke awk. The code is given below.

================================================================================

#!/bin/sh
# The driver script for the =non-aggressive chem tool.
# Author: Riza Dindir
# License: The GNU General Public License applies (<https://www.gnu.org/licenses/>)

swd=$(dirname $0)

awk -f ${swd}/_chem-na/chem-na.awk $*

Riza Dindir <rdindir>
Tue 23 Aug 2022 04:38:56 AM UTC, comment #6: 

I did not test this script thoroughly. It is a simple script that only processes the chem sections (cstart/cend).

I share it here. The script has been written by myself and is not being used on other projects.

The tabs have been removed (after I viewed it in the preview). Hope that is not a problem.

The license should be GNU general public license.

The file is below

======================================================================================

# This awk script is a non-aggressive chem tool.
# Author: Riza Dindir
# License: The GNU General Public License applies (<https://www.gnu.org/licenses/>)

# Resets the file by emptying its contents.
function resetFile(fn) {
cmd = "cat /dev/null > " fn;
print cmd | "/bin/sh";
}

# Appends line to the file with name "fn".
function appendFile(fn, line) {
cmd = "echo \"" line "\" >> " fn;
print cmd | "/bin/sh";
}

# Cleans the line by converting " to \".
function cleanLine(line) {
gsub(/"/, "\\\"", line);
}

# Runs the chem tool with the given file name.
function runChem(fn) {
system("chem " fn);
}

BEGIN {
inchem = 0;
fn = "~/tmp/chem.tr";
resetFile(fn);
}

inchem == 0 && /^\.cstart/ {
line = $0;
cleanLine(line);
appendFile(fn, line);
inchem = 1;
next;
}

inchem == 1 && /^\.cstart/ {
print "You can not start another chem region when inside one.";
exit 1;
}

inchem == 0 && /^\.cend/ {
print "You can not end chem region when you are not in one.";
exit 1;
}

inchem == 1 && /^\.cend/ {
line = $0;
cleanLine(line);
appendFile(fn, line);
runChem(fn);
resetFile(fn);
inchem = 0;
next;
}

inchem == 1 {
line = $0;
cleanLine(line);
appendFile(fn, line);
next;
}

{ print $0; }

END {}


Riza Dindir <rdindir>
Mon 22 Aug 2022 09:59:07 PM UTC, comment #5: 

comment #4:

> Hi Riza,
>
> You mentioned in http://lists.gnu.org/r/groff/2022-08/msg00199.html that you wrote an awk script to work around this problem.  If you're willing to share it (and legally permitted), would you mind attaching it to this bug report, so that others who encounter the problem can benefit from your work until the chem code can be patched?  (And even if it's fixed quickly, most users won't run the updated code until it makes it into an official release.)  Thanks for considering this.


There's no urgency here.  I've already started refactoring chem.

I suspect the state machine is a bit buggy near the top level; chem should pass through unmolested any lines that aren't in a "chem" region.

The amount of pointless work that is being done in argument processing here is truly staggering.  But I need to make sure I don't screw anything up.

Anyway, I've got some of the refactoring set for my next push and to go farther I need to write a regression test, possibly using the very examples from the Kernighan chem paper that we already ship.  If that doesn't already handle cases of leading space- and hash-mark-containing lines outside of chem regions, it will be straightforward to add these.


commit b28c6b5ea7834bb25a9b5c7c704f8062fa0b6410
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu Aug 18 09:06:27 2022 -0500

    [chem]: Stop copying "pic.tmac" into output.

    * contrib/chem/chem.pl: Stop copying "pic.tmac" (fallback troff macro
      definitions) into the output.  It might not be necessary and it is
      inappropriate to do so if a macro package offers its own definitions
      or the user has made other arrangements on the command line.  (The
      same thing can be achieved with the "-mpic" argument to the formatter
      or a front end.)  Bump version number.

 contrib/chem/ChangeLog |  9 +++++++++
 contrib/chem/chem.pl   | 10 +---------
 2 files changed, 10 insertions(+), 9 deletions(-)

commit bb329e944af54da2ae46a32555133dd2ca72d01a
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Thu Aug 18 09:46:00 2022 -0500

    [chem]: Refactor.

    * contrib/chem/chem.pl: Refactor.
      - Rename scalars.
        Copyright -> copyright
        Program_Version -> chem_version
        Groff_Version -> groff_version
        Chem_Name -> chem
        before_make -> is_in_source_tree
      - Rename hash.
        at_at -> makevar
      - Drop unused hash member `$makevar{'BINDIR'}`.
      - Drop scalar `Groff_Version_Preset`, which apparently hadn't
        been updated since groff 1.20.  Instead, follow grog(1) and
        set the `groff_version` scalar to "DEVELOPMENT" if this is
        the version from the groff source tree.  Overwrite its value
        with that determined by make(1) if available.
      - Tighten usage and version messages; make the latter more
        conformant with the format recommended in the GNU Coding
        Standards.  Explicitly identify license as GNU GPLv2.

      (usage): Refer to "pic" with its command prefix if it is known
      to have one.  (If running "unbuilt", we have no way to know.)

 contrib/chem/ChangeLog |  23 ++++++++++
 contrib/chem/chem.pl   | 111 ++++++++++++++++++-------------------------------
 2 files changed, 64 insertions(+), 70 deletions(-)


And here's what's pending with the argument processing cleanup, from my Git stash.


 contrib/chem/chem.pl | 279 +++++++++++++++++++++------------------------------
 1 file changed, 114 insertions(+), 165 deletions(-)


That includes about 30 lines that are simply commented out.  If I can persuade myself that we can do without them, the proportion of deletions will jump up.

G. Branden Robinson <gbranden>
Group administrator
Mon 22 Aug 2022 08:59:43 PM UTC, comment #4: 

Hi Riza,

You mentioned in http://lists.gnu.org/r/groff/2022-08/msg00199.html that you wrote an awk script to work around this problem.  If you're willing to share it (and legally permitted), would you mind attaching it to this bug report, so that others who encounter the problem can benefit from your work until the chem code can be patched?  (And even if it's fixed quickly, most users won't run the updated code until it makes it into an official release.)  Thanks for considering this.

Dave <barx>
Group Member
Fri 19 Aug 2022 02:51:34 AM UTC, comment #3: 

Hello Branden

I was thinking of how I would get notifications, but then realized at the end of the page that I could add myself to the mail notification list. :)

As to a solution, maybe a filter could be written that will accumulate the chem regions into a file and at the end the file could then be run through the original chem preprocessor, and output that result.

Might not be an optimal solution, but might work.

Riza

Riza Dindir <rdindir>
Thu 18 Aug 2022 11:27:44 AM UTC, comment #2: 


comment #1:

> When I submitted this bug report, I did not have an account.
>
> How can I change the submitter. Currently the submitter is shown as None.


Hi Riza,

I don't know of a way to do this.  However by commenting on the ticket you've added yourself to the auto-CC list, which will keep you in the loop on developments.  For practical purposes this is just as good as being the submitter.

This is a common issue with first-time groff bug filers.  Don't worry, we'll remember who reported this issue.  :)

--Branden

G. Branden Robinson <gbranden>
Group administrator
Thu 18 Aug 2022 04:21:56 AM UTC, comment #1: 

When I submitted this bug report, I did not have an account.

How can I change the submitter. Currently the submitter is shown as None.

Riza Dindir <rdindir>
Thu 18 Aug 2022 04:07:00 AM UTC, original submission:  

Hello,

I am having trouble with chem and dformat preprocessors.

I have downloaded dfromat from (https://github.com/arnoldrobbins/dformat).

I have an example document that has one section that is a dformat structure, another that has a chem structure.

I think that the chem preprocessor processes incorrectly all the lines in the document. It should only process the lines defined in its own block delimited with cstart/cend.

But to the contrary, I am seeing that the chem preprocessor removes the whitespaces that are infront of the lines in the whole document, and other filters that depend on these whitespace characters (dformat) can not function properly.

Also chem processes comments in the whole document (commensts start with a # and extend to the end of line), but it should only process its own block for comments (delimited with cstart/cend).

Below are the problems...

Problem 1: Chem strips all the whitespace preceeding the lines in dformat (and maybe all the whitespace of all the lines in the document). It should only do this inside its own block (cstart/cend).

When I run

cat test.ms | chem > test-chem.tr

chem strips all the whitespace in the dformat section. dformat requires whitespace preceeding the format specs. You can check the dformat block in the test.ms file and test-chem.tr file.

Problem 2: chem processes all the comments (comments start with a # and continue to the end of line) in the document, not in its own block (cstart/cend)

When I run

cat test.ms | dformat | chem > test-dformat-chem.tr

chem will process all the comments in the whole document. But it should only process comments in its own block (delimited with cstart/cend).

If you inspect the test-dformat-chem.tr You will see that the item in dformat that reads "Channel #" is processed to

"Channel

the line is incorrect. It should have been

   "Channel #" at BoxB.c

with whitespaces infront of it. But chem did strip these too. See and compare test-dformat.tr and test-dformat.chem.tr.

Note: The test-dformat.tr has been generated using

cat test.ms | dformat > test-dformat.tr

I have attached the document (test.ms) and the outputs to the bug report. I could not include the dformat preprocessor. But have mentioned the link to it and am including these below. I am using the preprocessor as such:

dformat (shell script)
======================

#!/bin/sh

awk -f dformat.awk $*

dforamt.awk
===========

function error(s)
{
print("dformat error: " s " near input line " NR) | "cat 1>&2"
}

BEGIN {
s = "recht 0.3 addrht 0.055 recspread 0.15 "
s = s "charwid 0.07 textht 0.167 addrdelta 4 "
s = s "bitwid 0.125 linedisp 0.04 addr both "
s = s "fill off linethrutext 1 "
s = s "shaded off"
n = split(s, x, FS)
for (i = 1; i <= n - 1; i += 2) oparm[x[i]] = parm[x[i]] = x[i + 1]
}

inlang == 0 {
if ($0 !~ /^\.begin[ \t]/ || $2 != "dformat") {
print
} else {
inlang = 1
print ".PS"
boxacnt = 0
if (firstpic != 1) {
firstpic = 1
print "fillval = 0.1"
}
}
next
}

/^\.end/ {
inlang = 0
print ".PE"
next
}

$1 == "style" {
if (! ($2 in parm)) {
error("unrecognized name: " $2)
} else if ($3 == "reset") {
t = oparm[$2]
oparm[$2] = parm[$2]
parm[$2] = t
} else {
oparm[$2] = parm[$2]
parm[$2] = $3
#error("set shaded to: " $3)
}
next
}

$1 == "pic" {
$1 = ""
print $0
next
}

/^[^ \t]/ {
printf "BoxA: box invis ht %g wid 0", parm["recht"]
if (boxacnt++) {
printf " with .n at BoxA.s - (0, %g)",
   parm["recspread"] + maxdy * parm["textht"]
}
printf "\n"

maxdy = sumboxlen = 0
gsub(/[ \t]+$/, "", $0)
if ($0 != "noname") {
printf " \"%s \" rjust at BoxA.w\n", $0
printf " box invis with .e at BoxA.w ht 0 wid %g\n",
    parm["charwid"] * (length($0) + 3)
}
printf " BoxB: box invis ht %g wid 0 at BoxA\n", parm["recht"]

next
}

/./ {
boxname = ""
if ($1 ~ /:$/) {
boxname = substr($1, 1, length($1) - 1)
$1 = ""
$0 = " " $0
}
range = $1
$1 = ""
gsub(/^[ \t]+/, "", $0)
gsub(/[ \t]+$/, "", $0)
text = $0

n = split(range, x, "-")
rlo = x[1]
rhi = (n >= 2) ? x[2] : rlo
cwid = (rhi >= rlo) ? rhi - rlo + 1 : rlo - rhi + 1
rwid = (n >= 3) ? (0 + x[3]) : cwid
btype = x[4]
if (btype !~ /^(dot|dash|invis)/) {
btype = "solid"
}

textlen = parm["charwid"] * length(text)
boxlen = parm["bitwid"] * rwid

dy = 0
if (textlen > boxlen) {
# set dy, the channel for this text
chan[maxdy + 1] = -999
for (dy = 1; chan[dy] + textlen > sumboxlen; dy++);
if (dy > maxdy) {
maxdy = dy
}
if (parm["linethrutext"] == 0) {
for (k = 1; k <= dy; k++) {
chan[k] = sumboxlen + boxlen
}
} else {
chan[dy] = sumboxlen
}
}
sumboxlen += boxlen
fill = ""

if (parm["fill"] == "on") {
fill = " fill "
}

shaded = ""
if (parm["shaded"] != "off") {
shaded = " shaded "
shaded = shaded "\"" parm["shaded"] "\" "
}

if (boxname != "") {
printf " %s:", boxname
}
printf "  BoxB: box %s %s %s ht %g wid %g with .w at BoxB.e\n",
   shaded, fill, btype, parm["recht"], boxlen

if (dy == 0) {
printf "    \"%s\" at BoxB.c\n", text
} else {
if (rwid < 2) {
start = "BoxB.s"
} else {
start = "BoxB.se - (" parm["linedisp"] ",0)"
}
printf "    line from %s down %g\n", start, dy * parm["textht"]
printf "    \"%s\\|\" at last line .s rjust\n", text
printf "    box invis with .e at last line .s ht 0 wid %g\n",
    textlen
}

if (parm["addr"] ~ /^(left|right|both)$/) {
dp = int(parm["addrdelta"])     # Delta Point size
if (dp < 0 || dp > 9) {
error("bad addrdelta value: " dp)
}

dah = parm["addrht"]            # Delta Addr Height
pb = parm["addr"]               # Parameter for Bits

if (rlo == rhi) {
printf "    \"\\s-%d%s\\s+%d\" at BoxB.s + (0,%g)\n",
   dp, rlo, dp, dah
} else {
if (pb == "left" || pb == "both") {
printf "\t\"\\|\\s-%d%s\\s+%d\" ljust at BoxB.sw + (0,%g)\n",
   dp, rlo, dp, dah
}
if (pb == "right" || pb == "both") {
printf "\t\"\\s-%d%s\\s+%d\\|\" rjust at BoxB.se + (0,%g)\n",
   dp, rhi, dp, dah
}
}
}
}

END {
if (inlang) {
error("eof inside begin/end")
}
}

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53575:  test.ms added by None (1KiB - text/x-troff-ms - main document and output files.)
file #53576:  test-chem.tr added by None (4KiB - text/troff - main document and output files.)
file #53577:  test-dformat.tr added by None (4KiB - text/troff - main document and output files.)
file #53578:  test-dformat-chem.tr added by None (7KiB - text/troff - main document and output files.)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by barx (Posted a comment)
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by rdindir (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-08-18 gbranden Item GroupNone Incorrect behaviour
        SummaryPreprocessror problem - Groff with dformat and chem [chem] does not restrict whitespace/comment changes to its own regions
    2022-08-18 None Attached File- Added test.ms, #53575
        Attached File- Added test-chem.tr, #53576
        Attached File- Added test-dformat.tr, #53577
        Attached File- Added test-dformat-chem.tr, #53578

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code