bugGNU roff - Bugs: bug #54101, documentation for .ss either...

 
 

bug #54101: documentation for .ss either incorrect or incomplete

Submitter:  Dave <barx>
Submitted:  Tue 12 Jun 2018 06:46:21 AM UTC
   
 
Category:  Core Severity:  3 - Normal
Item Group:  Documentation Status:  Fixed
Privacy:  Public Assigned to:  gbranden
Open/Closed:  Closed Planned Release:  1.23.0
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 04 Jun 2020 03:21:51 PM UTC, comment #21: 

Thanks, Dave.  I did back out the parenthetical cross-reference in bfbc45358ed4b7b41be2dba98a640f1b86661d86, because that was the node the material was already in.

But "automatically" remains, and I think that was the remaining issue requiring clarification.

G. Branden Robinson <gbranden>
Group administrator
Mon 01 Jun 2020 06:27:12 PM UTC, comment #20: 

Yes, thank you -- I think we can put this one to bed!

Dave <barx>
Group Member
Mon 01 Jun 2020 01:34:33 PM UTC, comment #19: 

Hi Dave,

Let me know if 1b35343a5c7ddc3f972c086aab6a1794f16d9446 does the trick.

G. Branden Robinson <gbranden>
Group administrator
Wed 27 May 2020 06:46:15 AM UTC, comment #18: 

Reopening per Dave's request and the remaining issue in comment #16.

G. Branden Robinson <gbranden>
Group administrator
Wed 27 May 2020 04:52:24 AM UTC, comment #17: 

comment #16:

> The issue discussed in comment #14 probably warrants a new bug report.


That new report is bug #58450.

As for the issue raised in comment #16: it appears savannah doesn't allow a bug's original author to reopen a bug report.  With this report closed, it's no longer showing up on anyone's radar, but I hate to open a new bug report for such a minor thing.  Would someone be willing to reopen this one until it's addressed?

Dave <barx>
Group Member
Sun 10 May 2020 01:28:11 PM UTC, comment #16: 

comment #15:

> commit 3704a6b25e2886fae1491df08a6d6a25f6d4f7ad


This just about nails it.  One slight clarification could be made:

This commit adds the sentence, "The first argument, the inter-word space size, is a minimum; if adjusted..."  The intended reading here is "if [the text is] adjusted...," but given the context of the preceding clause, a reader could reasonably interpret this as "if [the first argument is] adjusted..."

Possible fix 1: replace "if adjusted, it may increase" with "groff may use a wider space in adjusted text," to make it abundantly clear we're talking about adjustments groff does automatically.

Possible fix 2: remove the sentence altogether, since the concept is explained a couple paragraphs later, in: "Additional inter-sentence space is not adjusted, but the inter-word space that always precedes it may be."  The only thing this explanation is missing is an explicit statement that the given word-space size is a minimum; that is, that adjustment is always done by expanding space, never contracting it (however much bug #40963 may wish otherwise).  Changing this sentence's "adjusted" to "expanded" or "widened" should cover this, though.

The issue discussed in comment #14 probably warrants a new bug report.

Dave <barx>
Group Member
Sat 09 May 2020 01:14:03 AM UTC, comment #15: 

Naturally I'd goof the commit ID when making a reference to Rocky and Bullwinkle.


commit 3704a6b25e2886fae1491df08a6d6a25f6d4f7ad
Author: G. Branden Robinson <g.branden.robinson@gmail.com>
Date:   Sat May 9 09:44:47 2020 +1000

    Further improve .ss documentation.

    More strongly distinguish the minimality (adjustability) of inter-word
    space and fixedness (non-adjustability) of inter-sentence space.  This
    was the main point of the problem report, Savannah #54101.

    * doc/groff.texi:
      + Note that negative values are not permitted in either argument to
        .ss.  (Right now the prohibition is enforced by assertion failure
        and core dump; see Savannah #58337.)
      + Add index nodes to help readers find discussion of details of
        spacing between words and sentences.
      + Improve grammar.

    * man/groff.7.man: Make descriptions of \n[.ss] and \n[.sss] more
      self-contained.

    * man/groff_diff.7.man: Sync with applicable change to groff.texi.

    Thanks to Dave Kemper for his feedback.


G. Branden Robinson <gbranden>
Group administrator
Sat 09 May 2020 01:13:30 AM UTC, comment #14: 

comment #12:

> The question I have is, should extra space be distributed in
> keeping with the relative proportion of additional
> intersentence space to interword space?
>
> That's more of a typography question than a code question.
> One of us can take it to the list, I reckon.


Indeed, the answer isn't clear.  According to this ranting but well-documented blog post, professional typography stopped using extra sentence spacing between the 1920s and the 1950s, breaking with two centuries of nearly universal typographic practice.  Ideally, groff would adjust sentence spaces in line with that historical practice, but finding someone familiar with such details may require casting the net wider than the groff email list.

This blog post states, "the aesthetics of how to handle the various width spaces... had complex rules that can be found in many of the manuals cited above," a vagueness that makes sense given that the question of adjusting is beyond its scope.  But it does quote a passage from the 1906 Chicago Manual of Style with some Byzantine (and, to my modern eyes, not entirely comprehensible) rules for adjusting a line, wherein space is added or removed depending on the shapes of the adjacent characters, among other considerations.  This is probably not only inadvisable in modern groff (it concerns a typesetting paradigm where other marks of punctuation also got something other than a standard word space after them), but not even possible, as I don't think groff has any awareness of letter shapes (else why would user intervention be required to avoid a collision between an italic f and a roman end parenthesis?).

But it seems to me that groff's current system is not ideal.  Your example from comment #4 probably illustrates this best, showing that with large adjustments, spaces between words and those between sentences become nigh indistinguishable, surely not the desired outcome when the user has asked for sentence spaces to be two to four times the width of word spaces.

Dave <barx>
Group Member
Sat 09 May 2020 01:09:01 AM UTC, comment #13: 

Fixed in commit 866bc203326528237f08f7debe9061ba8f430e1e.

This time for sure!

G. Branden Robinson <gbranden>
Group administrator
Fri 08 May 2020 11:14:22 PM UTC, comment #12: 

Okay, for the time being I've satisfied myself that you're right.

Here's a sample document that people can play with for their own experiments.


.pl 1v
.ll \V[LL]n
.ss \V[SS1] \V[SS2]
One two.
Three.
.brp
.warn 0
restore\ parity
.brp
.warn
One two. Three.
.brp


So, for instance, here's what I see.


$ for LL in $(seq 16 20); do LL=$LL SS1=12 SS2=12 ./build/test-groff -Tutf8 ./EXPERIMENTS/ss-minimal.groff; done
One two.  Three.
restore parity
One  two. Three.

One  two.  Three.
restore parity
One  two.  Three.

One  two.   Three.
restore parity
One   two.  Three.

One   two.   Three.
restore parity
One   two.   Three.

One   two.    Three.
restore parity
One    two.   Three.


Note that in each pair of "One two. Three", groff has detected a sentence ending in the first, but not the second.

Here's how I interpret it.

In the first pair (LL=15), the line length is precisely enough for the input line with two spaces between the sentences; no adjust is required.  In the second of the pair, that means a space is left over.  groff adjusts from the left and places additional space at the first word break it sees.

In the second pair, LL=16, both items in the pair require adjustment.  The first gets one space, and the second gets two, the latter of which happens in the space between sentences, so the output is identical.

The pattern repeats as the line length increases.

This interpretation is also consistent with my understanding of the code.  There is a function called distribute_spaces() in src/roff/troff/env.cpp, which handles adjustment.  By the time processing reaches that point, groff is working over a list of nodes.  There are several types of these, but the important one here belongs to the class 'space_node'.

If we instrument this function, we see that the additional inter-sentence space has already been coalesced into the interword space:


diff --git a/src/roff/troff/env.cpp b/src/roff/troff/env.cpp
index c11d2dfc..26324b6c 100644
--- a/src/roff/troff/env.cpp
+++ b/src/roff/troff/env.cpp
@@ -2086,6 +2086,8 @@ static void distribute_space(node *n, int nspaces, hunits desired_space,
     if (Ems > spread_limit)
       output_warning(WARN_BREAK, "spreading %1m per space", Ems);
   }
+  error("GBR: distributing %1 units of space across %2 nodes",
+       desired_space.to_units(), nspaces);
   for (node *tem = n; tem; tem = tem->next)
     tem->spread_space(&nspaces, &desired_space);
   if (force_reverse || reverse)


Here's the noise this produces.


troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':6
troff: ./EXPERIMENTS/ss-minimal.groff:6: error: GBR: distributing 0 units of space across 2 nodes
troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':9
troff: ./EXPERIMENTS/ss-minimal.groff:9: error: GBR: distributing 0 units of space across 0 nodes
troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':12
troff: ./EXPERIMENTS/ss-minimal.groff:12: error: GBR: distributing 24 units of space across 2 nodes
One two.  Three.
restore parity
One  two. Three.

troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':6
troff: ./EXPERIMENTS/ss-minimal.groff:6: error: GBR: distributing 24 units of space across 2 nodes
troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':9
troff: ./EXPERIMENTS/ss-minimal.groff:9: error: GBR: distributing 0 units of space across 0 nodes
troff: backtrace: file './EXPERIMENTS/ss-minimal.groff':12
troff: ./EXPERIMENTS/ss-minimal.groff:12: error: GBR: distributing 48 units of space across 2 nodes
One  two.  Three.
restore parity
One  two.  Three.
[...and so forth...]


The question I have is, should extra space be distributed in keeping with the relative proportion of additional intersentence space to interword space?

That's more of a typography question than a code question.  One of us can take it to the list, I reckon.

In the meantime, as you noted, clarifying the documentation to tell the truth should resolve this ticket.  :)

G. Branden Robinson <gbranden>
Group administrator
Fri 08 May 2020 09:27:42 AM UTC, comment #11: 


comment #10:

> The core problem is that when adjusting, groff stretches word spaces and not sentence spaces, yet the documentation states (or at least strongly implies) groff may stretch both.
>
> The original problematic sentence (as documented in comment #0) was "In fill mode, the values specify the minimum distance."  The values in question are the two parameters to .ss, only one of which (in current groff) is actually a minimum distance; the other is a fixed distance.  The cited commit recasts this sentence as "When adjusting, the values specify minimum distances."  While overall a better wording, it retains the inaccuracy of the original.


Thanks.  I'll zero in on this point and take another look.  I got pretty distracted digging into the internals of env.cpp and node.cpp.

G. Branden Robinson <gbranden>
Group administrator
Fri 08 May 2020 09:21:37 AM UTC, comment #10: 

comment #9:

> Fixed, as in documentation rewritten, in 866bc203326528237f08f7debe9061ba8f430e1e.


This commit contains some great improvements to the entire .ss section of the manual.  But it doesn't fix the specific problem cited in this bug report.

The core problem is that when adjusting, groff stretches word spaces and not sentence spaces, yet the documentation states (or at least strongly implies) groff may stretch both.

The original problematic sentence (as documented in comment #0) was "In fill mode, the values specify the minimum distance."  The values in question are the two parameters to .ss, only one of which (in current groff) is actually a minimum distance; the other is a fixed distance.  The cited commit recasts this sentence as "When adjusting, the values specify minimum distances."  While overall a better wording, it retains the inaccuracy of the original.

Dave <barx>
Group Member
Thu 07 May 2020 10:14:42 PM UTC, comment #9: 

Fixed, as in documentation rewritten, in 866bc203326528237f08f7debe9061ba8f430e1e.

I did look into what it would take to get adjustment to be weighted more strongly for the additional inter-sentence space than for ordinary inter-word space, and it is not trivially easy, at least not with my current level of understanding of src/roff/troff.

By the time extra space needs to be distributed, all interword spacing on the line has been converted to generic "space nodes" which don't know whether they're between words or between sentences.

A vague picture in my mind began to form of how I might overcome that (add a flag to the space_node class, or subclass it), but at the same time I realized I don't actually know if it is typographically sound to distribute extra space the way I was planning.

I guess I should bring this to the discussion list.

Let me know if the new documentation is (or is not) an improvement.

Thanks for the report!

G. Branden Robinson <gbranden>
Group administrator
Thu 07 May 2020 02:40:38 AM UTC, comment #8: 


comment #7:

> Dave, I think you're incorrect about inter-sentence spaces being "fixed", not adjusted.  Please try the following with `groff -Tutf8` (or `nroff`, whatever :P ).
>
> From what I can tell, not only does the inter-sentence space get adjusted, it's adjusted before inter-word spaces are.


Wait, no, I'm wrong.  You're right.  Hmm.  I wonder how hard this would be to fix.

G. Branden Robinson <gbranden>
Group administrator
Thu 07 May 2020 02:20:01 AM UTC, comment #7: 

Dave, I think you're incorrect about inter-sentence spaces being "fixed", not adjusted.  Please try the following with `groff -Tutf8` (or `nroff`, whatever :P ).

From what I can tell, not only does the inter-sentence space get adjusted, it's adjusted before inter-word spaces are.

At least for grotty, where things are blissfully easy to measure.


.pl 1v
.\".ll 70n \" both inter-word and inter-sentence spaces adjusted
.\".ll 65n \" only inter-sentence spaces adjusted
.ll 60n \" neither adjusted
.ss 12 48
.tm 12345678901234567890123456789012345678901234567890123456789012345678901234567890
1.\& J.\& Fict.\& Ch.\& Soc.\& 6 (2020), 3\[en]14.
2.\& Better known for other work.


Play around with the commented `.ll` lines to see the results.

I'm not going to close this as invalid because I think you're absolutely right that the documentation is inadequate on this point.  The issue has come up at least once a year on the groff mailing lists, which is a high frequency for our volume of traffic.  I'm got a revamp in progress.

G. Branden Robinson <gbranden>
Group administrator
Thu 07 May 2020 02:08:49 AM UTC, comment #6: 

This issue came up again about a year later.

https://lists.gnu.org/archive/html/groff/2019-07/msg00005.html

G. Branden Robinson <gbranden>
Group administrator
Mon 18 Jun 2018 11:12:36 AM UTC, comment #5: 

The second .ss argument has an effect: the ps file you attached definitely shows a difference between its various values.  This file also further illustrates (probably better than my original example) the problem reported in this bug report: because SENTENCE_SPACE_SIZE spaces are not stretchable, the massive stretching of the WORD_SPACE_SIZE spaces dwarfs the fixed-width SENTENCE_SPACE_SIZE spaces to the point that they're nearly imperceptible.  But if you zoom in, you can tell that the words on different lines are not aligned.

This more strongly makes me think, as I began to ponder in comment #1, that the proper fix is to alter the software to match the documentation rather than vice versa.  It makes little typographic sense to have extra sentence spacing when this amount of space cannot be adjusted for filling.

But again, this decision should probably be made according to historical typesetting practice, and I'm not familiar with the details of how additional sentence space was historically implemented.

Dave <barx>
Group Member
Sat 16 Jun 2018 11:50:27 PM UTC, comment #4: 

Distressingly, I can't get the second argument to .ss to have any effect at all.

See attachments.

(file #44379, file #44381)

G. Branden Robinson <gbranden>
Group administrator
Sat 16 Jun 2018 06:34:28 PM UTC, comment #3: 

Hi, thanks for the followup.

I'm not sure what you're asking.  The discrepancy between what the info file says and what groff actually does exists no matter what any other piece of groff documentation has to say.  This discrepancy is what this bug report is documenting.

The groff_diff(7) passage you cite doesn't address stretching of spaces in fill mode at all, so I'm afraid I'm not following the connection you're trying to make.

Dave <barx>
Group Member
Sat 16 Jun 2018 05:05:11 PM UTC, comment #2: 

Hi Dave,

Do you find the following discussion, from groff_diff(7), more helpful?

".ss m n                                                                

When two arguments are given to the ss request, the second argument gives the sentence space size.  If the second argument is not given, the sentence space size is the same as the word space size.  Like the word space size, the sentence space is in units of one twelfth of the spacewidth parameter for the current font.  Initially both the word space size and the sentence space size are 12.  Contrary to UNIX troff, GNU troff handles this request in nroff mode also; a given value is then rounded down to the nearest multiple of 12.  The sentence space size is used in two circumstances.  If the end of a sentence occurs at the end of a line in fill mode, then both an inter‐word space and a sentence space are added; if two spaces follow the end of a sentence in the middle of a line, then the second space is a sentence space.  Note that the behaviour of UNIX troff is exactly that exhibited by GNU troff if a second argument is never given to the ss request.  In GNU troff, as in UNIX troff, you should always follow a sentence with either a newline or two spaces."

G. Branden Robinson <gbranden>
Group administrator
Wed 13 Jun 2018 02:52:43 PM UTC, comment #1: 

The other possibility is that the documentation is correct, and groff's noncompliance with it is a bug.

Modern typography does not add additional space between sentences.  To emulate that, documents cannot use the default .ss setting; they must explicitly invoke this request and pass 0 as its second parameter.  Groff should (and seems to) treat this 0 as fixed rather than stretchable.

In traditional typography, I don't know whether the extra sentence space was of fixed width, or adjusted proportionately with the word space.  But when using a nonzero SENTENCE_SPACE_SIZE, groff should probably follow historical practice in this regard.

Dave <barx>
Group Member
Tue 12 Jun 2018 06:46:21 AM UTC, original submission:  

The documentation's .ss section states, "Initially both the WORD_SPACE_SIZE and SENTENCE_SPACE_SIZE are 12.  In fill mode, the values specify the minimum distance."

The latter sentence does not seem to be entirely accurate, in that a simple test shows that WORD_SPACE_SIZE specifies a minimum distance while SENTENCE_SPACE_SIZE specifies a fixed distance.  That is, if they were both minimum distances, they should both be stretched when filling text.  But this does not happen.  The input file:

Sentence one.
Sentence two.
Sentence three.\p

produces very large spaces between words, and that same size plus only a normal space width between sentences.

Although sentence spaces do not stretch in this simple example, are there cases where they do?

If not, the sentence claiming so in the documentation should be corrected.  (Possible wording: "In fill mode, WORD_SPACE_SIZE specifies the minimum distance.  The additional space added between sentences is always exactly SENTENCE_SPACE_SIZE.")

If there are, it might be useful for the documentation to specify under what circumstances this happens.

Dave <barx>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44379:  54101.trf added by gbranden (271B - application/octet-stream)
file #44380:  54101.png added by gbranden (4KiB - image/png)
file #44381:  54101.ps added by gbranden (6KiB - application/postscript)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by gbranden (Posted a comment)
  • -email is unavailable- added by barx (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 22 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-06-04 gbranden StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2020-06-01 gbranden StatusIn Progress Need Info
    2020-05-27 gbranden Item GroupIncorrect behaviour Documentation
        StatusFixed In Progress
        Open/ClosedClosed Open
    2020-05-09 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
        Planned ReleaseNone 1.23.0
    2020-05-08 gbranden StatusFixed In Progress
        Open/ClosedClosed Open
        Planned Release1.23.0 None
    2020-05-07 gbranden Planned ReleaseNone 1.23.0
    2020-05-07 gbranden StatusIn Progress Fixed
        Open/ClosedOpen Closed
    2020-05-07 gbranden StatusConfirmed In Progress
        Assigned toNone gbranden
    2018-06-16 gbranden Attached File- Added 54101.trf, #44379
        Attached File- Added 54101.png, #44380
        Attached File- Added 54101.ps, #44381
        Item GroupDocumentation Incorrect behaviour
        StatusNone Confirmed

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code