bugmake - Bugs: bug #57751, Improve POSIX support for SCCS

 
 

bug #57751: Improve POSIX support for SCCS

Submitter:  Bogdan Barbu <love4boobies>
Submitted:  Thu 06 Feb 2020 09:10:00 AM UTC
   
 
Severity:  3 - Normal Item Group:  Enhancement
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  None Operating System:  None
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 08 Nov 2020 04:37:43 PM UTC, comment #17: 

That is not dispositive because it tells you where the macro is evaluated but doesn't tell you where an internal macro is assigned.

Where it is assigned is the key question, not where it is evaluated.

For a rule like this:

    foo.o : $@.c ;

clearly the $@ is evaluated when the target line is read in, as POSIX says, but POSIX also says that internal macros "can be used in target and inference rules", so there is absolutely nothing in the standard saying that make should not work like this:

  * Parse/evaluate the target
  * Assign $@ to the target name
  * Parse/evaluate the prerequisites

That is why I asked for the text to be clarified saying that internal macros are not required to be assigned except when evaluating commands.

Paul D. Smith <psmith>
Group administrator
Sun 08 Nov 2020 03:14:57 PM UTC, comment #16: 

Just in case that helps, the syntax of the make language is not written down cleanly as a compact syntax description in the POSIX standard. You need to read a lot of the general text. My explanations about the expansions of macros at parse time are in the macro section of the standard.

See this quote from the standard:

Macros can appear anywhere in the makefile. Macro expansions using the forms $(string1) or ${string1} shall be replaced by string2, as follows:

-   Macros in target lines shall be evaluated when the target line is read.

.   Macros in makefile command lines shall be evaluated when the command is executed.

.   Macros in the string before the <equals-sign> in a macro definition shall be evaluated when the macro assignment is made.

Jörg Schilling <schily>
Sun 08 Nov 2020 01:57:18 PM UTC, comment #15: 

RE: what POSIX says about internal macros and where they are valid: I agree with Jörg that no POSIX-based make supports $@ in prerequisite lists.  However I agree with Bruce that the standard is at best ambiguous about this and I've filed:

https://www.austingroupbugs.net/view.php?id=1420

to request clarification.

RE: handling $$@ : it's not true that GNU make doesn't support this: GNU make DOES support this syntax but you have to enable it; see:

https://www.gnu.org/software/make/manual/html_node/Secondary-Expansion.html

There has been thought about making it enabled by default but it hasn't happened yet.

Paul D. Smith <psmith>
Group administrator
Sun 08 Nov 2020 03:05:47 AM UTC, comment #14: 

Please read my previous answer...

Any dependency line in e.g. the form:

$(TGT): $(DEPS)

is completely expanded at the time the parser reads the makefile, so in this example, $(TGT) and $(DEPS) already need to have a value at the time, the dependency line is read/parsed. So POSIX requires to expand $@ to an empty string and bmake is clearly in conflict with POSIX.

Special macros only have values at the time, when the update process is updating a target.

What you like to get is not make syntax and if you believe that the POSIX standard supports your wish, then the POSIX text should be fixed to become less ambiguous.

BTW: There is a SunPro Make enhancement (since 1986) that would need to use:

foobar : $$@.blurfl

with the behavior you like but this is not covered by POSIX nor by other make implementations. well.... except for bmake, that also seems to support it.

Unlike the syntax you reported for bmake, the $$@ syntax is a clean enhancement to UNIX / POSIX make that is not in conflict with the standard.

See the man page:

 $@    The name of the  current  target.  This  is  the  only
       dynamic  macro whose value is strictly determined when
       used in a dependency list. (In which case it takes the
       form $$@.)

Jörg Schilling <schily>
Sun 08 Nov 2020 01:56:37 AM UTC, comment #13: 


> that functionality couldn't be used here else the $@ would expand to the target, which in this case is .SCCS_GET so it's not what you want.


Good point.

> I don't believe that this (allowing $@ in prerequisites) is "traditional make" behavior.


I've seen it used, e.g. where there are many executables, each built from a single source file.  So, for example:


cat date echo ls pwd rm sleep sync test : $@.o


suffices to specify (with default rules) everything needed to build those executables.  Otherwise, you'd have to have many separate dependency lines, one each for every executable (or rely on default "suffix" rules with a null suffix (and that wouldn't provide for any grouping such as using a specific recipe with a special -l linker option attached to the dependency line, which can be done in the above case)).  Obviously, that only works for variants of make that support $@ (and/or its variants, like $*, $(@D), etc.) in the prerequisites.  There are portable ways to deal with cases like the above, but they're not as elegant.  There are cases where $(@D) and/or $(@F) in prerequisites is/are very useful, and the workarounds are tedious.

The POSIX (X/Open) specification appears to support this; the "Target Rules" section (2018 edition) applies where prerequisites are specified (as distinct from "Inference Rules", which have only a single target and no prerequisites).  And the "Internal Macros" section specifies the macros that can be used in both target and inference rules, and clarifies "prerequisite".  It specifies evaluation of $@ and $? for target rules, and $% for target rules when the target is an archive library member.  $* is required to be evaluated for inference rules, but evaluation is not prohibited for target rules.  $< is specified for inference rules only.  There is nothing in the clarification of "prerequisite" or elsewhere in the specification that excludes evaluation of those internal macros in prerequisites in target rules.  For that matter, there's only common sense (nothing explicit in the spec.) that precludes evaluation in the target portion of a target rule (likewise, $? would make no sense in a prerequisite).

The standard could be clearer on evaluation in prerequisites, as there is a definite, well-documented (e.g. the previously-mentioned O'Reilly book by Talbott and Oram) difference in historical (and, obviously, present) implementations.  There's no mention in the standard's APPLICATION USAGE or RATIONALE sections.

There's yet another variant with two dollar signs (which is mentioned in a note in the POSIX X/Open spec.) that is specifically not included.

> Is it really the case that you define the COMMAND to run SCCS get operations as PREREQUISITES to the special target?
>
> That's.... horrible and disgusting.


I'm guessing that you may have indeed found an error in the specification, but I'm not certain of that.

Many "special" targets have horrible and disgusting syntax (which often causes portability issues).  For example, NetBSD make (a.k.a. bmake) has a "special" target for specifying the shell to use for recipes:


     .SHELL   Sets the shell that bmake will use to execute commands.  The
              sources are a set of field=value pairs.

              name        This is the minimal specification, used to select
                          one of the built-in shell specs; sh, ksh, and csh.

              path        Specifies the path to the shell.

[...]
              Example:

              .SHELL: name=ksh path=/bin/ksh hasErrCtl=true \
                      check="set -e" ignore="set +e" \
                      echo="set -v" quiet="set +v" filter="set +v" \
                      echoFlag=v errFlag=e newline="'\n'"


That looks like a target with a bunch of very peculiar prerequisites; it can cause trouble for other make variants if it's the first (default) target, and some other make variants (particularly dmake) complain about the syntax.

Bruce Lilly <blilly>
Sat 07 Nov 2020 09:39:15 PM UTC, comment #12: 

Re: comment #8: you are mistaken.

gmake has some deviations from POSIX and a classical UNIX make but these deviations are much less important than the deviations found in bmake.

The BSD make program is not related to any UNIX make program, it was derived from a program called pmake.

$ cat Makefile
foobar : $@.blurfl
$ make
make: Fatal error: Don't know how to make target `.blurfl'
$ dmake
dmake: defaulting to parallel mode.
make: Fatal error: Don't know how to make target `.blurfl'
$ smake
smake: Can't find any source for '.blurfl'.
smake: Couldn't make 'foobar'.
$ gmake
gmake: * No rule to make target `.blurfl', needed by `foobar'.  Stop.
$ bmake
bmake: don't know how to make foobar.blurfl. Stop

BTW: Make macros to the right of a colon are expanded by the parser already and at that time, $@ is undefined.

I hope this explains the correct behavior of various make implementations.

Jörg Schilling <schily>
Sat 07 Nov 2020 08:29:27 PM UTC, comment #11: 

I filed https://austingroupbugs.net/view.php?id=1419 to address this typo.

Paul D. Smith <psmith>
Group administrator
Sat 07 Nov 2020 08:10:07 PM UTC, comment #10: 

I forgot my main point, which was: whether or not $@ is "traditionally" allowed to appear in the prerequisites list makes no difference in this situation.  Even if it were, that functionality couldn't be used here else the $@ would expand to the target, which in this case is .SCCS_GET so it's not what you want.

It's clear that if the command is provided as prerequisites to the target then the entire thing must be handled as a special case.

However, looking at the spec this is all moot because, indeed, the rule quoted below is a typo in the spec.

The text of the spec makes very clear that it's the commands of this special target that specify the rule:

> The application shall ensure that this special target is specified without prerequisites. If this special target is included in a makefile, the commands specified with this target shall replace the default commands associated with this special target (see Default Rules). The commands specified with this target are used to get all SCCS files that are not found in the current directory.

Paul D. Smith <psmith>
Group administrator
Sat 07 Nov 2020 07:50:48 PM UTC, comment #9: 

I don't believe that this (allowing $@ in prerequisites) is "traditional make" behavior.

However, I admit to being surprised when I went to look at the POSIX spec for make that it is very wishy-washy and unclear as to exactly where the internal macros are available.  It just says they "can be used in target and inference rules".  I would have expected it to say that specifically that they can be used in commands.

So, maybe I'm wrong.

In any event, I assumed that this is a typo.  I thought that the rule was intended to be:


.SCCS_GET: ; sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@


Note the added semicolon.

Is it really the case that you define the COMMAND to run SCCS get operations as PREREQUISITES to the special target?

That's.... horrible and disgusting.

Paul D. Smith <psmith>
Group administrator
Sat 07 Nov 2020 07:16:01 PM UTC, comment #8: 

A further note on:

> .SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@


in the original report.  That's likely to be a problem for gmake as it stands, separately from any SCCS-specific issues.

Gmake has a long-standing difference from other make variants in handling of $@ and variants of it (e.g. $(@D)) as prerequisites (as distinct from use in recipes).  Here's a greatly simplified example:

$ cat makefile.dollar-at
foobar : $@.blurfl
$ uname -sr
OpenBSD 6.8
$ type -a make bmake gmake smake dmake
make is a tracked alias for /usr/bin/make
ksh93: whence: bmake: not found
gmake is a tracked alias for /usr/local/bin/gmake
ksh93: whence: smake: not found
ksh93: whence: dmake: not found
$ make -f makefile.dollar-at
make: don't know how to make foobar.blurfl (prerequisite of: foobar)
Stop in /data/projects/computing/make/tests
$ gmake -f makefile.dollar-at
gmake: * No rule to make target '.blurfl', needed by 'foobar'.  Stop.
$

So:
1. Like the quoted line, $@ appears in what syntactically looks like a prerequisite on a single-colon dependency line
2. Two make variants tested here: OpenBSD native make and gmake
3. Note that the OpenBSD native make expands $@ in the prerequisite, which is traditional make behavior
4. Note that gmake neither expands nor copies $@; it is effectively treated as if it were whitespace! That difference between gmake and other make variants is fairly well known and documented in a few places (e.g. the old O'Reilly make books).

So handling that specific case verbatim is likely to first require handling $@ (and perhaps variants) on dependency lines in general in gmake.

Bruce Lilly <blilly>
Wed 04 Nov 2020 12:15:08 AM UTC, comment #7: 

Just a note to the XSI "enhancements" to POSIX....

Implementing all XSI enhancements is required in order to get the permission to call a platform "UNIX" compatible and thus to use the UNIX trademark.

A platform that implements support for XSI is a full blown POSIX system and if XSI is not or not completely implemented, this is typical choice for tiny platforms, e.g. embedded systems.

BTW: SunPro Make is POSIX+XSI certified and gmake is not faster than SunPro Make. Gmake even needs a tiny bit more CPU time than SunPro Make. So I am not sure whether implementing SCCS support really creates a performance problem.

Jörg Schilling <schily>
Thu 17 Sep 2020 09:50:26 PM UTC, comment #6: 

comment #3:

> I can't remember the last time I saw a system where SCCS or RCS were even installed.


SCCS is alive and well; it was open-sourced by Sun Microsystems as part of Open Solaris and is still being updated (slowly, because it's stable), has mailing lists, etc.  I use SCCS extensively.  But this isn't the time or place for advocacy; if you want to know why, here or off-list, ask me.

Some of the finer points about SCCS-related rules:
1. the tilde hack exists because make originally worked on suffixes whereas SCCS uses prefixes ("s." and related ones).  Obviously gmake has a more general pattern-based rules system; one way to achieve compatibility would be an internal translation of a tilde suffix rule to an equivalent pattern.  The most general way would be to translate a '~' suffix (in suffix rules) to a pattern "s.%" or something like that.  Note that some programs sometimes generate files with a literal tilde at the end of the filename (typically for backups), which is distinct from the make suffix rule tilde hack, which doesn't involve any tilde in the actual file name.  Anyway, as shown below, it appears to work in gmake, at least in version 4.2.1.
2. When an SCCS-cognizant version of make is used, it can retrieve the makefile itself (from s.makefile or s.Makefile or SCCS/s.makefile, etc.) with no special action on the part of the user; 'make' suffices.  Gmake already does this (including s.GNUmakefile, etc.).
3. Sometimes SCCS source files exist in the same directory as the target file, sometimes in an "SCCS" subdirectory.  Gmake appears to handle both cases adequately.
4. SCCS-cognizant versions of make typically have several built-in rules primarily aimed at C source programming (e.g. https://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/make/pwbrules.c [*]).  Users interested in version control and automation of other tasks (documentation, audio or video production, etc.) or in other programming languages may have additional defined suffixes and suffix rules.  For compatibility, at least the basic yacc/lex/.c/.h/.sh/.s rules should probably be supported.
5. Much of this already exists in gmake:

$ gmake --version ; gmake hw.o
GNU Make 4.2.1
Built for x86_64-suse-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
get   SCCS/s.GNUmakefile
Clock may be set wrong! (co11)
1.1
2 lines
No id keywords (cm7)
get   SCCS/s.makefile
Clock may be set wrong! (co11)
1.1
2 lines
No id keywords (cm7)
get   s.hw.c
1.1
6 lines
No id keywords (cm7)
cc    -c -o hw.o hw.c
rm hw.c


Neither of the makefiles have any specific rules for hw.[co] or explicit suffix rules for SCCS; everything that gmake did was based on internal rules.

6. Gmake appears to support the tilde hack reasonably well for additional user-defined suffixes:

$ ls -l *.pic*
-rwxrwxrwx 1 root root 281 Sep 17 16:11 s.foo.pic
$ gmake foo.pic
get   SCCS/s.GNUmakefile
1.2
9 lines
No id keywords (cm7)
get   SCCS/s.makefile
Clock may be set wrong! (co11)
1.1
2 lines
No id keywords (cm7)
get   s.foo.pic
1.1
6 lines
No id keywords (cm7)
$ ls -l *.pic*
-rwxrwxrwx 1 root root  99 Sep 17 16:11 foo.pic
-rwxrwxrwx 1 root root 281 Sep 17 16:11 s.foo.pic
$ cat SCCS/GNUmakefile
# add .pic and .pic~ suffixes
.SUFFIXES :     .pic .pic~

# SCCS get
GET = get

# rule to get
.pic~.pic :
        if test -n "$(GET)" ; then $(GET) s.$@ ; fi


Note: nothing there is gmake-specific; there are no files with a literal tilde, etc.  Gmake gets the desired file using get. Note also that the suffix rule is generic enough that if you have defined suffixes .foo and .foo~, it will work for them also by adding the concatenated suffix target .foo~.foo to the suffix rule target list.  So a more complete suffix rule might include targets .cpp~.cpp .tbl~.tbl .grap~.grap .1~.1 etc.

Bogdan wrote:

> Also, suffixes such as ".foo~" (meaning ".foo" files prefixed by a "s.") do not seem to be supported.


Do you have a specific example (note that it appears to work fine for .pic~)?  What specific suffix seems to be the problem, what suffixes and suffix rules are defined, are the relevant macros defined, which version of gmake, etc.?

So, I'm not seeing any general SCCS-related issues here, including some of the subtleties.

> .SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@


I believe that it's possible to get desired functionality using suffixes and suffix rules w/o such 'special' targets; see the examples above.  It might well be the case that gmake doesn't work the way the POSIX committee would like to see it work, but that's a different matter from solving real-world problems.  If the goal is strict conformity to the standards documents, that's one thing; if there's a specific real problem that you're trying to solve in a portable way, there may be a better way to address it -- certainly the installed base of pre-committee software isn't going to magically start working the way some document describes if it didn't work that way before; and both make and SCCS have been around for decades.  On the other hand, many existing make implementations, including gmake, work well with SCCS using portable, well-documented mechanisms such as .SUFFIXES and suffix rules.

* There should be enough information there (in concise form) to implement any missing support for specific recipes.  I don't know who "my" in the comments in the source file refer to; it's not me!

Bruce Lilly <blilly>
Wed 01 Apr 2020 01:28:37 PM UTC, comment #5: 

The FSF guidelines consider any changes more than 15 to 20 lines to be "legally significant" and require paperwork.  My suspicion is that the changes you're talking about would be more than that, but you can try it and see.  Maybe I'm overestimating.

You can certain disclaim copyright rather than assigning copyright to the FSF; some people don't want to assign copyright for various reasons.  However, "public domain" as a legal concept doesn't exist in many countries so it's not so easy, legally, as just writing a message at the top of the changes.  You still need to fill out a form :(

The FSF has different forms for assigning copyright and disclaiming copyright.  Either one is fine.  They also have forms for assigning all future changes so you don't have to continue to file paperwork if you plan to continue to contribute.

For many countries this can be done via email and there's no actual paper involved.  However if by "online" you mean a web form or something then no, we don't have that.

Paul D. Smith <psmith>
Group administrator
Thu 20 Feb 2020 06:31:00 PM UTC, comment #4: 

Ugh, paperwork? Is this something that has to be done everytime people send patches? I mean it's literally just a couple of small additions:

1. If foo is a dependecy and SCCS/s.foo exists, then make issues the commands specified in .SCCS_GET, whose default is this:

.SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@

but makefiles can provide their own .SCCS_GET instead.

2. A syntax for suffixes like .c~ which really means s.%.c.

If you don't have time to do it can't I just release the patches into public domain and have you incorporate them or something so I don't have to sign papers and send them to other countries? Sounds like a hassle, unless there's some simple procedure in place that I can waste a few minutes on online.

Bogdan Barbu <love4boobies>
Mon 17 Feb 2020 04:15:00 PM UTC, comment #3: 

Sure.  Be aware that as a GNU project, changes of that order will likely require copyright assignment paperwork before they can be included.

> Seems reasonable. A different approach would be to add a command-line option to disable the built-in support for these tools so people can just accelerate the performance of make in a transparent way to any makefile relying on the current behavior.


I don't think that a command line argument is the right way to go.  It should be up to the makefile, and defined there, whether these rules are omitted, not up to the user of the makefile.  There is precedent of course (the -r option for example) but I still don't think it's the right design.

The thing is that makefiles can already get rid of these rules if they want to; we could make it simpler but it's just a matter of magnitude.  What I would prefer is to have these rules disabled by default, without users having to do anything, since virtually no one wants them.  I can't remember the last time I saw a system where SCCS or RCS were even installed.  Even the BSD systems are using CVS now IIRC.  Then the people who do need this support can add it back: they likely already know how to do it.

Of course this would be a backward-compatibility break and that's always concerning.  If I discover there's any non-trivial audience relying on these built-in rules I probably wouldn't do it.

Paul D. Smith <psmith>
Group administrator
Mon 17 Feb 2020 10:08:02 AM UTC, comment #2: 


> I don't have any objection to including these features in GNU make if someone contributes them, but I have little enthusiasm for creating these myself since I doubt that SCCS is much more than an anachronism these days.  Certainly no one has ever complained about this before in the history of GNU make (that I'm aware of).  I'd prefer to spend my (very limited) time on more useful features and fixes.


I'd be willing to write a patch to add support for these features but I am not very familiar with GNU Make's code base yet so I'll probably have a few questions along the way to get my way around it faster and also to make sure I don't accidentally break anything. What would be the appropriate place to ask such questions?

> In fact, I've been toying with the idea of omitting the built-in rules for RCS and SCCS completely, unless the .POSIX: special target is given (I realize RCS is not part of the POSIX standard), because adding extra match-anything rules like these seriously impacts the performance of make, and not many makefiles think to delete these rules which virtually no one ever uses/needs anymore.  It's just a big waste of cycles.


Seems reasonable. A different approach would be to add a command-line option to disable the built-in support for these tools so people can just accelerate the performance of make in a transparent way to any makefile relying on the current behavior.

Bogdan Barbu <love4boobies>
Sun 16 Feb 2020 06:25:06 PM UTC, comment #1: 

Well, SCCS support in make is actually part of the XSI extension to POSIX, not base POSIX.

I don't have any objection to including these features in GNU make if someone contributes them, but I have little enthusiasm for creating these myself since I doubt that SCCS is much more than an anachronism these days.  Certainly no one has ever complained about this before in the history of GNU make (that I'm aware of).  I'd prefer to spend my (very limited) time on more useful features and fixes.

In fact, I've been toying with the idea of omitting the built-in rules for RCS and SCCS completely, unless the .POSIX: special target is given (I realize RCS is not part of the POSIX standard), because adding extra match-anything rules like these seriously impacts the performance of make, and not many makefiles think to delete these rules which virtually no one ever uses/needs anymore.  It's just a big waste of cycles.

Paul D. Smith <psmith>
Group administrator
Thu 06 Feb 2020 09:10:00 AM UTC, original submission:  

According to POSIX, makefiles should be able to replace the special rule .SCCS_GET in order to specify how to get SCCS files that do not exist in the current directory. The default is:

.SCCS_GET: sccs $(SCCSFLAGS) get $(SCCSGETFLAGS) $@

Also, suffixes such as ".foo~" (meaning ".foo" files prefixed by a "s.") do not seem to be supported. I am aware that these can be done in other ways with GNU Make, but for portability's sake, they should be there.

Bogdan Barbu <love4boobies>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by schily (Posted a comment)
  • -email is unavailable- added by blilly (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by love4boobies (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code