bugmake - Bugs: bug #47399, include sometimes fails for built...

 
 

bug #47399: include sometimes fails for built files

Submitter:  None
Submitted:  Sat 12 Mar 2016 11:51:38 PM UTC
   
 
Severity:  3 - Normal Item Group:  Enhancement
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.1 Operating System:  None
Fixed Release:  None Triage Status:  Medium Effort
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 17 Sep 2020 08:03:29 PM UTC, comment #8: 

Just to clarify, it's not support for the include keyword we're discussing here.  I have no problem believing that various make implementations provided some kind of include facility before GNU make even existed: it seems an obviously useful extension.

Here we're talking about the "rebuild included makefiles" feature specifically.

Perhaps one or all of these earlier incarnations DID provide for rebuilding included makefiles, I don't know.  But we'd need to look more deeply than checking to see if the grammar provides for an "include" keyword to find out.

But, that's certainly interesting!  I may pull some of those older sources down just for historical purposes.  Thanks for digging these up!

Paul D. Smith <psmith>
Group administrator
Thu 17 Sep 2020 05:45:38 PM UTC, comment #7: 

[comment #4:]

> Well, we'll just have to agree to disagree over who implemented it "incompatibly".  I've seen no evidence that any other implementation of make provided this feature before GNU make (Roland added it in September 1988).


This is a somewhat minor point, but 4 years and a global pandemic later, it's bubbled up the list of things to look into, so I'm adding some information here for reference by anybody who cares about makefile inclusion:

https://minnie.tuhs.org/cgi-bin/utree.pl has a vast repository of UNIX sources, including various versions of make, notably including the V7 version of Stu Feldman's original (at https://minnie.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src/cmd/make) [with a few tweaks, it can be built...]

The Ultrix version of gram.y (https://minnie.tuhs.org/cgi-bin/utree.pl?file=Ultrix-3.1/src/cmd/make/gram.y) certainly includes support for "include" and has copyright dates of 1984-1986, SCCS id 3.0 and date of "4/21/86", i.e. 1986-04-21. This version appears to have been derived from System V (the executables are named starting with "s5make").  That certainly pre-dates 1988.

The System III version of gram.y (https://minnie.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/make/gram.y) also has support for "include"; there's no date in the file itself (SCCS version id 3.6), but it's supposedly dated 1980-04-27, which seems to indicate that "include" support was added fairly early (the V7 version w/o it is dated 1979), at least in versions originating from USG/USL versions of UNIX.

On the other hand, it didn't find its way into Berkeley-derived systems until much later; it's not in the BSD 4.3-Reno source (https://minnie.tuhs.org/cgi-bin/utree.pl?file=4.3BSD-Reno/src/old/make/gram.y) with Berkeley SCCS id 4.2 dated 1987.

Side note: clearly the files followed diverging modification paths with few updates being merged back, as can be seen by the disparate SCCS ids.

Bruce Lilly <blilly>
Tue 29 Mar 2016 12:16:49 AM UTC, comment #6: 

Regarding licensing: I can't say for sure what CDDL-covered code might have been linked with GNU make on OpenIndiana.  However it's most likely simply things such as the C runtime library, since that's about all that GNU make relies on.

In that case, there's no problem because this falls under the "System Library" exception in the GPL, both versions 2 and 3.  So the licensing does not need to be compatible.

Paul D. Smith <psmith>
Group administrator
Mon 28 Mar 2016 11:10:05 PM UTC, comment #5: 


> Also it's not clear what you mean when you say "make"


Tested versions are listed in the comments at the top of the test makefile.
Note that on OpenIndiana, /usr/bin/make, /usr/bin/dmake, and /usr/xpg4/bin/make
all use the same executable, but with slightly varying semantics (and they
all work fine with the test makefile).


> Wouldn't it be significantly less work to use a single implementation of make which is portable across systems;


That would most likely have to be dmake, which "just works", has been ported to many systems,
is widely available (e.g. via pkgsrc), and does not carry onerous licensing restrictions.
But it is impractical to force such a requirement ("you must use such-and-such an implementation
of make"), as there are systems where administrative policy and/or legal considerations would
preclude such.

IANAL, but although there is a version of gmake on OpenIndiana, its legal status is
unclear (at least to me):

# uname -srvio
SunOS 5.11 illumos-bc0e913 i86pc Solaris
# gmake --version
GNU Make 4.1
Built for i386-pc-solaris2.11
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
# what $(whence gmake)
/bin/gmake:
        SunOS 5.11 illumos-33e8313 November 2015

Clearly, GPL'd gmake has been linked with CDDL illumos libraries (the linked executable
contains illumos SCCS ids), even though the FSF says that such linking is not permitted.
Their exact wording is stronger, and can be found at http://www.gnu.org/licenses/license-list.html#CDDL )

Bruce Lilly <blilly>
Sun 13 Mar 2016 03:10:38 PM UTC, comment #4: 

Well, we'll just have to agree to disagree over who implemented it "incompatibly".  I've seen no evidence that any other implementation of make provided this feature before GNU make (Roland added it in September 1988).

Also it's not clear what you mean when you say "make"; the original make implemented by Feldman didn't have this feature, of course.  And the POSIX definition of "make" doesn't mention it either (until recently POSIX didn't require "include").  When I say "make" I clearly mean "GNU make".  I don't know what variation of make you are referring to when you say "make".

I admire your attempt to create a single set of makefiles for a complex project which could work across such disparate implementations of make; they have barely more than the basic makefile structure in common.  It's an impressive feat, but it must be a huge amount of frustrating effort.  Wouldn't it be significantly less work to use a single implementation of make which is portable across systems; it's much simpler to build a portable make one time than to maintain makefiles that squeak through all the portability cracks in different versions of make.  Or use a "meta language", which is what cmake and automake do.

> Anyway, I look forward to seeing the enhancement. Thanks.


Honestly this is not high on my personal TODO list.  The current makefile update behavior has been present for 27+ years and it's worked fine.  However maybe someone will find it an interesting project.

Paul D. Smith <psmith>
Group administrator
Sun 13 Mar 2016 12:47:01 PM UTC, comment #3: 


> This is the way make has always worked


See the notes in the sample makefile: make and dmake both work fine; gmake fails.  It might be the case that gmake has always had this issue; that is an incompatibility with make.

> you can change your "include baz" line to "-include" or "sinclude"


No, I cannot; that would cause the makefile to be non-portable, i.e. it would no longer work with make, dmake, bmake,...

> you can move the "include baz" line into the "bar" makefile


Maintenance issues, see the notes in the attached makefile.
Note also that the sample makefile is just that; a brief example that demonstrates the issue.  Typical cases such as the one where
this gmake incompatibility with make occur involves more than a dozen files, several of which are built, with include directives
in one file except for the include directive in the main makefile, which includes the file containing the other include directives (intentionally, for ease of maintenance).

> [g]make's if conditionals [...] filtering MAKEFILE_LIST


More non-portable stuff.

Anyway, I look forward to seeing the enhancement. Thanks.

Bruce Lilly <blilly>
Sun 13 Mar 2016 01:28:59 AM UTC, comment #2: 

Actually I'm going to reopen this as an enhancement.

One thing GNU make could do would be, if some makefile cannot be included and cannot be built, if any makefile has already been built then rather than failing we'll try to re-exec.

I think this shouldn't be a backward-incompatibility.

Paul D. Smith <psmith>
Group administrator
Sun 13 Mar 2016 01:25:30 AM UTC, comment #1: 

This isn't a bug: this is how make is documented to work with building included makefiles; from the make manual "How Makefiles are Remade":

After all makefiles have been checked, if any have actually been changed, make starts with a clean slate and reads all the makefiles over again.

So, GNU make doesn't stop after the first included file, try to make it, and if it succeeds re-exec itself right away.  Instead it will try to make ALL the included makefiles it knows about, then afterwards if any of them have been remade it will re-exec itself.

Since when make first reads the makefile there is no file "baz" and no target that can build "baz", it fails.  This is the way make has always worked, and always been intended to work; this feature was already present in GNU make in 1992, when the first set of source code was added to source control.

You have various choices: first, you can change your "include baz" line to "-include" or "sinclude" which will prevent GNU make from failing if the "baz" makefile doesn't exist.  It will then remake "bar", ignore the fact that "baz" doesn't exist, re-exec, find a rule to build "baz", build that, re-exec, and complete as expected.

Alternatively, you can move the "include baz" line into the "bar" makefile so that make cannot see it until after "bar" has been created and make has re-exec'd to read it.  Or you can do a similar thing without moving "include bar" but using make's if conditionals to make the "include baz" invisible unless "bar" has been included: testing some variable defined only in "bar" for example, or even just filtering MAKEFILE_LIST to see if "bar" is present in it.

FWIW, the correct title of Orwell's work is "Nineteen Eighty-Four", not "Nineteen Hundred Eight Four".  Although I have no idea if that would be accepted or if the "it's a date" is supposed to be a hint that it wants the numeric "1984".

Paul D. Smith <psmith>
Group administrator
Sat 12 Mar 2016 11:51:38 PM UTC, original submission:  

See attached makefile for success/failure test results and other relevant notes.

For files which are built by recipes in an included makefile, GNU make fails catastrophically, even if all dependencies are satisfied and rules to build the file to be included are defined.
This differs from the behavior of other implementations of make (as noted in the attached makefile) as well as for the similar case of use of an internal rule to generate the file to be included (e.g. using SCCS).

On an unrelated note: the supposedly-anti-spam filter annoyingly does not accept the correct answer to "Please enter the title of George Orwell's famous dystopian book (it's a date):", which is "Nineteen Hundred Eighty Four"

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #36615:  makefile.gmake-bug added by None (1KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by blilly (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-03-13 psmith Open/ClosedClosed Open
        Triage StatusNone Medium Effort
    2016-03-13 psmith Item GroupBug Enhancement
        StatusNot A Bug None
    2016-03-13 psmith StatusNone Not A Bug
        Open/ClosedOpen Closed
    2016-03-12 None Attached File- Added makefile.gmake-bug, #36615

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code