bugmake - Bugs: bug #47880, Allow updates to .INCLUDE_DIRS to...

 
 

bug #47880: Allow updates to .INCLUDE_DIRS to change search path

Submitter:  None
Submitted:  Mon 09 May 2016 04:22:45 PM UTC
   
 
Severity:  3 - Normal Item Group:  Enhancement
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  4.2 Operating System:  Any
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 07 May 2021 01:14:50 PM UTC, comment #14: 


> when a user writes a makefile he should not have to know that
>
> depfiles:=$(obj:.o=.d)
> include $(depfiles)
>
> is less efficient than
>
> depfiles:=$(obj:.o=.d)
> depfiles:=$(addprefix $(CURDIR)/,$(depfiles))
> include $(depfiles)


Perhaps he should, specifically since the issue has been raised.

(Incidentally, $(abspath) achieves largely the same thing as prefixing CURDIR.)

Anyway, the various suggestions are not mutually exclusive:

  • Add -I- to allow the invoker to reset the search path.  (This is what I meant by a "switch analogous to -r/-R", in case I wasn't clear.)
  • Add includelocal to allow an author to indicate when he definitely doesn't want to use the search path for specific files.  (He could use abspath/CURDIR, but if the included files are also targets, those rules have to be changed too.)
  • Allow effective modifications to .INCLUDE_DIRS.


Steven Simpson <simpsons>
Mon 26 Apr 2021 08:41:20 PM UTC, comment #13: 


> > My opinion is that the users should not have to know this detail.


> I didn't realize you were wearing your 'user' hat!


My bad. By "users" here (and in other places) i meant make users, people who write makefiles. In other words, when a user writes a makefile he should not have to know that

depfiles:=$(obj:.o=.d)
include $(depfiles)

is less efficient than

depfiles:=$(obj:.o=.d)
depfiles:=$(addprefix $(CURDIR)/,$(depfiles))
include $(depfiles)

Dmitry Goncharov <dgoncharov>
Mon 26 Apr 2021 06:51:30 PM UTC, comment #12: 


> > include $(CURDIR)/foo.mk
>
> This indeed suppresses the lookup.
>
> My opinion is that the users should not have to know this detail.


I didn't realize you were wearing your 'user' hat!  To me, this is a problem ideally for the author to solve.  However, I'm not against an option to either reset the include path or disable its initial population, allowing a user to work around the issue.

> Also, most (as far as i can tell) existing makefiles do not do this.


(Indeed, I only found out about CURDIR yesterday!  I had thought prefixing ./ was enough, and I shall be modifying my makefiles accordingly.)

> > this might be better considered a question of disabling the search path on a per-include basis
>
> Is there a need for such flexibility?


Yes, from the author's perspective.  Since you report that using $(CURDIR) allows the author to resolve the issue, my suggestions such as includelocal or making a special case for ./-prefixed paths are redundant.

> A command line switch or +I  get the job done and relieve the users from having to change the makefiles.


I'm not against such user-oriented solutions.  :-)  Sorry if you got that impression.

Steven Simpson <simpsons>
Mon 26 Apr 2021 02:41:43 PM UTC, comment #11: 


> Can you use this?:
> include $(CURDIR)/foo.mk



This indeed suppresses the lookup.

My opinion is that the users should not have to know this detail. The behavior should preferably be optimial for the most common scenarios.
Also, most (as far as i can tell) existing makefiles do not do this.


> this might be better considered a question of disabling the search path on a per-include basis


Is there a need for such flexibility? A command line switch or +I  get the job done and relieve the users from having to change the makefiles.

Dmitry Goncharov <dgoncharov>
Sun 25 Apr 2021 05:53:35 PM UTC, comment #10: 


> One trouble of this default is, when your dep files are missing, make does a fs access per default directory for each missing dep file.


Are these always cases where you know you want a local file?  Can you use this?:


include $(CURDIR)/foo.mk


(Or does the full path cause more accesses?)

In terms of new features, this might be better considered a question of disabling the search path on a per-include basis, when you know you want a local file.  The '*include' directives are implemented with a call to eval_makefile, which has an RM_INCLUDED option to enable the search path, it seems.  Perhaps a leading '.' on the filename should suppress it (just as '/' does), or a new directive '*includelocal' would call without RM_INCLUDED.

Steven Simpson <simpsons>
Sun 25 Apr 2021 02:51:10 PM UTC, comment #9: 


> But, maybe it's better to grab that bull by the horns.


From the user's point of view, it may indeed be better to have a single interface (MAKEFLAGS), which controls the behavior from the makefile.

> Regarding removing default directories, I wonder what people think of using something like "-I-" to mean "delete all known include directories up to here and start over with a fresh set"?


That should work.
Another way is to mimics the shell and use the + sign. +I/usr/include removes /usr/include.

> It just seems to me that this issue of default include paths is too minor/rarely problematic to waste an entire command line option on it :).


One trouble of this default is, when your dep files are missing, make does a fs access per default directory for each missing dep file.
Another trouble is that, as far as i can tell, this default hurts most of the users to benefit few of them.
However, since Steven reported there are makefile which use this,  i agree, the default needs to stay.

Dmitry Goncharov <dgoncharov>
Sat 24 Apr 2021 03:39:02 PM UTC, comment #8: 

Sorry I guess I wasn't clear.  I wasn't going to remove them by default: the default behavior and search path won't change under any of these proposals.

I was suggesting that instead of creating a new flag specifically to remove default includes, we allow users to provide a special version of -I to remove default includes.  So if users give (in my example) the "-I-" option that would mean "reset the include search path to empty".

If they didn't given the "-I-" option, then the current behavior including default search path would be preserved.

It just seems to me that this issue of default include paths is too minor/rarely problematic to waste an entire command line option on it :).

Paul D. Smith <psmith>
Group administrator
Sat 24 Apr 2021 07:31:31 AM UTC, comment #7: 


> Regarding removing default directories, I wonder what people think of using something like "-I-" to mean "delete all known include directories up to here and start over with a fresh set"?  So if you used "make -I- -I/foo -I/bar" the include directories would be "/foo" and "/bar" with no defaults; if you use "make -I/foo -I- -I/bar" then the include path would just be "/bar".


I've written makefiles that are installed and used like libraries.  They are installed by default in /usr/local/include so that users can find them without any extra configuration, so it would be inconvenient to lose the notion of default include directories.  A switch analogous to -r/-R seems like a more backward-compatible option than removing them all by default.

Steven Simpson <simpsons>
Fri 23 Apr 2021 06:50:29 PM UTC, comment #6: 

Having makeflags take effect immediately could be a lot of effort so I was hoping to avoid it.  That's why I looked at modifying the .INCLUDE_DIRS variable instead.

But, maybe it's better to grab that bull by the horns.  I already solved some issues with it locally, in order to allow long options to appear in MAKEFLAGS.  I'll look into it.

Regarding removing default directories, I wonder what people think of using something like "-I-" to mean "delete all known include directories up to here and start over with a fresh set"?  So if you used "make -I- -I/foo -I/bar" the include directories would be "/foo" and "/bar" with no defaults; if you use "make -I/foo -I- -I/bar" then the include path would just be "/bar".

I don't know if that's too baroque.

Paul D. Smith <psmith>
Group administrator
Tue 20 Apr 2021 08:33:11 PM UTC, comment #5: 


> The problem is that MAKEFLAGS is only parsed once before all makefiles are read, and again after all makefiles are read.


So, this all boils down to having make consider changes to MAKEFLAGS immediately.


> I was hoping to avoid adding a new command line switch.


Well, even with MAKEFLAGS having immediate effect there needs to be a mechanism to remove the default directories.
Another option is to remove the default directories from make. Is there a makefile that includes anything from /usr/include or /usr/local/include or /usr/gnu/include?

Dmitry Goncharov <dgoncharov>
Tue 20 Apr 2021 06:39:34 PM UTC, comment #4: 


> 5. Introduce a command line switch to allow the user to remove the default include directories. https://savannah.gnu.org/bugs/?60412.

I was hoping to avoid adding a new command line switch.  In general I don't like them, plus we may need them for something else, plus having things on the command line means that the behavior isn't encapsulated in the makefile itself which is not great for many reasons.

So what I was hoping is that if this issue could be addressed then we wouldn't need bug #60412.

> 2. Allow the makefile to modify the search path through adding -I<path> to MAKEFLAGS. https://savannah.gnu.org/bugs/?58341.

This is, unfortunately, not good enough to do what you suggest here.  The problem is that MAKEFLAGS is only parsed once before all makefiles are read, and again after all makefiles are read.  It is not re-parsed every time it is updated: see bug #45211 (enhancement).  That means it's unsuitable for resetting -I since by the time it's reparsed all included files have already been read in.

> Could be a problem. make tests use this feature. Users may use it as well.

Make tests test this feature; obviously if it were changed the tests would change to match the new behavior.  But users may be using it, I agree.  That's especially true since up until now you couldn't learn the -I options by examining MAKEFLAGS so looking at .INCLUDE_DIRS was the only way to see them.  I can't speculate on how many peoples' makefiles care to examine the value of the -I options or why they might do so.  I'm sure it's rare but I can't say no one does it.

Paul D. Smith <psmith>
Group administrator
Tue 20 Apr 2021 04:47:28 PM UTC, comment #3: 


> The change I have allows someone to reset the search path from within the makefile


Awesome. i personally, need this change to be able to skip the default search directories.
 

> In make generally (and, IMO, correctly) the person invoking make gets the final say


agree

> -I directories would no longer appear in .INCLUDE_DIRS.


Could be a problem. make tests use this feature. Users may use it as well.


What about the following?

1. All directories specified on the command line with -I are prepended to include_directories. This is the case today.

2. Allow the makefile to modify the search path through adding -I<path> to MAKEFLAGS. https://savannah.gnu.org/bugs/?58341.

3. Ignore changes to .INCLUDE_DIRS. This is the case today.

4. Document that modifications of .INCLUDE_DIRS (from within the makefile or on the command line) do not affect the seach path.

5. Introduce a command line switch to allow the user to remove the default include directories. https://savannah.gnu.org/bugs/?60412.

As far as i can see, this alternative is backward compatible and gives both the user and the makefile the power to change the search path arbitrarily. At the expense of an additional command line switch.




Dmitry Goncharov <dgoncharov>
Mon 19 Apr 2021 10:01:02 PM UTC, comment #2: 

I have a change which fixes this issue.  There is one thing to consider, which I wonder if anyone has input on.

Today, .INCLUDE_DIRS contains all files in the search path.  That means it contains both the default search directories and also the directories added with -I.

The change I have allows someone to reset the search path from within the makefile by modifying the value in .INCLUDE_DIRS.  That includes both removing any default search paths and also adding new ones that are specific to the build system, so that they don't have to be provided on the command line.

There is one issue: what do we do about the -I options provided by the user?

The way it currently works in the changes I have, the makefile can modify the search path completely: it's initialized to the default value of -I directories plus default directories before the first makefile is parsed and that's it: if the makefile resets .INCLUDE_DIRS then that's the set of paths that're used.

But, I don't really like this.  In make generally (and, IMO, correctly) the person invoking make gets the final say: it doesn't matter what a variable is set to in the makefile, if the user sets it on the command line then that's what's used (I'm ignoring override here, I realize).  It's Not Good(tm) that any makefile author can add ".INCLUDE_DIRS =" to their makefile and that essentially prevents any -I option on make invocation from having any effect.

But I don't see a nice alternative.  I've considered these:

1) Have .INCLUDE_DIRS variable only contain the default search path, then users can reset it and/or add their own by changing .INCLUDE_DIRS, but no matter what they set it to the values provided with -I would always be searched first.  This, though, is a backward-compatibility change since the -I directories would no longer appear in .INCLUDE_DIRS.

2) Leave .INCLUDE_DIRS containing all the search paths, but then always search for -I directories first regardless of what .INCLUDE_DIRS is set to.  This could be confusing for users as the search path will not be what they have set it to (for example, if they use ".INCLUDE_DIRS := mydir $(INCLUDE_DIRS)" the invoker's -I directory will be searched before "mydir", regardless of what they see when they run "$(info $(.INCLUDE_DIRS))").  Definitely we'd need to massage the search path to avoid duplicate directory lookups.

3) Leave it as I have it now and say "oh well".

I'm currently leaning towards #2 as the "least bad" option but I'm interested in opinions.  I would prefer #1 except I don't know if I can justify the backward-compatibility break.

Paul D. Smith <psmith>
Group administrator
Sat 21 May 2016 09:54:22 PM UTC, comment #1: 

There is no need to add a new environment variable for this.  The MAKEFLAGS variable already exists, and you can just add -I options into it which will be parsed as if you'd entered it on the command line.  Also it's already possible when building GNU make on Windows to set the INCLUDEDIR preprocessor variable to add a new default pathname to GNU make: this just isn't done by default because there's no good choice for what it should be.

I've retargeted this issue as a generic enhancement to allow setting the .INCLUDE_DIRS variable to change the search path.

Paul D. Smith <psmith>
Group administrator
Mon 09 May 2016 04:22:45 PM UTC, original submission:  

Attached a patch (to read.c only) which might serve as a starting point for allowing an additional (hardcoded) default include path given in a Windows environment variable (proposed variable name: 'MAKELIBPATH').

Please note that the patch ignores multiple paths (delimited by ';') given in MAKELIBPATH, i.e. MAKELIBPATH should contain one path only.


Futher enhancement:
===================

The solution above might be considered as an intermediate step to where all paths in .INCLUDEDIRS are searched for makefiles to be included - regardless of where the paths are defined. As of now paths provided by the '-I' switch on the command line and default paths are the only ones which are searched for include makefiles, while any paths added to .INCLUDEDIRS within a makefile are not searched.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #37115:  read.c.diff added by None (1KiB - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by simpsons (Posted a comment)
  • -email is unavailable- added by dgoncharov (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-05-21 psmith Operating SystemMS Windows Any
        Summary[Win] added MAKELIBPATH as default include directory Allow updates to .INCLUDE_DIRS to change search path
    2016-05-09 None Attached File- Added read.c.diff, #37115

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code