bugmake - Bugs: bug #16286, VPATH and directory cache

 
 

bug #16286: VPATH and directory cache

Submitter:  None
Submitted:  Thu 06 Apr 2006 06:15:54 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Not A Bug Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  3.79.1 Operating System:  Any
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 10 Apr 2006 12:50:56 PM UTC, comment #4: 

Okay, yes, you're right that VPATH isn't intended to do what I thought it would be useful for. That's not a bug, it's a feature. (sorry, couldn't resist)
Anyway, what you say about dismisisng the VPATH prefix isn't exactly true. In my project, when doing a complete build after a make clean (all .o deleted), I ended up with some of the .o with and some without vpath prefix in the linking stage. According to your explanations there should have been none with prefix (while I erroneously expected all with).

Anywa, a change in the (expected) behaviour would increase the usefulness. So count this as a feature request ;)

Anonymous
Thu 06 Apr 2006 08:33:03 PM UTC, comment #3: 

All of the behavior you see, as far as I can tell, is expected.  The documentation of vpath shows that if the target needs to be rebuilt then the vpath-qualified pathname is thrown away and the target is rebuilt locally (which means that when this target is listed as a prerequisite it will also be referred to locally).  If the target is not rebuilt then the vpath path will be preserved, obviously.

Also, any time you have a make rule that builds ANY target that is not the one make thinks it will build (which is the value of $@, exactly) then your makefile is incorrect, and you cannot (or, at least, should not) expect valid behavior from make.  Your makefile is a contract with make that tells the make program "if you want to build target A, then run these commands and it will be built".  If your rules don't build target A but instead something else such as ../A or $(OBJDIR)/A, then you have a bug in your makefile and make won't work properly.

All of the behavior you're seeing is explained in my webpages referenced earlier.  If a .o is found in the local directory then vpath searching is not performed, and you will just get that filename with no vpath prefix.  If the .o is not found locally but is found by vpath, but it is out of date and needs to be rebuilt, then again you will get that filename with no vpath prefix.  If the .o is found via vpath and doesn't need to be rebuilt, then and ONLY then will you get the vpath pathname.

You can modify the latter behavior with GPATH (see the manual) which will keep make from throwing away the vpath prefix for targets that need to be rebuilt.

In fact, the current behavior of VPATH is not at all useless.  It  is used in two ways, very commonly.  There is only one set of source code, but you can have a huge number of targets built from that code: different platforms, different compiler options, etc.  So VPATH can be used as it is with autoconf/automake: you create a separate directory, then run the build in that directory with a reference back to the source code.  This allows your makefiles to be much simpler to write, since you just write it to create all the objects in the current directory and the source is found via VPATH.  Otherwise you'd have to be very careful to prefix every target name with an "object directory" pathname so it wouldn't write to the source directory.

The other common use of VPATH is where you have a "master workspace" which is pre-compiled.  Then all the people working from that baseline make a local workspace which doesn't have any source in it.  They only check out into their local workspace the files that they want to work on (or maybe they use a link farm or whatever).  When they run make it will look locally, then in the master workspace (via vpath).  If the file exists locally, it's used.  If it exists in VPATH and it doesn't need to be rebuilt, then the master workspace copy is used.  If it exists in VPATH and does need to be rebuilt, then it's rebuilt into the local workspace (with no vpath) not the master copy.  This is exactly how VPATH works.

Paul D. Smith <psmith>
Group administrator
Thu 06 Apr 2006 07:55:06 PM UTC, comment #2: 

Well, that VPATH is useless for anything except leaf targets is what I figured out on my own after some time of inverstigation.

If one has to include the object dirs or whatever all over the place anyway, using VPATH just for leaf targets makes things not much easier. It has its (very limited) use anyway.

What bugged me was that the prerequisites of the linking process are partly prefixed through vpath, and partly not.
So instead of getting a command

 gcc foo1.o foo2.o foo3.o ...

(which of course won't work since the .o files are in obj/*) I end up with a command

  gcc foo1.o obj/foo2.o obj/foo3.o foo4.o ...

where the object files with prefix and the ones without prefix change with each call of make (after a clean) and their magnitude (how many of this or that) with the number of object files. (for the first one I suspected XRays as random generator, but maybe it's the OS or the amount of free memory or just coincidence)

So the bug is: either NO non-leaf-target (or rather its prerequisites) is handled using vpath (which would at least give a predictable result), or ALL are (which would give an even more pleasant result) but not a mixture of both, depending on whatever. Since Make does not know which target is a leaf, the second alternative seems to be easier to do. (before pruning a file, just verify its existence once and update the entry if necessary)

In its current incarnation, vpath is pretty useless and does not save much work. Saving to write the path of external source files in a makefile does not seem to remotely justify the work put into vpath. But that's just my opinion.

Anonymous
Thu 06 Apr 2006 06:26:12 PM UTC, comment #1: 

All of the things you say are true (except perhaps the stuff about cosmic x-rays :-)).  This is how the VPATH feature is documented to work and it works as documented (if you think there are cosmic x-rays involved please try the latest release, GNU make 3.81, and see if you get the same unexplainable behavior).

It's not clear to me what you are saying the bug is.  VPATH/vpath is definitely only useful with "leaf" targets; it is not useful with built targets.

See my paper on using VPATH, here: http://make.paulandlesley.org/vpath.html


Paul D. Smith <psmith>
Group administrator
Thu 06 Apr 2006 06:15:54 PM UTC, original submission:  

Normally, all prerequisites are searched in ./ and in all directories found in VPATH.
If found, the $^ variable holds the prerequisites with the actual path where they are found.
Fine.

Unfortunately this utterly fails if the prerequisite was the target of a previously executed rule.

Following the manual, make assumes that a rule creates its target. But what if the target is created in a different directory than then ./ ? make adds the file to its directory cache - without noticing that it was created somewhere else.
As soon as the former target appears as prerequisite, make does no longer search in VPATH but simply assumes that the file is in the current directory. And of course the rule fails to build its target then.

When running make a second time, it skips building the object files (of course), finds them through vpath and works as expected.

************************************************
VPATH = $(INCSRC):obj:bin
SRCS =  foo1.c foo2.c foo3.c foo4.c foo5.c foo6.c
OBJS =  $(SRCS:.c=.o)
LIBS =  $(ADDLIBS)
TARG =  $foo.elf

all: $(OBJS) $(TARG)
avr-objcopy -O binary -I ihex $(PROJ).hex $(PROJ).bin

%o : %c
@$(CC) -c $(CPFLAGS) $(INCFIRST) -I$(INCDIR) $(INCLAST) $< -o obj/$@

%elf: $(OBJS)
@$(CC) $^ $(LDFLAGS) $(LIBS) -o bin/$@


**********************************************************

make compiles all c files properly, the object files placed in the obj subfolder. but then the linking of the object files fails.
 While all compiled .o files are in obj/*.o , make constructs a $^ variable which contains some prerequisites with and some without the obj/ prefix.

A debug output reveals that whils considering targets and rules for the prerequisites in the %elf rule, make has pruned some files form the search (apparently because of their -wrong- appearance in the directory cache) while others are searched for and found by the VPATH mechanism.
Which ones are found and which are simply (and wrongly) assumed depends on the number of source files and seems to be influenced by cosmic x-rays too. The only fix to this problem was to change the rule like this:

%elf: $(OBJS)
@$(CC) $(addprefix obj/,$(notdir $^)) $(LDFLAGS) $(LIBS) -o bin/$@


which renders the vpath mechanism almost useless, only working properly for files that aren't made by make itself.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only logged-in users can vote.

 

Follow 2 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2006-04-06 psmith StatusNone Not A Bug
    Open/ClosedOpen Closed

Back to the top

Powered by Savane 3.14-962f.
Corresponding source code