bugmake - Bugs: bug #47913, newlines lost with $(foreach)

 
 

bug #47913: newlines lost with $(foreach)

Submitter:  Oliver Kiddle <opk>
Submitted:  Fri 13 May 2016 02:41:26 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Not A Bug Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  4.2 Operating System:  Any
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 17 May 2016 03:43:59 PM UTC, comment #6: 

Markup in Savannah is described here, at least: https://savannah.gnu.org/cookbook/?func=detailitem&item_id=125


 Generally you want to use +verbatim+ / -verbatim-

Paul D. Smith <psmith>
Group administrator
Tue 17 May 2016 10:49:18 AM UTC, comment #5: 

Using subst to replace newlines with some other character works fine and I'm happy with that as a solution. Thanks. I prefer that to the MAKE_DEPS macro because it is backwards compatible with the individual Makefiles (they span a large number of projects and directories).

By the way, my original Makefile had the lines inside the FILEDEPS macro indented with a couple of spaces. Sorry but I couldn't (and still can't) find how to do the required markup for the Savannah tracker in the Savannah help. Thanks again.

Oliver Kiddle <opk>
Mon 16 May 2016 01:51:18 PM UTC, comment #4: 

You could get the same behavior as before as long as the dependency list is exactly 1 file:


define ASDF =
Filee.cpp:File.h
Other.cpp:Other.h
endef

SPACE:=
SPACE:=${SPACE} ${SPACE}
here = here
$(foreach x,${ASDF},$(subst ${SPACE},:,$(foreach y,$(subst :,${SPACE},${ASDF}),$(abspath ${here}/${y}))))


... though I would probably be more inclined to abandon that approach in favor of something like:


# Usage:
#   $(call MAKE_DEPS, list of source files, list of header dependencies)
define MAKE_DEPS =
$(if $(filter undefined,$(origin 1) $(origin 2)),,$(if ${1},$(if ${2},$(eval $(foreach x,${1},$(abspath ${here}/${x})) : $(foreach x,${2},$(abspath ${here}/${x}))))))
endef

# ...

$(call MAKE_DEPS, File.cpp, File.h)
$(call MAKE_DEPS, Other.cpp, Other.h)
$(call MAKE_DEPS, File.cpp Other.cpp, Test.h Blah.h Stuff.h)



Brian Vandenberg <phantal>
Sun 15 May 2016 03:50:30 PM UTC, comment #3: 

Oops I meant to say it is directly related to bug #46995

There's no way to fix bug #46995 without breaking this usage.  That bug doesn't have anything to do with whether all arguments or just the first argument in the foreach is stripped: it has to do with how the contents of a define variable are tokenized when it's expanded: that happens before we even know we're going to be running a foreach function.

Paul D. Smith <psmith>
Group administrator
Sun 15 May 2016 03:45:30 PM UTC, comment #2: 

I'm having a hard time understanding what you're doing, not least because there are apparently arbitrary newlines added to your examples :).  Also, I assume the output you expect to get from the operation below would be:


File.cpp: /my/dir/here/File.h
Other.cpp: /my/dir/here/Other.h


However, that's not what you actually get.  What you get (with prior versions of GNU make) is:


/my/dir/here/File.cpp: /my/dir/here/File.h
Other.cpp: /my/dir/here/Other.h


If you change the makefile to show the deps in the loop you'll see what's happening:


$ cat Makefile
define FILEDEPS
File.cpp: File.h
Other.cpp: Other.h
endef

here = here

ifdef FILEDEPS
$(foreach dep,$(FILEDEPS),$(info dep = '$(dep)'))
endif

all:;@:

$ make
dep = 'File.cpp:'
dep = 'File.h
Other.cpp:'
dep = 'Other.h'


So you can see why you get the output you do: there are actually only three "words" in the FILEDEPS variable, not four, because the newline is not considered to be a word-separating character in this context.  Which is sort of bogus (IMO).  Your setup is somewhat rickety anyway: if you have any whitespace at the end of a line, for example, it will break (even in the old version of GNU make).

The change you're running into isn't directly related to bug #46995.  As above, previously make had a kind of schizophrenic attitude towards newlines: sometimes they were treated as word-separating and sometimes they were not treated as word-separating.  I made a change to sanitize this, so that make treats newlines as word-separating whitespace in virtually all situations, which is why you're seeing this; if you run the above makefile using make 4.1.90 you get:


dep = 'File.cpp:'
dep = 'File.h'
dep = 'Other.cpp:'
dep = 'Other.h'


This is much more clean, and IMO correct, but it does break your usage.  To work around it, you can use subst to replace newlines with some other character that you're sure will never appear in your FILEDEPS, then convert it back again, like this:


$ cat Makefile
# create a variable containing exactly one newline
# (requires 2 newlines since make strips the last one)
define N


endef

define FILEDEPS
File.cpp: File.h
Other.cpp: Other.h
endef

here = here

ifdef FILEDEPS
$(foreach dep,$(subst $N,!,$(FILEDEPS)),$(info dep = '$(subst !,$N,$(dep))'))
endif

all:;@:

$ make
dep = 'File.cpp:'
dep = 'File.h
Other.cpp:'
dep = 'Other.h'


which should work equivalently in both old and new versions of GNU make.  If for some reason this isn't going to work please provide updated details for why.

Paul D. Smith <psmith>
Group administrator
Fri 13 May 2016 04:04:00 PM UTC, comment #1: 

This is probably related to the fix for bug 46995.  I haven't looked at the fix for that bug but I had expected the fix would be to strip the 1st argument in $(foreach) as opposed to stripping all the arguments.

Brian Vandenberg <phantal>
Fri 13 May 2016 02:41:26 PM UTC, original submission:  

I tried the 4.1.90 release candidate with my Makefiles and found a problem due to it behaving differently to previous releases with regard to newlines being lost from a variable when using $(foreach).

Simplifying things a lot, what I have is roughly along these lines:

define FILEDEPS
  File.cpp: File.h
  Other.cpp: Other.h
endef

here = here

ifdef FILEDEPS
  $(eval $(foreach dep,$(FILEDEPS),$(or $(filter :,$(dep)),$(abspath $(here)/$(dep)
))))
endif

To put this into context, I use a non-recursive setup where a small Makefile in each individual directory contains definitions for that directory's contents and then includes a complex common makefile which defines all the rules and includes all the other Makefiles. So this allows certain extra file depenencies to be defined without specifying the directory. here is actually set with something like the following so that it points to the directory of the current directory specific
Makefile:

here := $(dir $(lastword $(filter-out %.d %.D %.mk %/config.inc $(lastword $(MAKEFI
LE_LIST)), $(MAKEFILE_LIST))))

With 4.1.90, I get a "multiple target patterns" error. If you change $(eval) for $(info), current releases of GNU make show the two file dependencies. With 4.1.90, it has wrapped them onto a single line.

Is this an intentional change, a bug or just an implementation quirk that I shouldn't have relied on?

Oliver Kiddle <opk>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by phantal (Posted a comment)
  • -email is unavailable- added by opk (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-05-15 psmith StatusNone Not A Bug
        Open/ClosedOpen Closed

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code