bugmake - Bugs: bug #59247, function shell eats a newline

 
 

bug #59247: function shell eats a newline

Submitter:  Dmitry Goncharov <dgoncharov>
Submitted:  Sat 10 Oct 2020 06:24:33 PM UTC
   
 
Severity:  3 - Normal Item Group:  Enhancement
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
Component Version:  None Operating System:  None
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 05 Dec 2020 11:23:09 PM UTC, comment #3: 

This behavior is not related to bug #59395; that issue has to do with how target recipes are split into individual command lines.  This issue is discussing how an individual command line (or in this case the arguments to the shell function) are handled when they are sent to the shell (or not).

However, I am also not sure that the provided patch is correct.  This will always convert a newline to a space, regardless, which may not be correct in all situations (consider single-quoted strings for example).

I think there's a larger problem here.  Consider this makefile:


x := $(shell echo $(FOO))
all:;: x='$x'


The original request clearly expected that FOO would be treated the same way as a shell variable.  In the shell you'd get this:


$ FOO='
foo
bar
'

$ echo $FOO
foo bar


But, it's not a shell variable it's a make variable!  Make variables are expanded by make, before invoking the shell, and they don't follow the same escaping and quoting rules as the shell does.  So, if we have the above makefile and we run it like this:


$ make FOO='
foo
bar
'


then I'd expect that make would invoke the equivalent of this shell command:


/bin/sh -c 'echo
foo
bar
'


and that would yield errors because the commands "foo" and "bar" cannot be found.

Of course, if we had a makefile like this:


x := $(shell echo "$(FOO)")
all:;: x='$x'


(with quotes) then this would be the shell invocation:


/bin/sh -c 'echo "
foo
bar
"'


and the results (after newlines are converted on the output as discussed in the manual) would be " foo bar"

So, I'm a bit unsure how to proceed here.  Clearly the current behavior is not right, where adding a semicolon gives different output : the slow-path is causing issues.  But I'm not sure that fixing it to work completely correctly wouldn't be an issue with compatibility even though it's the right thing to do.

The problem is the overly complex way that make invokes subshells in the "slow path".  The slow path should be the SIMPLEST solution but instead it's more complicated and I'm not sure why.  It's not documented anywhere how shell will handle input strings that contain newlines; it only talks about output newlines, so I think that this behavior is more of an unanticipated result of how the parsing is done than anything intended.  Which doesn't mean that it won't be a backward-compatibility problem to change it.

I'll need to think about this a bit.

Paul D. Smith <psmith>
Group administrator
Mon 02 Nov 2020 01:25:09 PM UTC, comment #2: 

I am not sure whether your patch is the right fix.

Your bug may be related to bug 59395, I reported recently, but your patch does not fix the problem in bug 59395.

In general, gmake has several oddities in the area of handling space, tab, newline and backslash. I suspect that there may be a common reason that should be investigated and fixed in one single shot


Jörg Schilling <schily>
Sat 10 Oct 2020 06:33:14 PM UTC, comment #1: 

The patch in the attachment fixes the issue.

Dmitry Goncharov <dgoncharov>
Sat 10 Oct 2020 06:24:33 PM UTC, original submission:  

A user reported a bug here

https://lists.gnu.org/archive/html/bug-make/2020-10/msg00016.html.

From: Byrnes, Robert
Subject: embedded newlines in shell function variable expansion
Date: Fri, 9 Oct 2020 15:03:24 +0000

If I use this Makefile ...

----------------
bash$ cat Makefile
FOO := $(shell echo $(ENTRIES) ; )
BAR := $(shell echo $(ENTRIES)   )

all:
        @echo FOO = $(FOO)
        @echo BAR = $(BAR)

.PHONY: all
----------------

... and set ENTRIES with embedded newlines, then this happens:

----------------
bash$ make 'ENTRIES=
blartz
blurfl
'
FOO = blartzblurfl
BAR = blartz blurfl
----------------

Dmitry Goncharov <dgoncharov>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by schily (Posted a comment)
  • -email is unavailable- added by dgoncharov (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-01-08 psmith Item GroupNone Enhancement
    2020-10-10 dgoncharov Attached File- Added sv_59247_func_shell_eats_newline.diff, #49959

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code