bugmake - Bugs: bug #16505, Line-continuation backslashes are...

 
 

bug #16505: Line-continuation backslashes are not stripped

Submitter:  None
Submitted:  Wed 03 May 2006 06:45:29 AM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Not A Bug Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  4.0 Operating System:  POSIX-Based
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 05 May 2006 12:55:17 AM UTC, comment #12: 

I understand the workarounds. (Make variables are probably the way to go, because double-quoting script languages tends to be cumbersome.) My makefiles won't be that much of a problem, anyway, compared to those of others---I first encountered this new "feature" in running a makefile from XScreenSaver. Doubtful that'll be the only instance.

I kept mentioning the unexpected-EOF errors just to emphasize how brain-dead the POSIX behavior is, because those are what you get if you (reasonably) think that POSIX treats single-quoted EOL backslashes as literal characters (such that all you need to do so that they don't appear in your inline scripts is, well, delete them).

Anonymous
Thu 04 May 2006 03:47:34 PM UTC, comment #11: 

I can understand everything you are saying -- the backward incompatibility, the suggestion to add an explicit warning, the works.  The only thing I do NOT understand is why do you insist on telling about the EOF error etc., when you have at least two ways of working around the problem: either use double quotes instead of single quotes, or put the entire multi-line script into a Make variable (in which case you can keep the single quotes)?  It should not be too hard to modify your Makefile to work with the new release of Make, without breaking the old releases as well.

Eli Zaretskii <eliz>
Group Member
Wed 03 May 2006 08:37:54 PM UTC, comment #10: 

Keeping the newline + discarding the backslash would be a reasonable behavior. That would spare most inline scripts.

If precedent is desirable, however, I think the old behavior is perfectly defensible. It's worked that way forever, various Unix systems do it identically, and it allows a useful idiom. Sure, you can't embed newlines in the string---but how often is this useful? And what good is it if a backslash is forced before every one? (I can't even think offhand of what Unix tools will ignore a backslash at the end of a line...)

Ultimately, I don't see who is being served by this enforcement of POSIX. It's a loss of functionality, and backwards compatibility, for no corresponding gain other than POSIX compliance. And those who really care about compliance would be no less satisfied if it were enabled with POSIXLY_CORRECT, as that's already the case with most other GNU tools where the standard is relevant.

I think that's about as strong a case as I can make, so I'll leave it at that before horse-beating ensues. Let me just finish by saying, however, that if this is to stand, it's going to have to be disseminated much more broadly than a NEWS file. My first reaction to this change was that it was clearly, obviously, undeniably a bug; others will react similarly. Perhaps have make(1) print a warning when it sees such a multi-line string, maybe only if the command fails. But something to immediately make clear that "this is not a regression, but a deliberate change whose purpose is to follow the Posix specifications."

Anonymous
Wed 03 May 2006 07:35:16 PM UTC, comment #9: 

Argh.  I mean "You have to have the BACKSLASH in the makefile", in the second paragraph of the previous comment :-/

Paul D. Smith <psmith>
Group administrator
Wed 03 May 2006 07:21:20 PM UTC, comment #8: 

I think the INTENT was to allow embedded newlines.  However, the implementation only works for tools which will ignore the backslash as well.  Some UNIX tools do this; many don't.

You have to have the newline in the makefile, because make does not and absolutely cannot parse the contents of your command line to determine whether the newline is inside single-quotes or not.  Make needs a 100% objective criteria as to whether to continue a line, and a backslash/newline is it.

What I suppose you REALLY want is that, for backslash/newline pairs in a command script, the BACKSLASH is removed but the NEWLINE is preserved when make invokes the shell.  This allows for all possible options: you can preserve newlines (which is important in some situations!) and you can choose to not have a backslash before the newline (the default) or to have one (by adding "\\" before the backslash/newline).

But, that's not what POSIX defines, and it's not how any other version of make I'm aware of works.  So... :-/

Paul D. Smith <psmith>
Group administrator
Wed 03 May 2006 06:50:44 PM UTC, comment #7: 

That's what I was getting at with "if you drop the backslashes" and "Bourne-shell-like behavior." It wouldn't be so bad if you could do...

$ cat >Makefile
foo:
    echo 'foo
    bar
    baz'

$ make foo
foo
bar
baz

...except that you can't. You get an EOF error. Or are you alluding to a different way of embedding newlines?

Anonymous
Wed 03 May 2006 06:36:16 PM UTC, comment #6: 

The Posix behavior has its merits: without it, how does one pass a command line with embedded newlines from a Makefile?  The Posix handling of newlines in single quotes permits this.

Eli Zaretskii <eliz>
Group Member
Wed 03 May 2006 06:05:17 PM UTC, comment #5: 

Sorry; I wasn't too clear there. My point was, basically, that the POSIX behavior sucks }:-)

Okay, so multi-line single-quoted strings now retain the backslashes. This implies that removing the backslashes is all you need to do, that they are superfluous, that the behavior is now Bourne-shell-like in that an open quote automatically causes a continuation at the end of the line. But no; when you remove the backslashes, there's no continuation at all, hence the EOF error. So the situation is, backslashes are still your continuation character, except that they're useless---unless you don't mind them showing up in the string. So for all but a tiny minority of cases, multi-line single-quoted strings in command scripts have been removed as an usable idiom in GNU Make.

I recognize the value of POSIX compliance, especially in regulatory contexts, but the approach usually taken by other GNU tools when the POSIX behavior is annoying/limiting is to enable it only when POSIXLY_CORRECT is defined. I'm wondering why that wasn't done here, especially when the POSIX behavior disallows a useful/common idiom, and breaks a number of existing makefiles.

Anonymous
Wed 03 May 2006 05:06:11 PM UTC, comment #4: 

Drop the backslashes?  Certainly you can't just drop the backslashes; neither Eli nor I suggested this.

You can drop BOTH the backslashes AND the newlines; that will work although your makefiles will presumably be harder to read due to long lines.

Or (as I suggested) you can take the quoted string and put it into a make variable (preserving the backslashes and newlines just as they are now).  That will also solve the problem.

Paul D. Smith <psmith>
Group administrator
Wed 03 May 2006 04:54:00 PM UTC, comment #3: 

If you drop the backslashes from the single-quoted string, you get an "unexpected EOF" error. Isn't this a case where the POSIX behavior is counterproductive, and would better be made contingent on a POSIXLY_CORRECT variable?

My testing shows that the vendor-supplied make on AIX, HP-UX, Tru64 and FreeBSD has the non-POSIX behavior; Solaris is the only standout. In light of the breakage that this change will cause, I don't think many users will be amenable to the view that the behavior isn't a bug.

Anonymous
Wed 03 May 2006 11:25:25 AM UTC, comment #2: 

Eli's answer is generally correct.  The NEWS and manual explain the change, and this is not a bug in GNU make but rather a deliberate change (other versions of make already work the new way).

However, one minor detail: the simplest change to get it to work is usually NOT changing to double-quotes; that would involve (as Eli mentions) a LOT of extra work quoting things and make the results confusing and difficult to read and edit.

As recommended in the manual, the simplest change to get it to work is to put the script into a make variable:

  SCRIPT := 'print "this is my "; \
             print "perl script";'
  foo: ; perl -e $(SCRIPT)

because make variable settings continue to use the standard make backslash/newline handling, rather than the shell backslash newline handling used in the command scripts.

Paul D. Smith <psmith>
Group administrator
Wed 03 May 2006 08:33:07 AM UTC, comment #1: 

This is not a regression, but a deliberate change whose purpose is to follow the Posix specifications.  From NEWS:

------------- quotation ----------------------

  • WARNING: Backward-incompatibility! 

In order to comply with POSIX, the way in which GNU make processes backslash-newline sequences in command strings has changed.  If your makefiles use backslash-newline sequences inside of single-quoted strings in command scripts you will be impacted by this change.  See the GNU make manual subsection "Splitting Command Lines" (node "Splitting Lines"), in section "Command Syntax", chapter "Writing the Commands in Rules", for details.
------------- end quotation -------------------

So your Makefile is in violation of Posix.  The simplest change to get it to work is to use double quotes instead of single quotes (and escape the inner double quotes as appropriate).


Eli Zaretskii <eliz>
Group Member
Wed 03 May 2006 06:45:29 AM UTC, original submission:  

Scenario: A makefile command (i.e. a shell command executed for some target) that makes use of a quoted string spanning multiple lines via continuation backslashes. E.g. an in-place awk or perl program.

Problem: make 3.81 and CVS do not strip the backslashes before passing the string to the shell. This is a regression from make 3.80.

The attached makefile clearly illustrates the bug, and the havoc that it can play on a perl program.

(Reporter: Daniel Richard G. <skunk@iSKUNK.ORG>)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #9881:  Makefile added by None (109B - text/x-makefile - Bug test case)

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only logged-in users can vote.

 

Follow 4 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2006-05-03 psmith StatusNone Not A Bug
    Open/ClosedOpen Closed
    Carbon-Copy- Added -email is unavailable-
2006-05-03 None Attached File- Added Makefile, #9881

Back to the top

Powered by Savane 3.13-02a9.
Corresponding source code