bugmake - Bugs: bug #16132, Quoting problem in 3.81rc1

 
 

bug #16132: Quoting problem in 3.81rc1

Submitter:  None
Submitted:  Mon 20 Mar 2006 12:12:20 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Not A Bug Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  None Operating System:  MS Windows
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 01 Apr 2006 11:32:42 AM UTC, comment #23: 

The problem is that README.W32 mentions this in the part some impatient people might never read.  The first section, the one with the concise build instructions, should at least suggest looking in those deeper parts where it says to edit config.h.

However, given that make 3.81 is already out, this is now a moot point, as the instructions are likely to change considerably in the next release.

Eli Zaretskii <eliz>
Group Member
Sat 01 Apr 2006 12:32:49 AM UTC, comment #22: 

I looked at README.W32 and this already seems pretty well documented there; at least I couldn't think of anything to add to the existing comments.  It already tells you to define HAVE_CYGWIN_SHELL if you want to use Cygwin sh.exe, and to not define both HAVE_CYGWIN_SHELL and BATCH_MODE_ONLY_SHELL.  And in fact, there are guards in config.h.W32 that will cause the compilation to fail if both are defined.

I'm closing this bug.  If someone discovers why "echo" is handled specially and wants to change that handling please open a new bug or provide a patch (with explanation).

Paul D. Smith <psmith>
Group administrator
Mon 27 Mar 2006 08:01:44 PM UTC, comment #21: 

Thanks, Pieter.  This means there's no bug at all: the comments inside config.h.W32 clearly say to define HAVE_CYGWIN_SHELL if you are using such a shell.  (Paul, do you think we should emphasize this in README.W32?)

I don't know why "echo" is defined as a built-in only for batch-mode sh.exe (note that it is always defined for cmd.exe).  Perhaps it's because of some quoting issue, since basically, invoking Cygwin programs from a VC-compiled Make is playing with fire.  As Paul points out, this code is there for a long time, so let's not change it now, before the release.  You can always use echo.exe from Coreutils if you need `echo', right?

Eli Zaretskii <eliz>
Group Member
Mon 27 Mar 2006 05:39:48 PM UTC, comment #20: 

Pieter: thanks for debugging this, it's very helpful!  I'd like to do something with this bug before the release (which is imminent) if possible; is it not a bug after all?

I notice that the change to include echo in the BATCH_MODE_ONLY_SHELL was made in job.c version 1.106, back in 1998 (!) just as GNU make 3.77 was released, so that's not a new change itself.  I don't know why it's there though.

Paul D. Smith <psmith>
Group administrator
Mon 27 Mar 2006 10:08:26 AM UTC, comment #19: 

Hi Eli,

I looked deeper into the issue this weekend.
I discovered that from 3.81b3 to 3.81b4, the default config.h settings for W32 builds changed.
Up to 3.81b3, BATCH_MODE_ONLY_SHELL was defined in config.h.W32. 
Starting with 3.81b4, BATCH_MODE_ONLY_SHELL is undefined in config.h.W32. 
So it turned out I was doing batch mode only builds up to now, even when using cygwin sh.exe.

After some experimentation, the following worked for me (W32 native make + cygwin sh.exe) in 3.81rc2:
(1) I left BATCH_MODE_ONLY_SHELL undefined
(2) I defined HAVE_CYGWIN_SHELL.
(3) In line 2294/2296 in job.c, I commented out the ifdef:
    #ifdef BATCH_MODE_ONLY_SHELL
                     "echo",
    #endif

Do you know why the "echo" is guarded ?

Regards,
Pieter.


Anonymous
Fri 24 Mar 2006 09:23:26 AM UTC, comment #18: 

Then I suggest that you try replacing the shell as well.  As I already wrote, I cannot reproduce this with my version of sh.exe.

Alternatively, you could try to debug this and see where exactly the problem lies.  If you can afford spending some time with the Visual Studio stepping through the code that invokes the shell, you will find here several people ready to give advice and suggest ideas.

As yet another possibility: can your build work without a Unix shell?  If so, rename sh.exe (or remove it from PATH), and see if things work for you.

Finally, you could try building Make with either HAVE_CYGWIN_SHELL or BATCH_MODE_ONLY_SHELL (but not both!) defined in config.h.  Or if you already have one of these defined, try undefining them.  Maybe one of these will solve your problems.  (If it does, please tell the details, so that we could update the installation instructions.)  FWIW, my Make binary is built with both of these undefined.

Eli Zaretskii <eliz>
Group Member
Thu 23 Mar 2006 08:18:33 AM UTC, comment #17: 

We have been using the Windows "native" version of make 3.80 built with VC6.  I won't bother you with the details of the joys of having to juggle with multiple cygwin1.dll versions on your PC due to the various third-party provided tools in our build environment. Having a version of GNU make that doesn't depend on cygwin1.dll certainly is a blessing.

We are not trying to replace a cygwin make.exe with a Windows "native" make; we are just trying to upgrade the Windows "native" make from version 3.80 to 3.81, to get a fix for the "missing 'endif'" problem.

Anonymous
Wed 22 Mar 2006 07:00:23 PM UTC, comment #16: 

Yes, I'm the same Eli Z.

Paul is correct asking you for the details of the old versions: how they were built and with what compier.

Eli Zaretskii <eliz>
Group Member
Wed 22 Mar 2006 05:21:37 PM UTC, comment #15: 


> I've been using make 3.80 with cygwin sh.exe quite comfortably
> for the past 2 years.


But which make 3.80?  The Cygwin one or the Windows "native" one?  They're quite different configurations.  If you're trying to replace a Cygwin make with a "native" one, that would explain the change in behavior.  That's what Eli was getting at in comment #13.

Martin Dorey <mdorey>
Wed 22 Mar 2006 08:33:05 AM UTC, comment #14: 

Hi Paul/Eli,

I've been using make 3.80 with cygwin sh.exe quite comfortably for the past 2 years. It was only when I bumped into a "missing 'endif'" problem that I considered upgrading to a newer version (hinted by Paul:  http://lists.gnu.org/archive/html/help-make/2005-11/msg00016.html). Not being able to use make with cygwin sh.exe anymore would be quite problematic.

Regards,
Pieter.

PS Is Eli the same as the one involved in DJGPP >= 10 years ago ?

Anonymous
Wed 22 Mar 2006 04:35:33 AM UTC, comment #13: 

Did the OP build the previous versions of Make with the same compiler as the release candidate?  If not, perhaps therein lies the reason for the difference, especially if the other versions are Cygwin ports (as opposed to to rc1 that was built with Microsoft Visual C).

We did change the backslash handling, but only before the newline, and there are no such combinations in the Makefile presented here.

I don't have the Cygwin shell installed on any of my machines, so I cannot test this myself.  If the OP can debug this, I'm willing to provide guidance.

Eli Zaretskii <eliz>
Group Member
Tue 21 Mar 2006 09:04:39 PM UTC, comment #12: 

I almost agree with you Eli.  The fact that make prints the right string before it invokes the shell would normally show that make is doing the right thing and the shell is broken.  However, as we know Windows is much more complex: the string make prints is not necessarily the same one sent to the shell.  And, the way make modifies the command line is different depending on whether it thinks it's invoking a command.com or a UNIX-y shell.  So, it's still possible that make is not managing the handling of quotes and backslashes properly when it invokes a shell.

The fact that it works for you with a zsh port is telling, but on the other hand the fact that it works for the OP with older versions of GNU make and not the current release candidates is ALSO telling, especially when you consider that we did modify the handling of backslashes (insofar as they escape newlines).

Given this, I'm not entirely convinced that this is not a problem with make.  Is there some way we could reproduce the problem?  Do you (or anyone on the Windows team) have access to a Cygwin or MKS shell?  Failing that we'll need the poster to do a bit of debugging on his own I'm afraid.

Paul D. Smith <psmith>
Group administrator
Tue 21 Mar 2006 08:32:53 PM UTC, comment #11: 

The result without sh.exe is the expected one.  So Make is working correctly, and the problem is is (in)compatibility with the Cygwin shell.

When I need a Unixy shell, I use the old port of zsh.  You should be able to find it on the net (sorry, I don't have a URL handy).  It isn't ideal, since it has bugs that prevent it from running a typical configure script, but it works very well for Makefiles that expect a Unixy shell.

Eli Zaretskii <eliz>
Group Member
Tue 21 Mar 2006 05:03:52 PM UTC, comment #10: 

Lastly, this is the result when no sh.exe is in PATH:


D:\GNUMake\test>PATH
PATH=C:\WINDOWS;C:\WINDOWS\system32

D:\GNUMake\test>gmake381rc2.exe
echo -aap="noot"
-aap="noot"
echo '-aap="noot"'
'-aap="noot"'
echo -aap=\"noot\"
-aap=\"noot\"
echo -aap="noot" -mies="wim"
-aap="noot" -mies="wim"
echo '-aap="noot" -mies="wim"'
'-aap="noot" -mies="wim"'
echo -aap=\"noot\" -mies=\"wim\"
-aap=\"noot\" -mies=\"wim\"

So this seems to work correctly...

Anonymous
Tue 21 Mar 2006 04:59:29 PM UTC, comment #9: 

I have an old MKS sh.exe lying around; behavior is somehwat different: (\n gets converted to newline...)


D:\GNUMake\test>PATH
PATH=C:\FromOldPc\isimip\sniff\mks\mks-6.1\mksnt;C:\WINDOWS;C:\WINDOWS\system32

D:\GNUMake\test>gmake381rc2.exe
echo -aap="noot"
-aap=noot
echo '-aap="noot"'
-aap=
oot"
echo -aap=\"noot\"
-aap=
oot"
echo -aap="noot" -mies="wim"
-aap=noot -mies=wim
echo '-aap="noot" -mies="wim"'
-aap="noot" -mies="wim"
echo -aap=\"noot\" -mies=\"wim\"
-aap=
oot" -mies="wim"

Anonymous
Tue 21 Mar 2006 04:53:33 PM UTC, comment #8: 

I also tried with
#define HAVE_CYGWIN_SHELL
in config.h
but this gives the same result.

Eli/Paul, can you suggest another sh.exe (instead of cygwin) that I could use/try ?

Anonymous
Tue 21 Mar 2006 04:47:42 PM UTC, comment #7: 

(And this is the makefile (with TABS of course:)


BLA1 :=  -aap="noot"
BLA2 := '-aap="noot"'
BLA3 :=  -aap=\"noot\"

BLA4 :=  -aap="noot" -mies="wim"
BLA5 := '-aap="noot" -mies="wim"'
BLA6 :=  -aap=\"noot\" -mies=\"wim\"    

all:
echo $(BLA1)
echo $(BLA2)
echo $(BLA3)
echo $(BLA4)
echo $(BLA5)
echo $(BLA6)

Anonymous
Tue 21 Mar 2006 04:43:58 PM UTC, comment #6: 

Hi Paul, here it is:


D:\GNUMake\test>PATH
PATH=C:\cygwin\bin;C:\WINDOWS;C:\WINDOWS\system32

D:\GNUMake\test>gmake381rc2 --version
GNU Make 3.81rc2
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for Windows32

D:\GNUMake\test>gmake381rc2
echo -aap="noot"
-aap=noot
echo '-aap="noot"'
-aap=\noot"
echo -aap=\"noot\"
-aap=\noot"
echo -aap="noot" -mies="wim"
-aap=noot -mies=wim
echo '-aap="noot" -mies="wim"'
-aap="noot" -mies="wim"
echo -aap=\"noot\" -mies=\"wim\"
-aap=\noot" -mies="wim"

Anonymous
Tue 21 Mar 2006 04:25:35 PM UTC, comment #5: 

Can you please remove the "@" prefix from your echo commands, and then re-run your test.  Show us the output make prints (which are the commands that it's sending to the shell) against the command you expect to be run and what actually gets invoked by the shell.

Can you also try the latest rc2 version (ftp://alpha.gnu.org/gnu/make/make-3.81rc2.tar.gz)?  There were some small Windows-related fixes in that version.

Paul D. Smith <psmith>
Group administrator
Tue 21 Mar 2006 04:10:28 PM UTC, comment #4: 

Latest cygwin didn't help...

Anonymous
Tue 21 Mar 2006 03:36:41 PM UTC, comment #3: 


> What kind of sh.exe are you using, and how (with what compiler and
> development environment) did you build Make 3.81rc1?


I use Windows XP SP2.
I have built make3.81rc1 with both VC6 and VC7.1 (giving the same results).
I am using sh.exe from cygwin. (cygwin1.dll version 1.5.18-1)
I will try to see if the latest cygwin (1.5.19-4) fixes the issue...

Anonymous
Mon 20 Mar 2006 08:11:39 PM UTC, comment #2: 

I cannot reproduce this, either, on Windows XP.  I tried with and without a Unixy sh.exe on my PATH, and both behave as expected, identically to v3.80 and v3.80b4.

(The behavior with and without sh.exe is not identical, but that's because the `echo' built-in behaves differently in each case, as far as removing single and double quotes is concerned.)

What kind of sh.exe are you using, and how (with what compiler and development environment) did you build Make 3.81rc1?

Eli Zaretskii <eliz>
Group Member
Mon 20 Mar 2006 01:26:04 PM UTC, comment #1: 

I can't reproduce this on GNU/Linux; both rc1 and the latest CVS code work fine.  Please provide more details about your OS: what version of Windows; which shell are you using; etc.

Paul D. Smith <psmith>
Group administrator
Mon 20 Mar 2006 12:12:20 PM UTC, original submission:  

# GNU Make 3.80 gives:
#  -aap=noot
#  -aap="noot"
#  -aap="noot"
#  -aap=noot -mies=wim
#  -aap="noot" -mies="wim"
#  -aap="noot" -mies="wim"

# GNU Make 3.81 b3 gives:
#  -aap=noot
#  -aap="noot"
#  -aap="noot"
#  -aap=noot -mies=wim
#  -aap="noot" -mies="wim"
#  -aap="noot" -mies="wim"

#
# GNU Make 3.81 rc1 gives:
#  -aap=noot
#  -aap=\noot"
#  -aap=\noot"
#  -aap=noot -mies=wim
#  -aap="noot" -mies="wim"
#  -aap=\noot" -mies="wim"


BLA1 :=  -aap="noot"
BLA2 := '-aap="noot"'
BLA3 :=  -aap=\"noot\"

BLA4 :=  -aap="noot" -mies="wim"
BLA5 := '-aap="noot" -mies="wim"'
BLA6 :=  -aap=\"noot\" -mies=\"wim\"    

all:
@echo $(BLA1)
@echo $(BLA2)
@echo $(BLA3)
@echo $(BLA4)
@echo $(BLA5)
@echo $(BLA6)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #1507:  makefile2 added by None (764B - text/plain - example makefile (see submission))

 

Depends on the following items: None found

Items that depend on this one: None found

 

CC list is empty

 

There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

Only logged-in users can vote.

 

Follow 4 latest changes.

Date Changed by Updated Field Previous Value => Replaced by
2006-04-01 psmith StatusWorks for me Not A Bug
    Open/ClosedOpen Closed
2006-03-20 psmith StatusNone Works for me
2006-03-20 None Attached File- Added makefile2, #3508

Back to the top

Powered by Savane 3.13-3230.
Corresponding source code