bugmake - Bugs: bug #40159, Premature exit with incorrect...

 
 

bug #40159: Premature exit with incorrect error message, or garbage output characters.

Submitter:  padavo <padavo>
Submitted:  Wed 02 Oct 2013 05:39:01 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Duplicate Privacy:  Public
Assigned to:  None Open/Closed:  Closed
Component Version:  3.81 Operating System:  POSIX-Based
Fixed Release:  None Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 03 Oct 2013 03:52:15 PM UTC, comment #12: 

Thanks for testing.

Paul D. Smith <psmith>
Group administrator
Thu 03 Oct 2013 11:23:06 AM UTC, comment #11: 

My apologies for the stupid broken markup ... I gambled and lost :)

This is a repost of comment #10.

Summary:
--------

The procedure at comment #9 fixes the bug !
I will submit a Debian bugreport.

Procedure #1: Without the patch, the bug occurs
-----------------------------------------------

apt-get build-dep make
apt-get source make
cd make-dfsg-3.81/
dpkg-buildpackage -nc -us -uc
./make -R -r -f /home/david/work/src/c/make/lab/80-bug/clean/Makefile
/home/david/work/src/c/make/lab/80-bug/clean/Makefile:15: * unterminated variable reference.  Stop.

Procedure #2: With the patch, the bug is gone
---------------------------------------------

wget -O - https://savannah.gnu.org/patch/download.php?file_id=23309 | patch -p1
make
./make -R -r -f /home/david/work/src/c/make/lab/80-bug/clean/Makefile
bug/clean/Makefile:15:  i
(snip repeated lines)
make: Nothing to be done for `dummy'.


padavo <padavo>
Thu 03 Oct 2013 11:17:34 AM UTC, comment #10: 

### Summary:

* The procedure at comment #9 fixes the bug ! *

### Procedure #1: Without the patch, the bug occurs

apt-get build-dep make
apt-get source make
cd make-dfsg-3.81/
dpkg-buildpackage -nc -us -uc
./make -R -r -f /home/david/work/src/c/make/lab/80-bug/clean/Makefile
* /home/david/work/src/c/make/lab/80-bug/clean/Makefile:15: \\*\* unterminated variable reference.  Stop. **

### Procedure #2: After the patch is applied, the bug is gone

wget -O - https://savannah.gnu.org/patch/download.php?file_id=23309 | patch -p1
make
./make -R -r -f /home/david/work/src/c/make/lab/80-bug/clean/Makefile
* bug/clean/Makefile:15:  i *
* (snip repeated lines) *
* make: Nothing to be done for `dummy'. *

padavo <padavo>
Thu 03 Oct 2013 06:35:14 AM UTC, comment #9: 

I repeated all my tests with both makefiles.  In all tests, they behaved like the original.  So this is a bug that's been fixed here since 3.82.  Debian has taken many patches from here but hasn't taken whichever one fixed this.  Can we say which it is?  Here's one of the valgrind plaints:

==23456== Invalid read of size 1
==23456==    at 0x4C25FF8: memcpy (mc_replace_strmem.c:497)
==23456==    by 0x4058A1: variable_buffer_output (expand.c:72)
==23456==    by 0x405A15: variable_expand_string (expand.c:234)
==23456==    by 0x40610A: allocated_variable_expand_for_file (expand.c:574)
==23456==    by 0x4061EF: recursively_expand_for_file (expand.c:142)
==23456==    by 0x405EC8: variable_expand_string (expand.c:176)
==23456==    by 0x40610A: allocated_variable_expand_for_file (expand.c:574)
==23456==    by 0x4063AA: expand_argument (expand.c:458)
==23456==    by 0x40A8F2: handle_function (function.c:2241)
==23456==    by 0x405AAA: variable_expand_string (expand.c:263)
==23456==    by 0x40610A: allocated_variable_expand_for_file (expand.c:574)
==23456==    by 0x4061EF: recursively_expand_for_file (expand.c:142)
==23456==  Address 0x55dede0 is 16 bytes inside a block of size 17 free'd
==23456==    at 0x4C240FD: free (vg_replace_malloc.c:366)
==23456==    by 0x41CE84: define_variable_in_set (variable.c:228)
==23456==    by 0x41D26F: do_variable_definition (variable.c:1334)
==23456==    by 0x41D4AA: try_variable_definition (variable.c:1521)
==23456==    by 0x416085: eval (read.c:722)
==23456==    by 0x417407: eval_buffer (read.c:459)
==23456==    by 0x4081D9: func_eval (function.c:1374)
==23456==    by 0x40A92D: handle_function (function.c:2273)
==23456==    by 0x405AAA: variable_expand_string (expand.c:263)
==23456==    by 0x40610A: allocated_variable_expand_for_file (expand.c:574)
==23456==    by 0x4061EF: recursively_expand_for_file (expand.c:142)
==23456==    by 0x405EC8: variable_expand_string (expand.c:176)

It's https://savannah.gnu.org/patch/?7534https://savannah.gnu.org/patch/download.php?file_id=23309 applies cleanly to Debian Squeeze's make source and makes the problem go away.  Well, it makes my valgrind symptom go away.  I bet the cause is the same for the OP's problem.  OP should try something like this:

apt-get source make
cd make-dfsg-3.81
dpkg-buildpackage -nc
./make -f .../premature-Makefile -R -r
# check you still have the problem

wget -O - https://savannah.gnu.org/patch/download.php?file_id=23309 | patch -p1
make
./make -f .../premature-Makefile -R -r
# check you don't

If it fixes your symptom too, then I recommend you raise a Debian bug citing this one.

Martin Dorey <mdorey>
Thu 03 Oct 2013 02:55:50 AM UTC, comment #8: 
padavo <padavo>
Thu 03 Oct 2013 01:53:42 AM UTC, comment #7: 

I did a full update of my Debian Wheezy (7.1 stable), which did not affect the previously reported bug symptoms.

Thank you for the instructions at comment #4. Using these I built and tested 3.82 and 3.99.93

Summary:
  3.82 has the same bug symptoms
  3.99 appears bug free.

With both, I tested 'make -R -r' and the makefile in an empty directory, with both variable names 'ooo' (re premature termination) and 'oo' (re garbage characters).

Version 3.99 appears bug free, so I did not try it with gdb. Although that might just be the bug hiding, it is hard to trigger this bug.

Version 3.82 has the same bug behaviour as I reported for 3.81. There are slight differences in the number of lines output before premature termination, and slight differences in the garbage characters in the other case.

Before my initial report, I spent many hours fiddling with my example makefile to create one that behaved the same at the command line and with gdb. Happily this remains true for 3.82 (using args -r -R in both cases).

In case it is useful, I will describe my skill level. I am running several hobbyist machine at home with an ADSL internet connection. I am not scared of C, I have written a few 1000 lines of my own code to do text parsing and calculations, including my own libraries for dynamically sized string buffers.
But that was a few years ago and I am rusty now even when reading my own code. I barely remember the basics of gdb, setting breakpoints.

I wish I was more capable with gdb to be able to report the cause and fix of this bug instead of just the symptoms. If someone wants to provide me some guidance on what to do with gdb, I will be pleased to try.

Also I could consider allowing ssh access if that would assist.

padavo <padavo>
Wed 02 Oct 2013 07:01:56 PM UTC, comment #6: 

Debian's make 3.81-8.2, from Wheezy and Sid, behaves like Squeeze, as does make-3.82 built on Debian via Paul's instructions here.

Martin Dorey <mdorey>
Wed 02 Oct 2013 06:55:22 PM UTC, comment #5: 

On a version (3.99.92) from Git the other day, the example's output ends with:

Makefile:12:   i
Makefile:12:   i
make: Nothing to be done for 'dummy'.

It exits with status zero.  valgrind doesn't notice anything scary.

With the make ("3.81-8") from Debian Squeeze, I get the same output and exit status but valgrind throws off 23 invalid reads.

Martin Dorey <mdorey>
Wed 02 Oct 2013 06:45:07 PM UTC, comment #4: 

I'm not able to reproduce this bug with any of 3.81, 3.82, or the current release candidate for the next stable release.  So, either the problem is fragile enough that it's not reproducible on my system, or else the Debian version of this package has some patches applied which are changing the behavior of GNU make enough that I don't see it.  I don't have access to the Debian version of GNU make.

Testing a different version is quite trivial; just build it:


wget http://ftp.gnu.org/gnu/make/make-3.82.tar.gz
tar xzf make-3.82.tar.gz
cd make-3.82
./configure
make


You do not need to install this version of make.  Just leave it there in that directory.  There's no installation needed for make.

Now run .../make-3.82/make (use the full pathname to the new make you built) and see if you still have the problem.

Paul D. Smith <psmith>
Group administrator
Wed 02 Oct 2013 06:23:56 PM UTC, comment #3: 

Hi Paul.

Thanks for responding! I do not have access to 3.82 at this time.

3.81 is the current version in Debian stable.

I will be interested to hear if anyone tests my example makefile, on 3.81 or later. Do you see anything like what I reported?

padavo <padavo>
Wed 02 Oct 2013 05:52:58 PM UTC, comment #2: 

In case of any confusion, please make a small typo correction in my bug report. There is a 'y' that should be a 'i'.

The correct text is this:

More evidence that bad things are happening internally can be seen by trivially changing the variable name from 'ooo' to 'oo', ie two lines are changed as follows:
define def
$(eval $(call lazy,oo,i))      <------
$(warning $(oo))
endef

padavo <padavo>
Wed 02 Oct 2013 05:47:26 PM UTC, comment #1: 

GNU make 3.81 is over 7 years old.

Have you been able to reproduce this with the current stable version of GNU make, which is available at http://ftp.gnu.org/gnu/make/make-3.82.tar.gz?  Also you might try the current prerelease for the next stable version of GNU make, which you can find here: http://alpha.gnu.org/gnu/make/make-3.99.93.tar.gz

Paul D. Smith <psmith>
Group administrator
Wed 02 Oct 2013 05:39:01 PM UTC, original submission:  

I am seeing a bug with the Gnu Make parsing under certain conditions. It behaves like runtime corruption of internal data structures that is rarely triggered.

On debian 7:
$ dpkg -s make
Architecture: i386
Version: 3.81-8.2

Here is a complete minimal example makefile that I hope will demonstrate it. If the long foreach lines have been line-wrapped by this web interface, please restore them to a single line each as indicated by their newline escapes.

--------------------------------------
.PHONY: dummy
dummy:

# lazy evaluation, refer Mecklenburg "GNU Make" 3ed p189-190
define lazy
  $1 = $$($1_lazy) $$($1)
  $1_lazy = $$(eval $1 := $2)
endef

define def
  $(eval $(call lazy,ooo,i))
  $(warning $(ooo))
endef

$(foreach x,X X X X X X X X ,\
$(foreach y,Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y ,\
$(eval $(call def,A,B))))
--------------------------------------

Running 'make -R -r' with the above makefile gives
Makefile:15:  i
(repeated 100 times followed by)
Makefile:15: * unterminated variable reference.  Stop.

It does not complete all of the foreach expansions, and exits with status=2.

The purpose of the strange foreach arguments is to stimulate this bug to appear both in bash and under gdb. If these are shortened, the bug does not appear.

More evidence that bad things are happening internally can be seen by trivially changing the variable name from 'ooo' to 'oo', ie two lines are changed as follows:
define def
  $(eval $(call lazy,oo,y))
  $(warning $(oo))
endef

Running with this change, the above error does not occur! The foreach completes, and all the strings printed by $(warning $(oo)) include 'i' character as expected.
However several long groups of them have an extra garbage character after the 'i', such as byte 18hex (ASCII cancel character). So something bad is happening there still, even though the exit status is zero.

I also observe that all these symptoms might disappear depending on other apparently random factors such as what files might be present in the current directory, editing comments in the makefile, or running under gdb.

Also this bug likes to hide, it took several hours of trial and error for me to produce this minimal example which consistently demonstrates it in an empty directory. Small edits to the example are likely to make the bug disappear. I hope it actually occurs on your machine. I tested this on two different hardware.

padavo <padavo>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #29290:  Makefile added by padavo (443B - application/octet-stream - Makefile to trigger garbage characters as reported)
file #29289:  Makefile added by padavo (445B - application/octet-stream - Makefile to trigger premature exit "*** unterminated variable reference" )

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mdorey (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by padavo (Submitted the item)
  • -email is unavailable- added by padavo
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-10-03 psmith StatusNone Duplicate
        Open/ClosedOpen Closed
    2013-10-03 padavo Attached File- Added Makefile, #29290
    2013-10-03 padavo Attached File- Added Makefile, #29289
    2013-10-02 padavo Carbon-Copy- Added padavo

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code