bugmake - Bugs: bug #40361, make 4.0 under hpux 11.31 no...

 
 

bug #40361: make 4.0 under hpux 11.31 no longer prints output correctly

Submitter:  None
Submitted:  Thu 24 Oct 2013 12:18:54 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  4.0 Operating System:  POSIX-Based
Fixed Release:  4.1 Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 18 Dec 2013 05:41:49 PM UTC, comment #18: 
Anonymous
Sun 24 Nov 2013 03:43:13 AM UTC, comment #17: 

I've reworked the code so that it doesn't require vsnprintf() any longer.  It's kind of gross but it works and is ISO C90 compliant.

Paul D. Smith <psmith>
Group administrator
Fri 08 Nov 2013 05:12:55 AM UTC, comment #16: 

Hello everyone,

Please have a look at my patch for bug #40515. The culprit is vsnprintf().

On HP-UX, this function returns -1 if the buffer is not big enough, and vfmtconcat() wasn't handling that at all.

Daniel Richard G. <iskunk>
Mon 28 Oct 2013 11:21:10 AM UTC, comment #15: 

I noticed Gnulib does explicitly mention that vsnprintf on HP-UX doesn't correctly return a byte count:

http://www.gnu.org/software/gnulib/manual/html_node/vsnprintf.html

Anonymous
Fri 25 Oct 2013 03:32:25 PM UTC, comment #14: 

I would be surprised if it's va_copy().  On the other hand I'm surprised that a UNIX variant in 2013 doesn't comply with a C standard which is 14+ years old now...  I sort of expect that from Windows as they've all but abandoned C, but not a UNIX system.

On the other hand we do try to meet C89 compliance (at least for the standard library and language).  There are two alternative non-standard semantics of vsnprintf() that I'm aware of: one where it returns <0 if the buffer is too small (Windows does it this way) and the other where it returns the number of bytes written and doesn't write the last nul byte (like strncpy).

We may need to switch to gnulib functions or else avoid the *nprintf() variants.  I'll look into this.

Paul D. Smith <psmith>
Group administrator
Fri 25 Oct 2013 02:29:30 PM UTC, comment #13: 

@#12: Yes, exactly.

Now we'll have to find out which one of vsnprintf() and va_copy() is broken and how to work around it.

Are you familiar with C and can you do some tests to find out? (You can try standalone programs based on these functions, they're not much entangled with the rest of the code.)

Frank Heckenbach <frank>
Fri 25 Oct 2013 09:45:48 AM UTC, comment #12: 

Although, slightly odd that the 'Entering directory' appeared after the 'touch file.c' - although I presume that is just related to the temporary code replacement with vprintf causing an immediate print

Anonymous
Fri 25 Oct 2013 09:44:13 AM UTC, comment #11: 

Bingo!


$ /opt/tools/gnumake-4.0/bin/make -rR
/opt/tools/gnumake-4.0/bin/make -C subdir
touch file.c
make[1]: Entering directory '/tmp/temp/subdir'
rm file.c
make[1]: Leaving directory '/tmp/temp/subdir'


So Frank is correct, it looks like vfmtconcat() is the cause of these problems on HP-UX

Anonymous
Thu 24 Oct 2013 10:43:46 PM UTC, comment #10: 

Another thing that changed in the course of the output patches is that messages are now generated in vfmtconcat() using vsnprintf() as well as va_copy(), both C99 and at least the latter AFAIK not very widely used.

For testing you could try replacing the body of vfmtconcat() with "vprintf (fmt, args);". (Keep the return statement to avoid a compiler warning; the returned value isn't actually used.) This will break output-sync, of course, but if the messages are correct then, we've found the culprits.

Frank Heckenbach <frank>
Thu 24 Oct 2013 09:58:20 PM UTC, comment #9: 

If you run "make -rRn" does it still print the bogus lines and not print the "real" lines?

Can you try running "make -rRpn" and check the database and see if the recipes shown in the output look correct, or if they look corrupted?  Since the right commands are being invoked I'm pretty sure this will look OK but it's worth checking.

Paul D. Smith <psmith>
Group administrator
Thu 24 Oct 2013 09:41:56 PM UTC, comment #8: 

Hrm.  Not much of interest here.  What I'm beginning to think is that there's some kind of memory corruption or something, so that when make thinks it's printing out the command that it's about to execute it's instead printing out some other random string buffer.  I'm not sure that it's really related to the output changes in the end.

That would definitely explain this, since there's no place anywhere in the GNU make code that we ever print "$(IFS)" for example, and the output we get (three different output strings) is the same as the number of command lines make would print.

However, I've run make under valgrind etc. and not found any memory issues.  So this is very much a mystery to me still.

Paul D. Smith <psmith>
Group administrator
Thu 24 Oct 2013 08:14:16 PM UTC, comment #7: 

Sure. Updated tusc.log attached with -p -v added



(file #29466)

Anonymous
Thu 24 Oct 2013 06:37:12 PM UTC, comment #6: 

Wow, that is extremely bizarre.  I just have no idea where that output is coming from, or going to.  Can you re-run and add the "-p" option so that each line of output has a PID next to it?  I wonder if something else is printing those values.  I assume you did run it with "-f" and "-o" flags already (I see that there is a handy example specifically for "make" in the tusc man page :-p :-))

Also you might try adding "-v".  I don't know if the extra verbosity will be useful or not.

Paul D. Smith <psmith>
Group administrator
Thu 24 Oct 2013 02:32:47 PM UTC, comment #5: 

Removing those set_append_mode() calls didn't appear to make any difference.

I've attached a log from tusc (the HP-UX equivalent of strace/truss etc.) showing the invocations during the sample I described in the first post - perhaps that may help you to diagnose

(file #29459)

Anonymous
Thu 24 Oct 2013 01:38:27 PM UTC, comment #4: 

Yep the output files are created succesfully, so it is definitely just the output messages that are not getting through. I haven't tried 3.82, but I can do.

I have succesfully completed a large full product build with 4.0, and all the output was successfully generated, but the make.log was similarly very spartan to the reduced testcase I posted here.

I'll take a look at your suggested patches to output_init() shortly and get back to you.

On a related note, `make check` fully passed on 3.81, but fails on most tests on 4.00 due to the missing stdout messages.

Anonymous
Thu 24 Oct 2013 01:12:51 PM UTC, comment #3: 

Given that there are no env.var. settings that are odd, I'd agree with you.  You show results for 3.81; have you tried 3.82?

I find it very strange, the output you do get.  I have no idea where something like that would come from, at all.  In fact your makefile commands are simple enough that make won't even invoke the shell: it will simply fork/exec those programs directly.  If you remove the "rm file.c" from the rule, is "file.c" created?  That is, is make actually invoking the commands just not printing the output?  If you add a command like "echo hi", do you see the output printed ("hi")?

One thing you might try is to go to output.c and find the function output_init(), and comment out the two invocations of "set_append_mode()", one for stdout and one for stderr.  Maybe there's something broken about the way HPUX sets append mode on open file descriptors.

Paul D. Smith <psmith>
Group administrator
Thu 24 Oct 2013 12:46:22 PM UTC, comment #2: 

I'm happy to grab whatever debug that may be required.

`env | grep MAKE` shows that no make-related environment variables have been set.

I'm guessing this regression is most likely to have been caused during the consolidation of output generators into output.c perhaps something c99 that HP haven't implemented correctly?

Anonymous
Thu 24 Oct 2013 12:33:38 PM UTC, comment #1: 

I'm not sure what to say about this: clearly this doesn't happen on the systems we have access to (Linux, Solaris, Mac OS, Windows).  I, at least, don't have any access to an HPUX system. Without being able to reproduce it ourselves we'll need someone to do the investigation.  Probably easiest would be for you (or someone who can reproduce this issue) to email the -email is unavailable- mailing list with details so we can have a discussion.

My first suggestion would be to run "env | grep MAKE" and see if there are any unusual make-related environment variables.

Paul D. Smith <psmith>
Group administrator
Thu 24 Oct 2013 12:18:54 PM UTC, original submission:  

Easily demonstrable with simple pair of Makefiles

Built for ia64-hp-hpux11.31


.
├── Makefile
└── subdir
    └── Makefile

$ cat Makefile
all:
        $(MAKE) -C subdir

$ cat subdir/Makefile
all:
        touch file.c
        rm file.c

$ /opt/gnumake-3.81/bin/make -rR
/opt/gnumake-3.81/bin/make -C subdir
make[1]: Entering directory `/tmp/a/subdir'
touch file.c
rm file.c
make[1]: Leaving directory `/tmp/a/subdir'

$ /opt/tools/gnumake-4.0/bin/make -rR
$(IFS)make[1]: Entering directory '/tmp/a/subdir'
/bin/sh/bin/shmake[1]: Leaving directory '/tmp/a/subdir'


Notice the '/bin/sh' entries printed for the two rules and the odd $(IFS) printing as well

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #29466:  tusc.log added by None (126KiB - text/x-log - tusc.log of GNU Make 4.0 invocation (updated to include -p -v))
file #29459:  tusc.log added by None (86KiB - text/x-log - tusc.log of GNU Make 4.0 invocation)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by iskunk (Posted a comment)
  • -email is unavailable- added by frank (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-11-24 psmith StatusNone Fixed
        Assigned toNone psmith
        Open/ClosedOpen Closed
        Fixed ReleaseNone 4.1
    2013-10-24 None Attached File- Added tusc.log, #29466
    2013-10-24 None Attached File- Added tusc.log, #29459

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code