bugmake - Bugs: bug #39934, make closes a wrong FD, results in...

 
 

bug #39934: make closes a wrong FD, results in *** INTERNAL: readdir: Bad file descriptor

Submitted by:  Petr Machata <pmachata>
Submitted on:  Tue 03 Sep 2013 05:44:02 PM UTC  
 
Severity: 3 - NormalItem Group: Bug
Status: FixedPrivacy: Public
Assigned to: Paul D. Smith <psmith>Open/Closed: Closed
Component Version: 3.82Operating System: POSIX-Based
Fixed Release: 4.0Triage Status: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sun 15 Sep 2013 07:21:51 PM UTC, comment #3:

Fixed using part of the change here. I don't want to move all the jobserver stuff before reading makefiles, because that removes the ability to add -j into MAKEFLAGS in the makefile.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Wed 04 Sep 2013 12:11:01 AM UTC, comment #2:

This is consistently reproducible with current git HEAD (40a49f24) as well as 3.82.

Petr Machata <pmachata>
Tue 03 Sep 2013 05:54:33 PM UTC, comment #1:

Just to clarify: you've marked the Component Version as "SCM"; does that mean you've tried the latest source from Git and seen this bug? If not please specify which version you tested this with. Thanks!

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Tue 03 Sep 2013 05:44:02 PM UTC, original submission:

This was filed in Red Hat bugzilla here:
https://bugzilla.redhat.com/show_bug.cgi?id=885474

To reproduce this, use the following Makefile:
default:
/home/petr/src/make/build/make -f Makefile2

And the following Makefile2 (note: the path in vpath must exist):
vpath %.c /home/petr/
foo:

Then run make:
$ make -j 2
/home/petr/src/make/build/make -f Makefile2
make[1]: Entering directory `/home/petr/tmp/tst'
make[1]: warning: jobserver unavailable: using -j1. Add `+' to parent make rule.
make[1]: *** INTERNAL: readdir: Bad file descriptor
. Stop.
make[1]: Makefile2: Field 'stem' not cached: Makefile2

make[1]: Leaving directory `/home/petr/tmp/tst'
make: *** [default] Error 2

The following excerpt from an ltrace run illustrates the problem:
18706 16:46:43.945567 opendir("/home/petr" <unfinished ...>
18706 16:46:43.945713 SYS_open("/home/petr", 591872, 0172204710) = 4
18706 16:46:43.945785 SYS_brk(0) = 0x1e9d000
18706 16:46:43.945824 SYS_brk(0x1ec5000) = 0x1ec5000
18706 16:46:43.945870 <... opendir resumed> ) = { 4 }
[...]
18706 16:46:44.347777 close(4 <unfinished ...>
18706 16:46:44.347882 SYS_close(4) = 0
18706 16:46:44.347927 <... close resumed> ) = 0
[...]
18706 16:46:45.718922 readdir({ 4 } <unfinished ...>
18706 16:46:45.719037 SYS_getdents(4, 0x161f008, 32768) = -9
18706 16:46:45.719103 <... readdir resumed> ) = 0

The problem arises when parental make passes down jobserver FDs in the environment, but the child make is not run as part of this jobsever batch. Child make notices JOB_FDS numbers passed down, and tries to dup, which fails, because JOB_FDS[0] is invalid. In then switches to -j1, and proceeds to close JOB_FDS. But in the meantime make managed to reuse one of those descriptors for a directory stream for vpath. When that is closed it later causes the error message seen in the subject.

The immediate idea is not to close JOB_FDS if the dup fails. That's certainly part of the solution (because otherwise we end up calling close over invalid file descriptors). But that it fails is pure luck: make gets FD 3 for Makefile2, then FD 4 for vpath dirstream, and then closes 3 after it's done with it. Had --jobserver-fds been 4 and 5, we would dup it without trouble.

But perhaps it's possible to move read_all_makefiles down beyond the JOB_FDS business. Then there would be no file operations except those in the dynamic linker, and those don't leak descriptors. Then we can use the dup as a reliable test. However I don't know whether there are interactions that I don't see, and whether it's safe and desirable to postpone the parsing to a later time like this. The test suite passes with this FWIW.

Petr Machata <pmachata>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by pmachata (Submitted the item)
  • -unavailable- added by pmachata
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 15 Sep 2013 07:22:19 PM UTCpsmithStatusNone=>Fixed
      Assigned toNone=>psmith
      Open/ClosedOpen=>Closed
      Component Version4.0=>3.82
      Operating SystemNone=>POSIX-Based
      Fixed ReleaseNone=>4.0
    Tue 03 Sep 2013 05:44:02 PM UTCpmachataAttached File-=>Added 0001-Don-t-close-JOB_FDS-if-we-know-they-are-invalid.patch, #28997
      Carbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup