bugmake - Bugs: bug #37703, job starts before dependency

 
 

bug #37703: job starts before dependency

Submitted by:  Koen Van Hoof <kvho>
Submitted on:  Fri 09 Nov 2012 01:17:19 PM UTC  
 
Severity: 3 - NormalItem Group: Bug
Status: FixedPrivacy: Public
Assigned to: Paul D. Smith <psmith>Open/Closed: Closed
Component Version: 3.82Operating System: POSIX-Based
Fixed Release: 4.0Triage Status: Verified

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 09 Jul 2013 02:35:37 PM UTC, comment #5:

I am seeing a very similar scenario with 3.82.

I have also downloaded the latest source and see the same issue.

When a very complex parallel build runs, and intermediate archive is generated, and some executables that depend on that archive are built before that parallel job completes. If I immediately reinvoke make with the same command line, it does then relink the executables.

There is no "patch" information in this bug. Can you let me know what the change was so that I can verify that it is in the source code release I am building?

Thanks,

Jay

Jay Lawrence <jaylaw>
Sat 06 Apr 2013 02:30:07 PM UTC, comment #4:

Fixed, thank you for the great repro case!

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Tue 02 Apr 2013 09:58:05 AM UTC, comment #3:

Could this be a duplicate of #26893?

Martijn Vermaat <martijnvermaat>
Wed 16 Jan 2013 01:16:35 PM UTC, comment #2:

The error I got did only happen as part of a much bigger system, and it is not easy to reproduce it in a small example.
But I can try to explain it with a small example.
The attached Makefile has a dependency tree
/x.a -\
all +-x.b x.d - x.e
\x.c -/

With a 'make -j' the following did happen.

1) The chain x.a - x.d -x.e is checked
the flag 'considered' is set for the files x.a x.d x.e
The command for x.e is launched.

2) x.b is checked.
The command for x.b is launched.
Before launching the command, there is a call to reap_children (0, 0)

Here is the situation that makes it hard to reproduce. In my setup, the job of x.e was finished by the time we got here. I only can reproduce this in this small example, by inserting a sleep(1) statement in the function new_job
new_job (struct file *file)
{
struct commands *cmds = file->cmds;
struct child *c;
char **lines;
unsigned int i;

/* Let any previously decided-upon jobs that are waiting
for the load to go down start before this new one. */
start_waiting_jobs ();

->sleep(1);
/* Reap any children that might have finished recently. */
reap_children (0, 0);
...

Within reap_children, the command_state of x.e is set to cs_finished

3) The chain x.c - x.d - x.e is checked
In the function update_file for file x.c, a call is done to update_file for x.d.
This one returns 0 because the flag 'considered' of x.d is already set.
As a result, the commands for x.c are launched, while x.d did not yet run.

Koen Van Hoof <kvho>
Mon 14 Jan 2013 12:51:36 PM UTC, comment #1:

I was not able to reproduce this situation, in either 3.82 or the current latest codebase. You mention "I had to add a 'sleep(1)' in function new_job, before the call to reap_children" but there are multiple calls to reap_children() in new_job() so it's not clear exactly what you did. Please provide more details on how to reproduce.

Thanks!

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Fri 09 Nov 2012 01:17:19 PM UTC, original submission:

I run make -j. In a rare case, a jobs starts before its dependency had run.
It happens if a dependency had finished after it has been checked, but before the update of a target.
With the attached Makefile I could reproduce the problem, but I had to add a 'sleep(1)' in function new_job, before the call to reap_children.
The error is in the attached log.
I have also attached a patch that seem to fix it.

Koen Van Hoof <kvho>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #26895:  Makefile added by kvho (116B - application/octet-stream)
file #26896:  reconsider.patch added by kvho (735B - application/octet-stream)
file #26897:  log added by kvho (3kB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by jaylaw (Posted a comment)
  • -unavailable- added by martijnvermaat (Posted a comment)
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by kvho (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 9 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 06 Apr 2013 02:30:07 PM UTCpsmithStatusNone=>Fixed
      Assigned toNone=>psmith
      Open/ClosedOpen=>Closed
      Fixed ReleaseNone=>4.0
      Triage StatusNeed Info=>Verified
    Mon 14 Jan 2013 12:51:36 PM UTCpsmithTriage StatusNone=>Need Info
    Fri 09 Nov 2012 01:17:19 PM UTCkvhoAttached File-=>Added Makefile, #26895
      Attached File-=>Added reconsider.patch, #26896
      Attached File-=>Added log, #26897

    Back to the top


    Powered by Savane 3.1-cleanup