bugmake - Bugs: bug #37703, job starts before dependency

 
 

bug #37703: job starts before dependency

Submitter:  Koen Van Hoof <kvho>
Submitted:  Fri 09 Nov 2012 01:17:19 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  3.82 Operating System:  POSIX-Based
Fixed Release:  4.0 Triage Status:  Verified
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 09 Jul 2013 02:35:37 PM UTC, comment #5: 

I am seeing a very similar scenario with 3.82.

I have also downloaded the latest source and see the same issue.

When a very complex parallel build runs, and intermediate archive is generated, and some executables that depend on that archive are built before that parallel job completes. If I immediately reinvoke make with the same command line, it does then relink the executables.

There is no "patch" information in this bug. Can you let me know what the change was so that I can verify that it is in the source code release I am building?

Thanks,

Jay

Jay Lawrence <jaylaw>
Sat 06 Apr 2013 02:30:07 PM UTC, comment #4: 

Fixed, thank you for the great repro case!

Paul D. Smith <psmith>
Group administrator
Tue 02 Apr 2013 09:58:05 AM UTC, comment #3: 

Could this be a duplicate of #26893?

Martijn Vermaat <martijnvermaat>
Wed 16 Jan 2013 01:16:35 PM UTC, comment #2: 

The error I got did only happen as part of a much bigger system, and it is not easy to reproduce it in a small example.
But I can try to explain it with a small example.
The attached Makefile has a dependency tree
     /x.a -\
all +-x.b   x.d - x.e
     \x.c -/

With a 'make -j' the following did happen.

1) The chain x.a - x.d -x.e is checked
the flag 'considered' is set for the files x.a x.d x.e
The command for x.e is launched.

2) x.b is checked.
The command for x.b is launched.
Before launching the command, there is a call to reap_children (0, 0)

Here is the situation that makes it hard to reproduce.  In my setup, the job of x.e was finished by the time we got here.  I only can reproduce this in this small example, by inserting a sleep(1) statement in the function new_job
new_job (struct file *file)
{
  struct commands *cmds = file->cmds;
  struct child *c;
  char **lines;
  unsigned int i;

  /* Let any previously decided-upon jobs that are waiting
     for the load to go down start before this new one.  */
  start_waiting_jobs ();

->sleep(1);
  /* Reap any children that might have finished recently.  */
  reap_children (0, 0);
  ...

Within reap_children, the command_state of x.e is set to cs_finished

3) The chain x.c - x.d - x.e is checked
In the function update_file for file x.c, a call is done to update_file for x.d.
This one returns 0 because the flag 'considered' of x.d is already set.
As a result, the commands for x.c are launched, while x.d did not yet run.

Koen Van Hoof <kvho>
Mon 14 Jan 2013 12:51:36 PM UTC, comment #1: 

I was not able to reproduce this situation, in either 3.82 or the current latest codebase.  You mention "I had to add a 'sleep(1)' in function new_job, before the call to reap_children" but there are multiple calls to reap_children() in new_job() so it's not clear exactly what you did.  Please provide more details on how to reproduce.

Thanks!

Paul D. Smith <psmith>
Group administrator
Fri 09 Nov 2012 01:17:19 PM UTC, original submission:  

I run make -j.  In a rare case, a jobs starts before its dependency had run.
It happens if a dependency had finished after it has been checked, but before the update of a target.
With the attached Makefile I could reproduce the problem, but I had to add a 'sleep(1)' in function new_job, before the call to reap_children.
The error is in the attached log.
I have also attached a patch that seem to fix it.

Koen Van Hoof <kvho>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #26895:  Makefile added by kvho (116B - application/octet-stream)
file #26896:  reconsider.patch added by kvho (735B - application/octet-stream)
file #26897:  log added by kvho (3KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jaylaw (Posted a comment)
  • -email is unavailable- added by martijnvermaat (Posted a comment)
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by kvho (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-04-06 psmith StatusNone Fixed
        Assigned toNone psmith
        Open/ClosedOpen Closed
        Fixed ReleaseNone 4.0
        Triage StatusNeed Info Verified
    2013-01-14 psmith Triage StatusNone Need Info
    2012-11-09 kvho Attached File- Added Makefile, #26895
        Attached File- Added reconsider.patch, #26896
        Attached File- Added log, #26897

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code