bugmake - Bugs: bug #17873, .NOTPARALLEL enhancements


bug #17873: .NOTPARALLEL enhancements

Submitted by:  Dave Yost <yost>
Submitted on:  Wed 27 Sep 2006 06:25:32 PM UTC  
Votes:  131  
Severity: 3 - NormalItem Group: Enhancement
Status: NonePrivacy: Public
Assigned to: NoneOpen/Closed: Open
Component Version: 3.81Operating System: Any
Fixed Release: NoneTriage Status: None

Add a New Comment (Rich MarkupRich Markup):

You are not logged in

Please log in, so followups can be emailed to you.


(Jump to the original submission Jump to the original submission)

Sat 20 Oct 2012 09:19:49 PM UTC, comment #16:

Another use case for this would be parallel tests where some of the tests are using OpenMP threads. It would be great to be able to say that specific targets take all (or whatever OMP_NUM_THREADS is set to) job slots.

Reinis Danne <abadar>
Fri 06 Jul 2012 03:19:10 PM UTC, comment #15:

I would very much like to see an enhancement along the lines of patch #5108. When building GCC, all of the compiler executables end up getting linked at about the same time. Just linking one is a significant burden on the system, but trying to link compilers for five different languages at the same time makes it impossible to do anything else on the system.

As mentioned in comment #7, trying to serialize these links via dependencies means lying to make and therefore doing unnecessary and undesirable work when you only want to link a single executable.

I note that patch #5108 seems to create a single global mutex, whereas the documentation for the SCO .MUTEX target suggests that each occurrence of .MUTEX creates a separate mutex, so that given

.MUTEX: this that
.MUTEX: yours mine

"this" and "yours" could be built in parallel, but not "this" and "that". I'm not sure that this is how SCO make works, but it seems useful to have multiple mutexes. Hard disk load is not the only resource that you might want a mutex to control: comment #8 gives another example, of latex temporary output files.

Jason Merrill <jasonm>
Thu 31 May 2012 04:50:00 AM UTC, comment #14:

Another scenario wherein sequential processing of dependencies is useful:

all: subdirs $(TARGETS)

(cd dir1 && make)
(cd dir2 && make)

and the $(TARGETS) depend upon libraries (.a files) that are
made by dir1 and/or dir2. If you do this with make -j, then
make immediately evaluates each $(TARGET) before the "make subdirs" gets a chance to touch/update any of the .a
files. Make therefore decides that nothing is out of date regarding the link steps, and so each link node in the build tree gets retired without doing anything. Issuing a second "make" command later, however, causes it to re-link the $(TARGETS) because now one or more .a files is newer than the corresponding $(TARGETS).

It is a pain to have to introduce time-stamp-files just to
force the links to wait until after the "subdirs" has
finished. Phony targets (e.g., install, subdirs, etc.)
often tend to involve recipes that need to be done serially
-- especially when stringing several together as dependencies
of another phony target. The previous example of:

reinstall: uninstall install

is precisely this sort of "dependency encoded sequential recipe".

The syntax is negotiable. But so long as -j exists, there will be a need to specify series/parallel constraints within dependency lists. The alternatives (e.g., time stamp files) are simply too horrible to contemplate.

Some would argue that the antidote to the above scenario is this:

$(TARGETS) : subdirs

But this gets rid of the ability to do "make target1" without forcing make to also do the "subdirs". In some cases the developer knows that nothing in the huge sub-tree of subdirs has changed and all he/she needs to do is re-link the local executable. The $(TARGETS) : subdirs kludge effectively eliminates this capability.

David Warme <davidwarme>
Wed 30 Nov 2011 06:54:32 PM UTC, comment #13:

> I don't see any functionality asked for in this bug, that cannot be emulated with order-only prerequisites. What am I missing?

One situation where I could use such a feature is to implement, e.g., a "reinstall" target like this:

reinstall: uninstall install

This works well with serial make (given suitable uninstall and install targets), but fails with parallel make, of course.

I don't see how to emulate this with order-only prerequisites. If I do:

reinstall: install | uninstall

both are still run in parallel.

If I do:

reinstall: uninstall install
install: | uninstall

then reinstall works as it should, but making just install will now also make uninstall first. That's not how I expected order-only prerequisites to work, but apparently they do. Maybe it's because the targets are phony.

Am I missing/misunderstanding something? Otherwise, I think such an extension would be useful (whatever the syntax).

Frank Heckenbach <frank>
 Spam posted by petividi
 Spam posted by jackbenny
 Spam posted by jackbenny
 Spam posted by jackbenny
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by sub103
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
(Why is this post is considered to be spam? Users may have reported it to be spam or, if it has been recently posted, it may just be waiting for spamchecks to be run.)
Spam content:

Below the waist https://myweb.liu.edu/sonic/viewtopic.php?p=7342 slightly flared hips hinted at approaching womanhood and https://myweb.liu.edu/sonic/viewtopic.php?p=7342 long slender legs tapered up to a beautiful little heart shaped arse.
It was elegant and classy, which was just perfect for her; the velvety material would caress https://myweb.liu.edu/sonic/viewtopic.php?p=7348 curves.
Tammy reached down and wrapped https://myweb.liu.edu/sonic/viewtopic.php?p=7369 warm hand around my shaft and slowly began stroking it.
He asked Tammy where https://myweb.liu.edu/sonic/viewtopic.php?p=7383 purse was, and demanded my wallet, which https://myweb.liu.edu/sonic/viewtopic.php?p=7383 gave him.
After a few minutes, Mandy looked up into http://www.pspvideo9.com/forums/viewtopic.php?p=268556 face, and http://www.pspvideo9.com/forums/viewtopic.php?p=268556 saw tears streaming down http://www.pspvideo9.com/forums/viewtopic.php?p=268556 cheeks.
Mandy batted http://www.pspvideo9.com/forums/viewtopic.php?p=268559 eyes and purred, “Well, http://www.pspvideo9.com/forums/viewtopic.php?p=268559 for one cannot wait to show http://www.pspvideo9.com/forums/viewtopic.php?p=268559 what I’d like to buy when http://www.pspvideo9.com/forums/viewtopic.php?p=268559 go shopping.
He quickly zipped http://www.pspvideo9.com/forums/viewtopic.php?p=268563 pants up and hurried toward the living room.
As the memory of Rebecca flashed through http://www.pspvideo9.com/forums/viewtopic.php?p=268596 head, http://www.pspvideo9.com/forums/viewtopic.php?p=268596 shivered.
His heart skipped a beat; http://www.wikio.com/user/redtube/bio whole body trembled with desire---no with love, as http://www.wikio.com/user/redtube/bio just realized that http://www.wikio.com/user/redtube/bio too loved Mandy.
http://www.wikio.com/user/tube8/bio watched http://www.wikio.com/user/tube8/bio go and slip behind some chairs and then http://www.wikio.com/user/tube8/bio was gone leaving http://www.wikio.com/user/tube8/bio slightly puzzled as my view had been slightly obscured at the vital moment.
Mandy’s fingers searched for http://www.wikio.com/user/pornhub/bio cock as http://www.wikio.com/user/pornhub/bio tantalizes http://www.wikio.com/user/pornhub/bio pussy.
‘Mmmm, that’s nice’ http://redtube.u.yuku.com/ said and moved closer to me, moving http://redtube.u.yuku.com/ hand on to my thigh and imitating my movements.
As http://tube8.u.yuku.com/ cooked the steaks on the gas grill, Mandy began to prepare, a salad, steamed broccoli, and French bread.
On the final go, http://pornhub.u.yuku.com/ was to hide again and this time http://pornhub.u.yuku.com/ searched pretty close to the youngsters.
She fought back more tears, because http://zootube.u.yuku.com/ really did miss http://zootube.u.yuku.com/ mother.
She kissed http://youporn.u.yuku.com/ cheek and squealed, “ http://youporn.u.yuku.com/ , I’d love to.
” http://youpornos.u.yuku.com/ 's cock hits bottom and http://youpornos.u.yuku.com/ trembled while watching http://youpornos.u.yuku.com/ move herself all the way up just holding http://youpornos.u.yuku.com/ cock head inside her.
”She smirked and said, “Well if http://yuvutu.u.yuku.com/ does guess http://yuvutu.u.yuku.com/ can do some more fucking huh!” http://yuvutu.u.yuku.com/ ’s eyes twinkled with lust and http://yuvutu.u.yuku.com/ lay on http://yuvutu.u.yuku.com/ back.
The tops of http://youjizz.u.yuku.com/ milk filled tits were barely contained in the bra cups.
Never in http://pornotube.u.yuku.com/ wildest dreams did http://pornotube.u.yuku.com/ think, after Rebecca died, would ever find love again.
“Are http://xtube.u.yuku.com/ ready for http://xtube.u.yuku.com/ honey?”Mandy licks http://xtube.u.yuku.com/ lips, tries to remain calm and replies, “Sure Dad, come on in.
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
Mon 23 Jun 2008 07:28:33 PM UTC, comment #12:

> What am I missing?

The chance to buy cheap viagra? ;-)


Mon 23 Jun 2008 04:54:26 PM UTC, comment #11:

I don't see any functionality asked for in this bug, that cannot
be emulated with order-only prerequisites. What am I missing?

Ralf Wildenhues <rwild>
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
Fri 04 Apr 2008 04:34:22 PM UTC, comment #10:

..oops, went too soon. I was going to say "only have to enter a basic plaintext captcha to posta anon, so it could be easily defeated either manually or programmatically - I think it's always 421.

(DaveK again)

Fri 04 Apr 2008 04:32:34 PM UTC, comment #9:

Nope, not hax0red, it just allows anyone to add comments without logging in. You only have to enter a

Presumably there haven't been enough serious enough spam attacks yet to make anyone want to do anything about it, so bug maintainers are just deleting the occasional ones that show up as they come across them.


Thu 03 Apr 2008 10:19:55 PM UTC, comment #8:

Because even the most correct dependencies fail to account for programs/scripts that use conflicting temporary/auxiliary files. For example, latex and pdflatex both use the same basename.aux files given the same source file (and with different contents) so they should not be run at the same time. (I frequently build pdf/dvi/ps this way, with .NOTPARALLEL.)

... and what's with these polluted spam URLs, is the system hax0red?

David Fang <fangism>
 Spam posted by an anonymous
 Spam posted by an anonymous
 Spam posted by an anonymous
Thu 12 Oct 2006 01:52:38 PM UTC, comment #7:

To answer Howard Chu: in very simple makefiles, dependencies could take care of the serial/parallel problem, but for complex makefiles they simply don't suffice.

Consider, for example, a makefile that builds 3 independent executables, exe1, exe2, and exe3. The only way to allow for the object files to be compiled in parallel, while linking the executables serially (linking is IO bound and massive parallel linking can lock up a machine), would be to introduce fake dependencies of this sort:

exe1: exe2
exe2: exe3

This is all good (if you forget for now that these dependencies are completely bogus and thus a hack) as long as you always want to build everything. But since the dependencies are fake, problems arise easily. In this particular case, even if all you want to do is build 'exe1', you are forced to build exe2 and exe3 due to the bogus dependencies. Namely,

make exe1

Will also build exe2 and exe3 for absolutely no good reason. Considering that some debug builds of executables I work on end up being 600MB, you can easily waste close to an hour on this, depending on the number of executables. A .NOTPARALLEL with decent semantics (not the current one) solves this problem the way it is supposed to be solved.

Cesar Crusius <cesarcrusius>
Thu 12 Oct 2006 01:44:30 PM UTC, comment #6:

I find the proposed solution confusing, in that it uses a dependency to affect how other dependencies on the same list are built. As far as I know that would be the only use of such semantics in make.

I'd prefer a solution on the lines of patch #5108, as it is more consistent with current behavior. It is also consistent with the solutions adopted by other makes (such as SCO make). Unfortunately .NOTPARALLEL is somewhat stillborn as it is a global switch, and some makefiles may be using it, so we'd have to use some other special target.

Cesar Crusius <cesarcrusius>
Thu 12 Oct 2006 01:33:27 PM UTC, comment #5:

Correction: similar to patch #5108, not bug.

Cesar Crusius <cesarcrusius>
Thu 28 Sep 2006 09:16:51 PM UTC, comment #4:

Similar to bug #5108

Dave Yost <yost>
Thu 28 Sep 2006 08:12:21 PM UTC, comment #3:

Haven't been paying a lot of attention, but when is such a lame thing nas .NOTPARALLEL needed? Why aren't people just writing proper dependencies so that parallel/serial is irrelevant?

Howard Chu <hyc>
Thu 28 Sep 2006 08:03:56 PM UTC, comment #2:


If .NOTPARALLEL is mentioned as a prerequisite .. built serially in the order given, even if ...

Dave Yost <yost>
Thu 28 Sep 2006 07:54:36 PM UTC, comment #1:

This proposal allows us to use make -j not only without worry but also with as much concurrency as possible, on any of a set of makefiles that have been appropriately annotated.

BTW, the doc section is:

Dave Yost <yost>
Wed 27 Sep 2006 06:25:32 PM UTC, original submission:

Today all a makefile can do to influence parallel execution is to mark the entire makefile as unsafe for -j by mentioning .NOTPARALLEL as a target.

Finer control is needed. The fix should go something like the following.

If .NOTPARALLEL is mentioned as a target, then by default the prerequisites of all targets in this makefile will be built serially, even if the `-j' option is given. If .NOTPARALLEL is mentioned as a prerequisite of a target, then subsequent prerequisites of that target will be built serially, even if the '-j' option is given (and even if .PARALLEL was mentioned previously in this prerequisite list). (See also 5.4 Parallel Execution and 5.7.3 Communicating Options to a Sub-make.)

If .PARALLEL is mentioned as a prerequisite of a target, then subsequent prerequisites of that target will be built in parallel if the '-j' option is given, even if .NOTPARALLEL is mentioned as a target (or was mentioned previously in this prerequisite list). (See also 5.4 Parallel Execution and 5.7.3 Communicating Options to a Sub-make.)

Dave Yost <yost>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

No files currently attached


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -unavailable- added by abadar (Posted a comment)
  • -unavailable- added by abadar
  • -unavailable- added by jasonm (Posted a comment)
  • -unavailable- added by davidwarme (Posted a comment)
  • -unavailable- added by frank (Posted a comment)
  • -unavailable- added by petividi (Posted a comment)
  • -unavailable- added by jackbenny (Posted a comment)
  • -unavailable- added by sub103 (Posted a comment)
  • -unavailable- added by rwild (Posted a comment)
  • -unavailable- added by fangism (Posted a comment)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 131 encouragements so far.

    Only logged-in users can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sat 20 Oct 2012 09:19:49 PM UTCabadarCarbon-Copy-=>Added abadar
    Thu 12 Oct 2006 01:53:07 PM UTCcesarcrusiusCarbon-Copy-=>Added cesarcrusius
    Wed 27 Sep 2006 10:55:46 PM UTCyostCarbon-Copy-=>Added yost

    Back to the top

    Powered by Savane 3.1-cleanup