bugmake - Bugs: bug #28134, inconsistent reporting of linux...

 
 

bug #28134: inconsistent reporting of linux kernel builds with -j2

Submitter:  Phil Carmody <fatphil>
Submitted:  Mon 30 Nov 2009 06:39:48 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  3.81 Operating System:  POSIX-Based
Fixed Release:  4.0 Triage Status:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 20 Oct 2013 05:52:10 PM UTC, comment #4: 

In 4.0 we set stdout and stderr to append mode (for systems where this is supported).  So you should not see this problem anymore.  Also in 4.0 there is the output-sync feature you can use if you want (but this is not necessary to avoid losing output).

Paul D. Smith <psmith>
Group administrator
Fri 15 Jan 2010 06:29:30 AM UTC, comment #3: 

As a complete digression, and based on 26 years of doing parallel builds (damn it's been a long time), I just wanted to point out that you can generally use 50% more jobs than your number of CPUs/cores. E.g., for a dual-core machine, use make -j3. Compilation tends to be heavily I/O bound; if you only use as many jobs as you have cores, you will tend to always have at least 30% idle CPU. Bump up the jobs by 50% and you can utilize that extra CPU without overloading the machine. Go beyond that and you tend to lose.

Howard Chu <hyc>
Thu 17 Dec 2009 05:45:52 PM UTC, comment #2: 

Well, that may look a little like voodoo, but it seems to work from the few brief tests I gave it (about 4 kernel builds, I think). It even seems to work in the less-well-behaved >>f 2>&1 case that I use.

It's part of an automated build checker, so if there are problems with it, then they will make themselves clear within a week or so.

Thanks!

Phil Carmody <fatphil>
Thu 17 Dec 2009 10:15:41 AM UTC, comment #1: 

Try >>/tmp/C_out 2>>/tmp/C_err so that append mode is used so that writes to the files from different processes are atomic and don't overwrite each other.

Ralf Wildenhues <rwild>
Mon 30 Nov 2009 06:39:48 PM UTC, original submission:  

This only seems to happen with both parallel builds and with 'sparse' checking (C=1 or C=2). However, it's almost 100% reproducable. The actual build itself seems correct, it's just that stdout and stderr don't tell the whole story about what's being built.

I did make mrproper and a make defconfig to ensure that the system was in an identical test for two builds. (A ls -lR proves that both builds were in an identical clean state.)

Then I used the following command:
make ARCH=arm CROSS_COMPILE=arm-linux- -j2 C=1 V=2 >/tmp/C_out 2>/tmp/C_err

(The V=2 is unnecessary, it just adds more info to the output)
Again I grabbed an ls -lR, cleaned, and made again, capturing stdout and stderr to new files.

I sorted the _out files, and then ran diff:
<pre>
$ diff /tmp/[CD]s_out
872d871
<   CC [M]  net/mac80211/ibss.o - due to target missing
972d970
<   CC      net/ipv4/tcp_timer.o - due to target missing
1294d1291
<   CHECK   drivers/mmc/card/queue.c - due to target missing
1902c1899
<   CHECK   net/ipv4/tcp_timer.c - due to target missing
---

>   CHECK   net/ipv4/tcp_timer.c - due to target missing  CC      net/ipv4/tcp_timer.o - due to target missing

</pre>

As you can see - some output (tcp_timer) is mangled, and some files seem to not be CC'd (ibss.o) or CHECK'ed (queue.c).

However, looking at the final ls -lR, all of the required targets have been build correctly. So for example, ibss.o exists both times.

For my sins, I'm using Ubuntu 9.04 on an Intel Core2Duo system.

Phil Carmody <fatphil>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by hyc (Posted a comment)
  • -email is unavailable- added by rwild (Posted a comment)
  • -email is unavailable- added by fatphil (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-10-20 psmith StatusNone Fixed
        Assigned toNone psmith
        Open/ClosedOpen Closed
        Fixed ReleaseNone 4.0

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code