bugmake - Bugs: bug #38420, $(realpath ...) doesn't recover...

 
 

bug #38420: $(realpath ...) doesn't recover from signals

Submitted by:  Brian Vandenberg <phantal>
Submitted on:  Tue 26 Feb 2013 10:21:44 PM UTC  
 
Severity: 3 - NormalItem Group: Bug
Status: FixedPrivacy: Public
Assigned to: Paul D. Smith <psmith>Open/Closed: Closed
Component Version: 3.82Operating System: POSIX-Based
Fixed Release: 4.0Triage Status: Small Effort

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Mon 11 Mar 2013 11:01:25 PM UTC, comment #6:

In case anyone attempts to use the posted workaround, the calls to variable_buffer_output can be interrupted which will set errno = EINTR. When this happens, gmake gets into a death spiral calling xrealloc, doubling the amount of memory allocated repeatedly (which makes it impossible for the process not to get interrupted by a child, setting errno = EINTR).

Anonymous
Wed 27 Feb 2013 07:27:55 PM UTC, comment #5:

Paul,

I'm using `realpath` for defensive coding purposes. I'm using a pattern rule of the form:

%.so :
$(if $(realpath $(filter %.o,${^})),,$(warning No dependencies specified for ${@})@false)
recipe goes here

... where the dependencies are specified elsewhere. Three basic things [can/should] trigger this:

  • Mis-spelled library names
  • Forgetting to specify dependencies
  • The dependency's recipe may be screwed up but may not cause the build to fail immediately; consequently the file may not actually exist.

$(shell) would work; but I've managed to cut out all uses of $(shell) inside recipes, which has improved the build times significantly. Given the choice, it sounds like a modified version of gmake would be preferable.

Brian Vandenberg <phantal>
Wed 27 Feb 2013 06:21:05 PM UTC, comment #4:

The only option I can think of is using $(shell ...) to determine the real path rather than the built-in realpath function. This will be slower, but more reliable (as far as I'm aware all the EINTR issues with shell have long since been solved).

Maybe something like: $(shell cd $(dir $<) && pwd)

Or, if you don't really need the full power of realpath you could switch to abspath. That is just a string manipulation function so it doesn't do any system calls that might be interrupted with EINTR.

It really depends on your situation. But there's no way I know of to fix the problem with realpath, without a code change.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Wed 27 Feb 2013 05:44:22 PM UTC, comment #3:

Paul,

You're spot on. It is Solaris / NFS 3.

In the interim I'm using a build with the fix I suggested.

Do you know of a workaround for this that wouldn't require deploying a modified version of make?

Brian Vandenberg <phantal>
Wed 27 Feb 2013 02:49:14 AM UTC, comment #2:

Let me guess, you're working on Solaris? And probably using an NFS-mounted filesystem? See, for example:

http://lists.gnu.org/archive/html/help-make/2006-04/msg00023.html

However, you're right that we should be doing a better job of error handling in these situations: currently we simply assume that if the function doesn't succeed for any reason, the file doesn't exist. In other places we have a macro EINTRLOOP we use to handle this. I'll look into using that here as well.

Paul D. Smith <psmith>
Project AdministratorIn charge of this item.
Tue 26 Feb 2013 10:39:03 PM UTC, comment #1:

Also, need to include errno.h.

Brian Vandenberg <phantal>
Tue 26 Feb 2013 10:21:44 PM UTC, original submission:

func_realpath doesn't check errno at all. When running in parallel builds, if the parent gmake process receives a signal in the middle of calling `realpath` it'll act as though the file does not exist.

Suggested fix:

2028c2017
< while ( errno == EINTR || (path = find_next_token (&p, &len)) != 0)
---

> while ( (path = find_next_token (&p, &len)) != 0)


The above use of || will shortcut and prevent it from pre-maturely grabbing the next token.

I haven't evaluated this to see if something more comprehensive could/should be done (eg, checking other "errno" values, etc).

I haven't finished validating whether this completely resolves the problem, but so far it seems to have done just that. Furthermore, I don't have any ideas for how to implement a test for this as it's a race condition that isn't necessarily easy to reproduce.

I highly recommend evaluating other system functions used by gmake to determine whether this sort of issue could crop up elsewhere (eg, `$(abspath ...)`, etc)

Brian Vandenberg <phantal>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by psmith (Posted a comment)
  • -unavailable- added by phantal (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 28 Feb 2013 05:56:58 AM UTCpsmithItem GroupNone=>Bug
      StatusNone=>Fixed
      Assigned toNone=>psmith
      Open/ClosedOpen=>Closed
      Component VersionNone=>3.82
      Operating SystemNone=>POSIX-Based
      Fixed ReleaseNone=>4.0
      Triage StatusNone=>Small Effort

    Back to the top


    Powered by Savane 3.1-cleanup1