bugmake - Bugs: bug #38420, $(realpath ...) doesn't recover...

 
 

bug #38420: $(realpath ...) doesn't recover from signals

Submitter:  Brian Vandenberg <phantal>
Submitted:  Tue 26 Feb 2013 10:21:44 PM UTC
   
 
Severity:  3 - Normal Item Group:  Bug
Status:  Fixed Privacy:  Public
Assigned to:  psmith Open/Closed:  Closed
Component Version:  3.82 Operating System:  POSIX-Based
Fixed Release:  4.0 Triage Status:  Small Effort
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 11 Mar 2013 11:01:25 PM UTC, comment #6: 

In case anyone attempts to use the posted workaround, the calls to variable_buffer_output can be interrupted which will set errno = EINTR.  When this happens, gmake gets into a death spiral calling xrealloc, doubling the amount of memory allocated repeatedly (which makes it impossible for the process not to get interrupted by a child, setting errno = EINTR).

Anonymous
Wed 27 Feb 2013 07:27:55 PM UTC, comment #5: 

Paul,

  I'm using `realpath` for defensive coding purposes.  I'm using a pattern rule of the form:

%.so :
  $(if $(realpath $(filter %.o,${^})),,$(warning No dependencies specified for ${@})@false)
  recipe goes here

... where the dependencies are specified elsewhere.  Three basic things [can/should] trigger this:

  • Mis-spelled library names
  • Forgetting to specify dependencies
  • The dependency's recipe may be screwed up but may not cause the build to fail immediately; consequently the file may not actually exist.


$(shell) would work; but I've managed to cut out all uses of $(shell) inside recipes, which has improved the build times significantly.  Given the choice, it sounds like a modified version of gmake would be preferable.

Brian Vandenberg <phantal>
Wed 27 Feb 2013 06:21:05 PM UTC, comment #4: 

The only option I can think of is using $(shell ...) to determine the real path rather than the built-in realpath function.  This will be slower, but more reliable (as far as I'm aware all the EINTR issues with shell have long since been solved).

Maybe something like: $(shell cd $(dir $<) && pwd)

Or, if you don't really need the full power of realpath you could switch to abspath.  That is just a string manipulation function so it doesn't do any system calls that might be interrupted with EINTR.

It really depends on your situation.  But there's no way I know of to fix the problem with realpath, without a code change.

Paul D. Smith <psmith>
Group administrator
Wed 27 Feb 2013 05:44:22 PM UTC, comment #3: 

Paul,

  You're spot on.  It is Solaris / NFS 3.

  In the interim I'm using a build with the fix I suggested.

  Do you know of a workaround for this that wouldn't require deploying a modified version of make?

Brian Vandenberg <phantal>
Wed 27 Feb 2013 02:49:14 AM UTC, comment #2: 

Let me guess, you're working on Solaris?  And probably using an NFS-mounted filesystem?  See, for example:

http://lists.gnu.org/archive/html/help-make/2006-04/msg00023.html

However, you're right that we should be doing a better job of error handling in these situations: currently we simply assume that if the function doesn't succeed for any reason, the file doesn't exist.  In other places we have a macro EINTRLOOP we use to handle this.  I'll look into using that here as well.

Paul D. Smith <psmith>
Group administrator
Tue 26 Feb 2013 10:39:03 PM UTC, comment #1: 

Also, need to include errno.h.

Brian Vandenberg <phantal>
Tue 26 Feb 2013 10:21:44 PM UTC, original submission:  

func_realpath doesn't check errno at all.  When running in parallel builds, if the parent gmake process receives a signal in the middle of calling `realpath` it'll act as though the file does not exist.

Suggested fix:


2028c2017
<  while ( errno == EINTR || (path = find_next_token (&p, &len)) != 0)
---

>  while ( (path = find_next_token (&p, &len)) != 0)



The above use of || will shortcut and prevent it from pre-maturely grabbing the next token.

I haven't evaluated this to see if something more comprehensive could/should be done (eg, checking other "errno" values, etc).

I haven't finished validating whether this completely resolves the problem, but so far it seems to have done just that.  Furthermore, I don't have any ideas for how to implement a test for this as it's a race condition that isn't necessarily easy to reproduce.

I highly recommend evaluating other system functions used by gmake to determine whether this sort of issue could crop up elsewhere (eg, `$(abspath ...)`, etc)

Brian Vandenberg <phantal>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by psmith (Posted a comment)
  • -email is unavailable- added by phantal (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-02-28 psmith Item GroupNone Bug
        StatusNone Fixed
        Assigned toNone psmith
        Open/ClosedOpen Closed
        Component VersionNone 3.82
        Operating SystemNone POSIX-Based
        Fixed ReleaseNone 4.0
        Triage StatusNone Small Effort

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code