bugfindutils - Bugs: bug #20594, allow ./configure to choose...

 
 

bug #20594: allow ./configure to choose default arg size limit

Submitter:  Eric Blake <ericb>
Submitted:  Thu 26 Jul 2007 12:36:53 PM UTC
   
 
Category:  xargs Severity:  1 - Wish
Item Group:  None Status:  Fixed
Privacy:  Public Assigned to:  ericb
Originator Name:  Eric Blake Open/Closed:  Closed
Release:  4.1.20 Fixed Release:  4.3.9
* Mandatory Fields

Add a New Comment Rich Markup
   

Mon 30 Jul 2007 12:42:00 PM UTC, comment #2: 

cygwin is an interesting beast.  From cygwin's point of view, there really is no limit when invoking another cygwin app; so sysconf(_SC_ARG_MAX) should either return -1 for unlimited, or start enforcing the same limit as Windows in spite of the fact that no limit is necessary.  In cygwin 1.5.24, the sysconf returned an arbitrary 1 meg (which matches neither of the two reasonable values I would have expected).  But it looks like the cygwin developers will change cygwin 1.7.0 to report ARG_MAX (32k) instead of 1 meg: http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/sysconf.cc.diff?r1=1.46&r2=1.47&cvsroot=src&f=h

So I agree with your sentiments that the patch does not solve the xargs problem for cygwin, and that, for now, that I will just continue patching the cygwin package of findutils manually until cygwin 1.7.0 is released.  Meanwhile, I will wait to see if Leslie's patch to auto-reduce the size on failure makes a difference.

However, as pointed out in the findutils-patches thread, the problem of fine-tuning the default size is independent of the cygwin limitation that prompted me to open the bug, and can be considered useful in its own right.  I will reply to that thread; I think there is still utility in applying something that lets a user tune the default argument size at ./configure time, so long as we are clear that it is a feature on its own, and not merely a workaround for a cygwin-specific limitation.

Eric Blake <ericb>
Group administrator
Thu 26 Jul 2007 07:00:31 PM UTC, comment #1: 

I don't think this is the correct solution. 

I would expect that the correct solution is for sysconf(_SC_ARG_MAX) to return 32K on the affected system, dspite the fact that there are circumstances where larger command lines will work.   (I concede that this is probably a QOI issue and that POSIX appears not to forbid what the Cygwin implementation is currently doing).

Making the change in configure means that the user compiling the software needs to know about this implementation limit.  It seems to me that it is the job of the implementation to tell the application what the relevant limits are.

Longer term, the changes that Leslie Polzer is working on shoud allow findutils to automatically deduce that the correct limit is 32K, even if it had earlier assumed that it was 128K.

James Youngman <jay>
Group administrator
Thu 26 Jul 2007 12:36:53 PM UTC, original submission:  

xargs and find have a default maximum argument size that controls how many bytes are collected before spawning a child process, when no other means is used to change it (such as xargs -s).  In lib/buildcmd.c, bc_use_sensible_arg_max, this value is hardcoded to 128k.

It would be desirable to let this default be chosen at ./configure time, so it can be tuned to the system.  For example, on cygwin, there is a 32k limit when invoking a Windows-native process, but unlimited when invoking a cygwin process.  Setting the default to 32k instead of 128k at configure time would make xargs usable for all applications, rather than just cygwin applications, without having to always pass -s to xargs.  When bundling the cygwin package for findutils, I have had to edit source code to make this change to the default value; but it would be much nicer to build the package out of the box merely with a ./configure change.

I'm working on a patch.

Eric Blake <ericb>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by ericb (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2007-11-11 jay Open/ClosedOpen Closed
        Fixed ReleaseNone 4.3.9
    2007-08-01 ericb Severity2 - Minor 1 - Wish
        Item GroupWrong result None
        StatusIn Progress Fixed
    2007-07-26 jay Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code