bugfindutils - Bugs: bug #23742, updatedb tries to index all...

 
 

bug #23742: updatedb tries to index all mounted network shares when running on Cygwin

Submitter:  None
Submitted:  Mon 30 Jun 2008 08:24:55 PM UTC
   
 
Category:  updatedb Severity:  3 - Normal
Item Group:  Wrong result Status:  Invalid
Privacy:  Public Assigned to:  ericb
Originator Name:  Jason Spiro Originator Email:  -email is unavailable-
Open/Closed:  Closed Release:  4.4.0
Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 27 Aug 2008 01:56:05 PM UTC, comment #10: 

In an email which he didn't send using the bugtracker :-), Eric wrote:

> I've uploaded a new build of findutils for cygwin 1.5.x (use cygwin's
> setup.exe to install it), which incorporates my same workarounds for
> getmntent as I used in 'df -T'.  That means that -printf "%F" now shows
> 'system,remote' or 'user,remote' for remote drives, so updatedb has also
> been patched to add these to its default PRUNEFS.  The patch should not be
> floated upstream, since cygwin 1.7.0 does not need it (since getmntent
> gives saner filesystem names).


I tested the new version[1], and when I run the command "updatedb", it no longer tries to index network shares.  I thank you all for kindly maintaining findutils, and Eric, thanks for this fix.

^ [1]. ftp://ftp.cygwin.com/pub/cygwin/release/findutils/findutils-4.4.0-3.tar.bz2

Jason Spiro <jasonspiro>
Tue 26 Aug 2008 12:28:19 PM UTC, comment #9: 

I'm not sure what (if anything) I should do for findutils on cygwin 1.5.x, since most of the focus on the cygwin lists these days is preparing for cygwin 1.7.0.  But such a patch will be cygwin-specific, so it is more appropriate to discuss it on the cygwin list.  On the other hand, for coreutils on cygwin 1.5.x, I added a patch to gnulib's mountlist.c that used a Windows API to determine whether drives are local or remote (witness cygwin's behavior on df -T).  I guess I can just rebuild cygwin's find to reuse that same gnulib patch.  I'll see if I can get to that today; but followup to this should be on the cygwin list, not here, as the problem is NOT a bug in upstream findutils.  So I'm closing this bug report as invalid (it is not an upstream bug, just something that tickles a bug in the cygwin-specific port).

For the (as-yet-unreleased) cygwin 1.7.0, the improvements in cygwin's getmntent should obviate the need for a cygwin-specific patch to findutils.

Eric Blake <ericb>
Group administrator
Tue 26 Aug 2008 10:12:15 AM UTC, comment #8: 

I'll leave that to Eric, since he understands much more about Cygwin than I do.   Eric, if you wouldn't mind summarising things here, I'd be grateful.

James Youngman <jay>
Group administrator
Mon 25 Aug 2008 11:12:13 PM UTC, comment #7: 

I was wrong; no patch got dropped.  Cygwin has indeed accepted a patch[1] which makes Cygwin report filesystem types more accurately.  Hopefully that patch will fix this updatedb problem.  The patch was accepted into the MAIN branch of CVS.  I don't feel like downloading and testing Cygwin 1.7.  But when Cygwin takes the Cygwin 1.7 branch out of beta and releases it to the world, hopefully this bug will be fixed.

Jay, it would be great if you could change this bug's status to "Waiting" or some such.  Maybe leaving it marked Invalid will make it hard for people to find it using Savannah bug search.  You might also want to rename the bug to "updatedb tries to index all mounted network shares because non-beta Cygwin reports filesystem types incorrectly".

^ [1].  http://thread.gmane.org/gmane.os.cygwin.patches/3244/focus=3285

Jason Spiro <jasonspiro>
Tue 22 Jul 2008 02:56:35 PM UTC, comment #6: 

It looks like someone submitted a patch to Cygwin last year to fix this, but it accidentally got dropped.  Today I opened a bug report at http://sourceware.org/bugzilla/show_bug.cgi?id=6760 asking them to apply the patch.  --Jason

Anonymous
Thu 10 Jul 2008 08:04:08 AM UTC, comment #5: 

Like this: updatedb --prunepaths='/cygdrive/c/cygwin /tmp /cygdrive/c/System.Volume.Information' --localpaths='/cygdrive/c' --netuser=nixda

Anonymous
Thu 10 Jul 2008 08:03:03 AM UTC, comment #4: 

For other sufferers: I managed to get it not to index network drives by setting --netuser to a non-existing user.

Anonymous
Sat 05 Jul 2008 11:06:52 AM UTC, comment #3: 

Thanks for your careful research into the problem.   I just took a look at http://cygwin.com/problems.html and can't see a way to enter this problem into a bug-reporting system other than their mailing list.

I'm not able to reproduce the problem myself, develop a fix or test a suggested fix, because I have no access to systems running any version of Windows (and hence, no access to Cygwin).

I don't propose though to leave this open as a bug in findutils; the problem isn't really in findutils, the problem is that Cygwin is returning inappropriate values in getmntent() results.

I'm not attempting to prevent you getting help, I just can't give it.  I will leave this bug open for a short period so that the denizens of the Cygwin mailing list can refer to it when responding to your bug report.

James Youngman <jay>
Group administrator
Fri 04 Jul 2008 06:05:26 PM UTC, comment #2: 


> What does find -printf '%F\n' print for files on such 'network
> shares'? Does the updatedb script exclude such filesystems?


It prints the word "system" for all such files.  The updatedb script excludes nfs, proc, smbfs, and many others, but not "system".

James, I did more research into this updatedb bug.  It is caused by a Cygwin bug.  The Cygwin system calls return untruthful information about filesystem types.  See http://thread.gmane.org/gmane.os.cygwin/14500/focus=14591 for details.  IMO, instead the system calls should return "ntfs" or "vfat" or "iso9660" for local drives.  Also, they should return "smbfs" or "ncpfs" or "nfs" for mounted network shares (depending whether they are mounted from SMB/CIFS or NetWare or Unix NFS file servers).

The cygwin bug was last discussed in 2004; Igor Pechtchanski said back then ( http://thread.gmane.org/gmane.os.cygwin/51465/focus=51466 ) that the bug was still on his TODO list and still unfixed.

So I guess this updatedb bug is caused by a Cygwin bug, and you might as well keep waiting until the Cygwin bug is fixed.

Or you could poke the Cygwin people to fix the Cygwin bug.

Or you could fix the Cygwin bug yourself; I personally don't mind the Cygwin bug that much that I would fix it.  :)

Or you could modify updatedb to prune the paths /cygdrive/h through /cygdrive/z by default, unless users specify a different prune option.  The advantage would be that most mounted network shares would be skipped, so updatedb would be much faster.  The disadvantage would be that if someone had more than 5 mounted local volumes, only C: through G: would be indexed.  That could frustrate users who don't know about the disadvantage; so updatedb should probably display a notice to all Cygwin users saying that any Windows drives with letters H: through to Z: will not be indexed.

Cheers, --Jason

Anonymous
Tue 01 Jul 2008 05:08:56 AM UTC, comment #1: 

What does find -printf '%F\n' print for files on such 'network shares'?     Does the updatedb script exclude such filesystems?

In general though, you are likely to get better support for the Cygwin port of GNU tools on the Cygwin mailing list.

James Youngman <jay>
Group administrator
Mon 30 Jun 2008 08:24:55 PM UTC, original submission:  

When running on Cygwin (a Unix emulation layer for Windows), updatedb tries to index all available network shares which have been "mapped" (mounted) from within Windows Explorer.  When the shares contain tens or hundreds of thousands of files, this is extremely slow.

If you need help learning Windows so you can reproduce this issue, please contact me by email; or phone me at 613-668-6096.

To reproduce this issue, you will need:
- a client PC connected to a LAN and running MS Windows
- another computer (either running Linux and Samba, or running Windows) acting as an SMB or CIFS file server.
(You can find all these things at many workplaces and university computer labs.)

Prerequisites:
- Sit down at the client PC, and map the network share to a Windows drive letter.  (Double-click "My Computer" then go to Tools -> Map Network Drive.)
- Install Cygwin on the client PC.  It is available from www.cygwin.com.  When the installer starts, just keep clicking Next until installation is finished.  If you have a high-speed Internet connection, the download-and-install procedure won't take long.
- Start the Cygwin bash shell.  The icon will be on your start menu.

Steps to reproduce:
- run updatedb
- type the following command to view most of the locate database:
        locate / | grep -v "/cygdrive/c"

Expected results:
- updatedb should not have indexed any attached network shares.  (I know this because the manpage documentation for --netpaths says "Default is none".)

Actual results:
- updatedb indexed all attached network shares

Details:
I am using the Cygwin package "findutils 4.4.0-1" on the Cygwin platform.  (uname -a reports "CYGWIN_NT-5.1 [...] 1.5.25(0.156/4/2) 2008-05-26 20:37 i686 Cygwin".)

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ericb (Posted a comment)
  • -email is unavailable- added by jasonspiro (Posted a comment)
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2008-08-26 ericb Open/ClosedOpen Closed
    2008-08-26 jay Assigned tojay ericb
    2008-07-05 jay Item GroupNone Wrong result
        StatusNone Invalid
        Assigned toNone jay

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code