bugfindutils - Bugs: bug #12162, Enhancement req: finding files...

 
 

bug #12162: Enhancement req: finding files less than 2Gb in size [needs community feedback]

Submitted by:  James Youngman <jay>
Submitted on:  Sun 27 Feb 2005 11:11:33 AM UTC  
 
Category: findSeverity: 1 - Wish
Item Group: NoneStatus: Need Info
Privacy: PublicAssigned to: James Youngman <jay>
Originator Name: AmirOpen/Closed: Open
Release: 4.1.20Fixed Release: None

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 15 Aug 2012 05:32:13 PM UTC, comment #27:

James, sure, would get to convoluted here otherwise. I reported the time stuff as bug #37110 find: alternative time handling. I think its important to keep times somewhere in mind in order to be able to find a solution that work consistently between sizes and times. Thanks, Martin

Martin Steigerwald <martin21>
Wed 15 Aug 2012 03:36:15 PM UTC, comment #26:

Could we discuss time handling in a separate bug please?

James Youngman <jay>
Project AdministratorIn charge of this item.
Wed 15 Aug 2012 03:05:03 PM UTC, comment #25:

-[a|c|m]time eq 1d makes no sense with nanosecond precision cause it hits extremely rarely. I´d skip -eq for times. Searching exact filesizes make more sense.

-[a|c|n]time eq "exact absolute time spec" makes more sense, cause search for an exact time doesn't seem to be possible at all at the moment as -[c|a|]newer goes with more or less (with !) but not equal

An option for timespec to limit resolution may be good to not have to type in the exact nanosecond timestamp.

Martin Steigerwald <msteamix>
Wed 15 Aug 2012 02:49:44 PM UTC, comment #24:

Updated time results for consistency comparison:

ms@mango:/tmp/find-times> ls -l
insgesamt 0
-rw-rw-r-- 1 ms teamix 0 Aug 14 18:11 lessthan1dayago
-rw-rw-r-- 1 ms teamix 0 Aug 13 18:12 lessthan2daysago
-rw-rw-r-- 1 ms teamix 0 Aug 12 18:12 lessthan3daysago
-rw-rw-r-- 1 ms teamix 0 Aug 14 16:11 morethan1dayago
-rw-rw-r-- 1 ms teamix 0 Aug 13 16:11 morethan2daysago
-rw-rw-r-- 1 ms teamix 0 Aug 12 16:11 morethan3daysago
ms@mango:/tmp/find-times> date
Mi 15. Aug 16:29:51 CEST 2012
ms@mango:/tmp/find-times> find -mtime +1 -printf "%p\t %t\n" | sort
./lessthan3daysago Sun Aug 12 18:12:10.0626806028 2012
./morethan2daysago Mon Aug 13 16:11:42.0727680142 2012
./morethan3daysago Sun Aug 12 16:11:50.0715653807 2012

As predicted -mtime +1 translated to more than two days ago.

ms@mango:/tmp/find-times> find -mtime +3 -printf "%p\t %t\n" | sort

+3 translates to more than 4 days ago, thus no results

ms@mango:/tmp/find-times> find -mtime +2 -printf "%p\t %t\n" | sort
./morethan3daysago Sun Aug 12 16:11:50.0715653807 2012

And +2 to more than 3 days ago.

Yet with -time it seems to be the other way around:

ms@mango:/tmp/find-times> find -mtime -2 -printf "%p\t %t\n" | sort
./lessthan1dayago Tue Aug 14 18:11:33.0444291877 2012
./lessthan2daysago Mon Aug 13 18:12:00.0302854716 2012
./morethan1dayago Tue Aug 14 16:11:23.0258103147 2012
. Wed Aug 15 16:12:10.0625203012 2012

-2 translates to less than 2 days ago (as I actually wrote before)

ms@mango:/tmp/find-times> find -mtime -1 -printf "%p\t %t\n" | sort
./lessthan1dayago Tue Aug 14 18:11:33.0444291877 2012
. Wed Aug 15 16:12:10.0625203012 2012

-1 to less than one day ago.

Thus we have:

-mtime -1: less than one day
-mtime 1: one day, 24-48 or <48 hours
-mtime +1: more than two days!

And:

-size -1M: 0 MBytes = 0 bytes
-size -2M: less or equal than 1 MiB!
-size +1M: more than 1 MiB

I'd suggest:

-size lt 1M: less than one MiB (< 1048576 bytes, 1048575 matches 1048576 does not match)
-size lte 1M: less than or equal one MiB (<= 1048576 bytes)
-size gt 1M: more than one MiB (> 1048576 bytes)
-size gte 1M: more or equal than one MiB (>= 1048576 bytes)
-size eq 1M: exactly one MiB (=1048576 bytes)

And for completeness for times as well:

-mtime lt 1d: less than one day (00:00:00 to 23:59:59.9999999999999, short < 24:00:00)
-mtime lte 1d: less or equal one day (00:00:00 to 24:00:00)
-mtime gt 1d: more than one day (24:00:00.0000000000001, short >24:00:00 to anything)
-mtime gte 1d: more or equal one day (24:00:00 to anything)
maybe -mtime eq 1d: exactly one day (24:00:00.000000000000)

Old semantic for -mtime 1 can be more clearly expressed by:

-mtime gte 1d -and -mtime lt 2d

Advantages:
1) a lot more intuitive to human beings

2) distinction between < and <= as well as > and >=

3) compatible with old scripts, cause old behavior can be kept (I actually teach the -[a|c|m]time wierdness since quite some time)

4) no conflict with input/output redirection

5) different units possible for times as well (d = days, h = hours, m=minutes, M=months, Y=years and such or even have it verbose to reduce ambiquity)

I am not so sure about the usefulness / semantics of the "eq" case, especially for times. For sizes it might make more sense. But then exactly matching a time could be useful.

I happily put the time stuff in an extra report and link to it from here. But I thought the comparison with time handling makes the inconsistency in find behaviour from the viewpoint of human beings even more obvious.

The current handling of -size / -[a|c|m]time IMHO is well suited for machines (integer math with any, even big units), but not for humans.

What do you think?

Martin

Martin Steigerwald <msteamix>
Wed 15 Aug 2012 02:24:43 PM UTC, comment #23:

Forget the time results. touch sets access and modification time, not status change time. I test again with mtime.

Martin Steigerwald <msteamix>
Wed 15 Aug 2012 02:20:32 PM UTC, comment #22:

I have:

mango:/tmp# find -name "nullen*" -printf "%p %s\n" | sort
./nullen0999 1022976
./nullen1000 1024000
./nullen1001 1025024
./nullen1024 1048576
./nullen1025 1049600
./nullen2047 2096128
./nullen2048 2097152
./nullen2049 2098176

Yet while

mango:/tmp# find -size +1M -size -3M -printf "%p %s\n" | sort
./nullen1025 1049600
./nullen2047 2096128
./nullen2048 2097152

Does not show up nullen2049 cause find rounds -3M to next integer unit <=2M. Yet find doesn't round up +1M to >=2M. I think this behavior is quite unpredictable and inconsistent.

Especially as -[a|m|c]mtime exactly behaves that way even for "+" – well at least in my last test it did.

But unless I am grossly off track this doesn't work at all ATM:

ms@mango:/tmp/find-times> touch -d "1 day ago" morethan1dayago
ms@mango:/tmp/find-times> touch -d "22 hours ago" lessthan1dayago
ms@mango:/tmp/find-times> touch -d "2 days ago" morethan2daysago
ms@mango:/tmp/find-times> touch -d "3 days ago" morethan3daysago
ms@mango:/tmp/find-times> touch -d "46 hours ago" lessthan2daysago
ms@mango:/tmp/find-times> touch -d "70 hours ago" lessthan3daysago

ms@mango:/tmp/find-times#2> ls -l --time=ctime
insgesamt 0
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:11 lessthan1dayago
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:12 lessthan2daysago
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:12 lessthan3daysago
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:11 morethan1dayago
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:11 morethan2daysago
-rw-rw-r-- 1 ms teamix 0 Aug 15 16:11 morethan3daysago

ms@mango:/tmp/find-times> date
Mi 15. Aug 16:16:26 CEST 2012

ms@mango:/tmp/find-times> find -ctime -2 -printf "%p\t %c\n" | sort
./lessthan1dayago Wed Aug 15 16:11:33.0439805365 2012
./lessthan2daysago Wed Aug 15 16:12:00.0298369109 2012
./lessthan3daysago Wed Aug 15 16:12:10.0625203012 2012
./morethan1dayago Wed Aug 15 16:11:23.0256899464 2012
./morethan2daysago Wed Aug 15 16:11:42.0723161276 2012
./morethan3daysago Wed Aug 15 16:11:50.0711165207 2012
. Wed Aug 15 16:12:10.0625203012 2012

Note how find display day as 15 August on every file. find -ls shows correct results.

Hmmm, but thats a different bug it seems, that I will report in a moment.

Still last I checked with -exec ls -ld {} \; -ctime behaved as follows:

- ctime 1: 24-48 hours earlier
- ctime -1: 0-24 hours earlier
- ctime +1: >=48 (!) hours earlier

So I suggest new time options as well. I'd like the aliasing idea with -size => -blocksize.

Or maybe "gt" and "lt" could serve for new fractional comparing? Like in "test" command. But thats > and <, not <= and =>. I think the semantics which one find uses should also be clarified.

Martin Steigerwald <msteamix>
Sun 27 May 2012 09:49:48 PM UTC, comment #21:

I'd just like to cast another vote for the view that the behavior of the -size flag is incorrect. Though I have read and understood the arguments in favor of maintaining the current behavior, I don't see any way around the following dilemma. Either:

1) -size is broken because of the rounding behavior, or

2) -size correctly implements a different search criterion (i.e. size by blocks), and therefore find lacks a flag for search by file size.

Although I would be mildly surprised to learn that anyone has written code which relies on the rounding behavior, there's surely no harm in adding another flag, (e.g. -filesize as Jay Youngman suggested) which has the semantics that 99.9% of users expect?

I find myself in sympathy with xeo's incredulity that a behavior that has bitten this many people since (at least) 2005 has not been remedied by any of the Pareto-optimal, compatability-preserving means available, like adding another flag that does the right thing. Can I ask what would constitute sufficient evidence that fixing this behavior (or remedying it with another flag) would be endorsed by the community?

Best regards,
Patrick.

Patrick O'Neill <poneill>
Thu 05 Aug 2010 10:28:21 AM UTC, comment #20:

Hello all,

I currently stumbled over the "feature", that filesizes are obviously rounded up before comparing.

So, I wanted to find all files smaller than 1MB and I - naivly - assumed that a

find . -type f -size -1M

would do the job, but instead of returning me every file less than 1MB, it gave me all files with exactly 0Byte back.

Apparently, -1MB means "less than 1MB" which means 0MB and everything rounded up which is >0Byte is 1MB, so there was no hit at all.

I thought, this would be a bug but surprisingly the community is aware of it and hasn't done something to fix it (the last 25yrs?) - so it has to be a feature.

Wouldn't it be possible to fix this behaviour?

I realized, this thread is 5 years old and I can't believe this topic is sooo difficult that there is absolutely no agreement possible ;-)

I agree with Peter Splichal, who suggested "--1MB" for disabling rounding up ...

Best Regards,
Xeo

Thomas P <xeo>
Mon 12 Apr 2010 12:17:50 AM UTC, comment #19:

Just call it filesize. Sure, it might not apply to just files, but that's the name everyone will be looking for.

Nigel McNie <nigel>
Sun 11 Apr 2010 10:10:32 PM UTC, comment #18:

I like Nigel's suggestion of -filesize.

i suggest adding "c" as a synonym for "b."

I further suggest that all the units be made case-insensitive.

I like Nigel's further suggestion about -size and -blocksize. Also, print a warning to stderr if the user uses "-size" with any unit other than "c".

Thanks for thinking about this.

dan pritts <danpritts>
Sun 11 Apr 2010 07:17:22 PM UTC, comment #17:

Ole is right; the <, > method won't work well. But we seem to have overlooked the idea of creating a new predicate which is similar to -size but does no rounding.

Any preferences for what we should call the test?

-exactsize
-length
-size.norounding
-size-norounding
-size unrounded -1G
...?

James Youngman <jay>
Project AdministratorIn charge of this item.
Thu 12 Nov 2009 08:38:21 PM UTC, comment #16:

I ran into this bug today. Having read the comments, may I make a suggestion: Add a -filesize option, which works the way everyone clearly expects.

-filesize n[bkMGTP]
File uses n bytes of space. The following suffixes can be used:
'b' for bytes
'k' for Kilobytes (1024 bytes)
'M' for Megabytes (1048576 bytes)
'G' for Gigabytes (1073741824 bytes)
'T' for Terabytes (...)
'P' for Petabytes (...)

For bonus points: add -blocksize, alias -size to -blocksize (so people reading the manpage can tell the difference between the two easily), and change the wording for the help -blocksize to refer to blocks to make it obvious that people don't want to do that when they want -filesize.

This preserves backward compatibility, while making 'find' a whole bunch more useful.

Nigel McNie <nigel>
Wed 11 Mar 2009 03:15:04 PM UTC, comment #15:

If my daughter's Rottweiler weighs more pounds than she, then changing
the units to metric (kilograms) does not imply she has suddenly put on
weight and now magically weighs the same as her dog.

If her 3 foot 8 inches seems small, when compared with the height of a
skyscraper, she nevertheless remains small by comparison even when the
units are changed from inches to lightyears.

It seems to me that the fundamental problem here is with the ambiguity
of "units". The maintainer evidently -- and rightly so -- perfers the
simplicity, utility, and consistency of the current implementation. I
believe that is justified, given that "units" refers to "granularity".

If the precision of measurement is so extremely coarse-grained as to
prevent a distinction in height between my daughter and Mount Everest,
it may be reasonable to deem their heights the same. The problem with
the find command is... well... there is no problem; there is a problem
with the documentation however. The documentation should explicitly
point out how "units" refers to granularity, to how coarse-grained
the measurements are. Moreover the succinct clarification provided by
Andreas Metzler should -- in my opinion -- be in the documentation.

vose <vose>
Tue 10 Mar 2009 08:36:02 PM UTC, comment #14:

I don't believe the behavior is correct, but here's the line of reasoning.

THe original find implementation counts disk blocks. A block is either used (partially or fully), or not.

Blocks are (typically? always?) 512 bytes long.

You are looking for blocks that are used, and therefore cannot be used by other files, which is the key distinction why this makes sense for blocks but not for megabytes or whatever.

So if you search for size -2, you are not really searching for anything that is using less than 1024 bytes, you are searching for something that is using 1 full block, and no portion of a second block.

If you apply the same logic to a different unit, gigabytes, you will end up with the situation where a file of 1 gigabyte + 1 byte is using a portion of the second gigabyte, therefore the test fails.

This is obviously the wrong logic to use when units are something other than disk blocks, but that's how it got this way.

The maintainer suggests in the comments below that changing this might break backward compatibility. Let's just say that I strongly disagree.

dan pritts <danpritts>
Tue 10 Mar 2009 06:13:42 PM UTC, comment #13:

"Would someone please explain -- completely and carefully -- just
exactly how find's behavior is actually correct?"

Step one: How many 1M blocks does the 3200 byte file take? Answer: one. (Clearly it cannot be zero.)
Step two: Is one stricly lower than 1? No it is not.
Test failed.

Andreas Metzler <ametzler>
Tue 10 Mar 2009 04:50:21 PM UTC, comment #12:

As near as I can tell, find is broken when it comes to -size.
It fails to realize that a 3200 byte file has a size which is
less than 1M, as evidenced by the failure of ... -size -1M ...

Reading comments here leads me to believe: whereas other people
are likewise confused, nevertheless somehow the behavior of
find is thought in some sense to be correct.

Would someone please explain -- completely and carefully -- just
exactly how find's behavior is actually correct? It does not
seem to me that the man page (or info) adequately explains this
subtlety.

Thanks in advance.

vose <vose>
Tue 14 Oct 2008 01:23:44 PM UTC, comment #11:

I see the "logic" of units, but this really isn't very
intuitive behaviour, which is confirmed by the fact
that more and more people run into this issue. Today it
was me:

https://bugzilla.redhat.com/show_bug.cgi?id=466462

I think the best solution now really is to provide an
alternative modifier for size parameter (not to break
backward compatibility) as has been already mentioned
in previous comments.

As we do not have a big choice as far as available
chars are concerned, we should choose better something
than nothing. I vote for using -- and ++ which do not
have to be escaped and (hopefully) will not break the
findutils' test suite.

++ and -- does not seem that bad after all...

find -size --1M
find -size ++1M

the only other that I can think of are _ and ^

find -size _1M
find -size ^1M

but I don't like them much...

If the testsuite could be modified so that < and > are
acceptable, I would probably vote for them because of
their clear meaning (despite they have to be escaped).

Petr Splichal <psss>
Mon 18 Aug 2008 10:16:20 PM UTC, comment #10:

While I was at it I checked Solaris 10.

Their find command does not have any units other than blocks & characters.

The -size N behaves like gnu find, ie, 513-byte files are not shown if you ask for -size -2

-size -1024c, however, works like FreeBSD.

dan pritts <danpritts>
Mon 18 Aug 2008 10:11:37 PM UTC, comment #9:

FreeBSD 7 find also works the way i'd want it to, and the man page snippet is identical to the one from MacOS I quote below.

thanks for taking the time to think about this.

dan pritts <danpritts>
Mon 18 Aug 2008 04:13:19 PM UTC, comment #8:

MacOS find behaves in the way i'd expect, at least in the obvious case of -size -1G;

It has this in the man page:

-size n[ckMGTP]
True if the file's size, rounded up, in 512-byte blocks is n. If
n is followed by a c, then the primary is true if the file's size
is n bytes (characters). Similarly if n is followed by a scale
indicator then the file's size is compared to n scaled as:

k kilobytes (1024 bytes)
M megabytes (1024 kilobytes)
G gigabytes (1024 megabytes)
T terabytes (1024 gigabytes)
P petabytes (1024 terabytes)

I've asked a bsd-head friend for a login on his box and will report back what i find.

dan pritts <danpritts>
Sat 16 Aug 2008 08:10:16 AM UTC, comment #7:

How about clarifying the documentation? I have been asked for this in Debian bug #489905. Patch attached.

Does anybody know how FreeBSD's find behaves?

(file #16299)

Andreas Metzler <ametzler>
Tue 12 Aug 2008 08:11:24 PM UTC, comment #6:

Due to the nature of filesystem disk allocation and use of blocks,
I guess i can imagine why someone who's searching on blocks would
be sad if his block search showed him files that were using that
many blocks, so the (undocumented?) rounding requirement may fit with the principle of least surprise.

For any other unit it seems horribly broken. Maybe the right solution is to leave the rounding logic for searches on blocks, but do as anonymous in comment #3 suggests and make the k/M/G suffixes act as multiplers of "c".

dan pritts <danpritts>
Tue 12 Aug 2008 07:43:47 PM UTC, comment #5:

I ran into this bug today. The degenerate case was that "find -size -1G" only matches files zero bytes in size. I am really having a hard time imagining who in their right mind would want this.

I find this behavior to be horribly broken. Maybe this deserves to be a different bug but I'd put the severity at the highest level.

I think I understand what you're getting at w/r/t the "rounding requirement" but i don't see it documented in the info page anywhere. the "info" program is user-hostile and I don't use emacs, so i suppose i may have not been able to find it, but i certainly found other uses of the word "rounding."

the documentation refers to "rounding" in the discussion of size, but only states that the sizes are rounded up ("true if the file uses N units of space, rounding up").

James, your comment reminds me of the old joke about "putting the backward in backward-compatible."

regarding the comment about < and > I agree that it would not be a good thing, users would be hosed by it. -- and ++ don't really seem like very good alternatives either but I don't have any better suggestions.

dan pritts <danpritts>
 Spam posted by an anonymous
Sat 17 Nov 2007 02:27:20 PM UTC, comment #4:

Anonymous, the problem with your suggestion that "-size -2147483648c" and "-size -2G" should be completely identical is that it would be a backward-incompatibel change.

James Youngman <jay>
Project AdministratorIn charge of this item.
Sat 15 Sep 2007 04:14:36 PM UTC, comment #3:

IMO the solution is obvious, the letters k,M,G,T,P,E,Z,Y are NOT units they are SI multipliers, they multiply the base unit which in this case is bytes. so the stanzas "-size -2147483648c" and "-size -2G" should be completely identical. I would suggest a suffix perhaps like "-size 2G/4k" so you can find all files that can be put on a filesystem with a blocksize of 4k and a max file size of just under 2GiB.

Anonymous
Thu 30 Aug 2007 03:25:16 PM UTC, comment #2:

In general < and > are bound to cause problems for normal users. Could we use ++ and -- instead?

Ole Tange <tange>
Fri 04 Mar 2005 10:25:58 AM UTC, comment #1:

I've written the attached patch which implements the feature but the way that the test suite works prevents us passing "<" or ">" as part of an argument to "find". Hence we can't include test cases for this feature. That being the case, I'm not going to apply the patch. Maybe we can come up with some alternative syntax that avoids this problem.

James Youngman <jay>
Project AdministratorIn charge of this item.
Sun 27 Feb 2005 11:11:33 AM UTC, original submission:

At the moment find . -size -2G will not find a file which is 1Gb in size, because of the rounding requirement (i.e. that find . -size -2 should not match a 750 byte file). The same rounding up works for -size arguments where the multiplier is much larger. The problem is that one would ordinarily assume that "-size -2G" would match a 1.5Gb file. Hence perhaps we should introduce "-size <2G" which works the way the user might expect.

James Youngman <jay>
Project AdministratorIn charge of this item.

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #16299:  12162.diff added by ametzler (1kB - text/x-diff - fix docs.)
file #2660:  find-4.2.19-CVS-sizerounding-sv12162.patch added by jay (6kB - text/x-patch - -size rounding patch implementing the desired feature)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by martin21 (Posted a comment)
  • -unavailable- added by martin21
  • -unavailable- added by msteamix (Posted a comment)
  • -unavailable- added by poneill (Posted a comment)
  • -unavailable- added by xeo (Posted a comment)
  • -unavailable- added by nigel (Posted a comment)
  • -unavailable- added by vose (Posted a comment)
  • -unavailable- added by psss (Posted a comment)
  • -unavailable- added by danpritts (Posted a comment)
  • -unavailable- added by jay (Posted a comment)
  • -unavailable- added by tange (Posted a comment)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 8 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 15 Aug 2012 02:55:25 PM UTCmartin21Carbon-Copy-=>Added martin21
    Sun 11 Apr 2010 07:17:22 PM UTCjayStatusNone=>Need Info
      SummaryEnhancement req: finding files less than 2Gb in size=>Enhancement req: finding files less than 2Gb in size [needs community feedback]
    Tue 10 Mar 2009 06:14:05 PM UTCametzlerCarbon-CopyRemoved 20807=>-
    Tue 14 Oct 2008 06:23:13 PM UTCametzlerCarbon-CopyRemoved 20807=>-
    Sat 16 Aug 2008 08:10:16 AM UTCametzlerAttached File-=>Added 12162.diff, #16299
    Fri 04 Mar 2005 10:25:58 AM UTCjayAttached File-=>Added find-4.2.19-CVS-sizerounding-sv12162.patch, #2269
    Sun 27 Feb 2005 11:11:33 AM UTCjayCarbon-Copy-=>Added -unavailable-

    Back to the top


    Powered by Savane 3.1-cleanup