bugfindutils - Bugs: bug #22099, Enhancement proposal: SQL-like...

 
 

bug #22099: Enhancement proposal: SQL-like interface to Find

Submitter:  Richard Neill <richardneill>
Submitted:  Thu 24 Jan 2008 04:28:49 AM UTC
   
 
Category:  find Severity:  1 - Wish
Item Group:  None Status:  Wont Fix
Privacy:  Public Assigned to:  jay
Originator Name:  Richard Neill Open/Closed:  Closed
Release:  None Fixed Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sun 22 Jun 2008 10:40:05 AM UTC, comment #2: 

Nobody commented or volunteered.

James Youngman <jay>
Group administrator
Fri 25 Jan 2008 05:02:41 PM UTC, comment #1: 

Somebody already implemented this with XPath instead of SQL.

http://uucode.com/texts/xfind/

Unfortunately, while there were some interesting ideas in there, the code was not designed in such a way that I could introduce a findutils/find/contrib/far-out-concepts directory in the code base and optionally build another executable; instead, the existing functionality of "find" was done away with and replaced with XPath.

It might be interesting to devise some kind of plugin interface.  I think this would have precede any of the more radical ideas for different query languages etc.  One reason for that is that the main find executable needs to remain fairly small, since it's usually part of the mandatory core of GNU/Liux systems.

It's an interesting idea though.  I supose this makes a more obvious addition to "locate" which already has an external database.  

I'm going to mark this bug as "Won't Fix".  Please don't take this as an outright rejection of your idea.  The thing is we'd need a generic interface for attaching this kind of extension before we even began to think about it, and the "right" design for such a plugin interface is not obvious at all.

I'll leave the bug open for a while in case people who stop by the bug database want to comment (or volunteer...).

James Youngman <jay>
Group administrator
Thu 24 Jan 2008 04:28:49 AM UTC, original submission:  

Here's an idea which suddenly struck me today. I was considering creating a small database as a way to help track certain files. Of course, that's what a filesystem is (even more so, if Reiser4 happens). So, it's a bit redundant to use a database. On the other hand, SQL is a very nice language for this sort of query. Hence my idea: implement an sql interface to find.

An example of how this might work is the following:

find --sql "SELECT * FROM directory WHERE NAME ILIKE '%abc&' AND SIZE > 15 MB  AND (date > 2007-12-30 or date < 2005-01-01) AND MIMETYPE == 'x/ogg' AND METADATA Contains 'xyz'"

This would basically consist of:
  i) a simple parser to convert between sql-like syntax, and the native syntax used by find

 ii) some extra criteria implemented, such as
      - match MIME-type  (should be easy, use `file`)
      - match (regex) CONTENTS  (grep through the file)
      - match (regex) part of CONTENTS (eg search the first 512
          and last 512 bytes for a string)
      - METADATA   (for certain files, eg jpg, ogg, mp3...
          look at the id3,jhead etc tags)
      - SUBDIRECTORY DEPTH == n   (eg max of 2 subdirs deep)
     

 (iii) it's only necessary to support SELECT, since other operations (DELETE etc) can be done as normal by the exec{} function.


I think this way of doing file searches, by scanning rather than indexing is quite useful for a certain category of workloads on modern hardware. Also, it would be really useful to be able to find based on a few more criteria.

Anyway, just an idea - feel free to use it, or ignore it.

HTH, and thanks for everything - Richard




Richard Neill <richardneill>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jay (Posted a comment)
  • -email is unavailable- added by richardneill (Submitted the item)
  • -email is unavailable- added by richardneill
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2008-06-22 jay Open/ClosedOpen Closed
    2008-01-25 jay Severity3 - Normal 1 - Wish
        StatusNone Wont Fix
        Assigned toNone jay
    2008-01-24 richardneill Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code