buggrep - Bugs: bug #48344, Add optional ODF document handling...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

bug #48344: Add optional ODF document handling to grep

Submitter:  Ken Beal <brotherken>
Submitted:  Wed 29 Jun 2016 03:35:00 PM UTC
   
 
Category:  None Severity:  1 - Wish
Item Group:  None Status:  None
Privacy:  Public Assigned to:  meyering
Open/Closed:  Open

Fri 01 Jul 2016 04:38:09 AM UTC, comment #3: 

However, if someone were to invest in a well-written wrapper script that can do what you suggest, I would be happy to consider including it in the grep distribution.

However, note that writing such scripts robustly and portably is tricky. For a good starting point, see the zgrep.in template in the GNU gzip repository: http://git.savannah.gnu.org/cgit/gzip.git/tree/zgrep.in

Jim Meyering <meyering>
Group administrator
Fri 01 Jul 2016 04:29:25 AM UTC, comment #2: 

Thank you for the suggestion, but adding such a feature to grep would open the gates to adding support for tens if not hundreds of other binary formats. What about pdf, docx, and older "word" versions. It doesn't make sense to encumber a fundamental tool like grep with such specialized code.

On the bug/request-submission front, there is no need for an account. You can do that simply by following instructions in grep's man page or in the output of grep --help: send email to -email is unavailable-

Jim Meyering <meyering>
Group administrator
Wed 29 Jun 2016 07:56:03 PM UTC, comment #1: 

Also note that in the sample scripts, the matched portion is discarded.  This isn't the desired behavior; I'd like it to show, as much as is possible, "the line it's on", and to have the "before" and "after" command-line parameters function properly with the new option as well.

So internally, it may need to do a bit more than just "tidy" (although it may be possible to have tidy produce the proper output -- I haven't investigated).

Ken Beal <brotherken>
Group Member
Wed 29 Jun 2016 03:35:00 PM UTC, original submission:  

The ODF document format is not plain-text; it is a compressed file, containing content.xml which has the text in it, in XML format.

There are some simple scripts that were developed (I'll link to and copy them below), but in one the arguments order is changed (file comes first -- so only one file can be specified); in another it only handles files in the current directory; and they're both not as easy to use as having this built in to grep.

I am putting a $50 bounty on fixing this, as well, at bountysource.com (I am "Brother Ken" there, with a space, which wasn't allowed in usernames here; I signed up there first).

The following page contains a writeup and a script:

http://www.techrepublic.com/blog/linux-and-open-source/how-to-search-for-text-inside-many-opendocument-files/

The following thread also has several scripts, developed by the users, and appears to be the source of the previous article's inspiration (the last one, on page 2):

http://ubuntuforums.org/showthread.php?t=899179&page=2

Here's the script from the first article:

function odfgrep(){
FILE=$1
shift
EXT=`echo ${FILE##*.}`
case $EXT in
   odt|ods|odp)
        unzip -p "$FILE" content.xml | tidy -q -xml 2> /dev/null | grep "$@" ;;
   txt|t2t)
        grep  "$@" "$FILE" ;;
   *) echo "Sorry, I don't know what to do with $FILE"
   ;;
esac
}

Here's the final script from the second article:

function odtgrep(){
    term="$1"
    for file in *.odt; do
        unzip -p "$file" content.xml | tidy -q -xml 2> /dev/null | grep "$term";
        if [ $? -eq 0 ]; then
            echo $file;
        fi;
    done
}

The important line in both cases is the one that starts with "unzip"; that's how to handle an ODF file.

This will require a command-line switch to activate this processing, so that grep's behavior continues to be as otherwise expected.  (Then I would set GREP_OPTIONS on my system to include it.)  The parameter name can be whatever is available and fits the existing naming heuristics.

Ken Beal <brotherken>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by meyering (Posted a comment)
  • -email is unavailable- added by brotherken (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2016-07-01 meyering Assigned toNone meyering
    2016-07-01 meyering Severity3 - Normal 1 - Wish

    Back to the top

    Powered by Savane 3.13-4448.
    Corresponding source code