bugGNU Octave - Bugs: bug #54064, Behavior of open with unknown or...

 
 

bug #54064: Behavior of open with unknown or non-existing files

Submitter:  Guillaume <gyom>
Submitted:  Tue 05 Jun 2018 03:36:07 PM UTC
   
 
Category:  GUI Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  In Progress Assigned to:  None
Originator Name:  Guillaume Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 06 Jun 2018 01:25:31 PM UTC, comment #6: 

I made an equivalent implementation (see attached) with Qt's mimedatabase class and it doesn't look as good as libmagic. Images are well recognized, data files not so well:


fmt = -v7
text/plain

fmt = -binary
application/octet-stream

fmt = -text
text/plain

fmt = -zip
application/gzip

fmt = -hdf5
application/x-hdf


Matlab's mat format is seen as plain text, and Octave's binary format is seen as generic binary data. In either case the "comment ()" property doesn't give more hint on the nature of the data.

(file #44309)

Pantxo Diribarne <pantxo>
Group Member
Wed 06 Jun 2018 12:36:11 PM UTC, comment #5: 

I fixed the first issue of non-existent files here:

http://hg.savannah.gnu.org/hgweb/octave/rev/d8aa0e66aa5c

@Mike: according to the SO discussion I mentioned in comment #2, "open" is ran whenever one double clicks a file in the file browser. "open" being a function present in the interpreter, it should work both in CLI and GUI modes. Now what "open" does when it finds e.g. data files (openvar? load? ...) may depend on wether the GUI is running.

Indeed, using Qt's capabilities is probably the way to go (I'd bet it's libmagic in the background anyway). The only disadvantage is the fact that those classes where introduced in Qt 5.0.

Pantxo Diribarne <pantxo>
Group Member
Tue 05 Jun 2018 10:43:45 PM UTC, comment #4: 

If this bug is only relevant for Octave with the GUI active, should we prefer the Qt desktop integration MIME functions to yet another additional third-party library?

https://doc.qt.io/qt-5/qmimedatabase.html

That's why I asked if this problem is only relevant for when the GUI is active.

If someone uses the open function in command-line mode, shouldn't it always open the file in an external program unconditionally?

If the GUI is active, then the interpreter can defer to the GUI to determine how the file should be opened.

Mike Miller <mtmiller>
Group Member
Tue 05 Jun 2018 10:31:19 PM UTC, comment #3: 

I attached a very simple wrapper around libmagic, that I tested as follows:


im = rand (10,10,3);
fname = "toto.titi";

## Images
fmts = {"png", "tiff", "gif", "jpeg"};
for ii = 1:numel (fmts)
  fmt = fmts{ii}
  imwrite (im, fname, fmt);
  disp (fileinfo (fname))
  disp ("")
endfor

## Octave generated data
fmts = {"-v7", "-binary", "-text", "-zip", "-hdf5"};
for ii = 1:numel (fmts)
  fmt = fmts{ii}
  save (fmt, fname, "im");
  disp (fileinfo (fname))
  disp ("")
endfor


and here is the result


fmt = png
PNG image data, 10 x 10, 8-bit/color RGB, non-interlaced

fmt = tiff
TIFF image data, little-endian

fmt = gif
GIF image data, version 89a, 10 x 10

fmt = jpeg
JPEG image data, JFIF standard 1.01

fmt = -v7
Matlab v5 mat-file (little endian) version 0x0100

fmt = -binary
Octave binary data (little endian)

fmt = -text
ASCII text

fmt = -zip
gzip compressed data, from Unix

fmt = -hdf5
Hierarchical Data Format (version 5) data


If you want to test, you need to install the developper version of libmagic and compile fileinfo.cc as follows


mkoctfile -lmagic ./fileinfo.cc




(file #44307)

Pantxo Diribarne <pantxo>
Group Member
Tue 05 Jun 2018 09:20:10 PM UTC, comment #2: 

@gyom: Testing if the file exists is reasonable (and easy).

For binary and text data handling I think it is necessary to have some mime type checking layer (maybe using libmagic, like the file command) if we want to do sensible choices when it comes to handling contents (image? numeric tables? hdf5 files ? ...). Matlab seams to have "finfo" to do that job, but I couldn't find documentation on the web ("help finfo" works in Matlab though). See e.g :

https://stackoverflow.com/questions/22067730/matlab-open-files-outside-matlab-by-default

The above is not a short project, so at least we should be able to determine if a suitable fall-back application has been found on the system, and fire a warning otherwise.

Pantxo Diribarne <pantxo>
Group Member
Tue 05 Jun 2018 05:32:33 PM UTC, comment #1: 

If random.dat happens to be a zip file does Matlab open it in WinZip or whatever program is installed to handle zip files? Does it matter if it is named random.dat or random.zip?

In what format does Matlab read random data to open it in the variable editor, as a uint8 array?

What is the behavior if Matlab is started in non GUI mode?

Mike Miller <mtmiller>
Group Member
Tue 05 Jun 2018 03:36:07 PM UTC, original submission:  

This is a follow-up of bug #50543 (comments 7 to 11).

Given a binary file, a text file and a non-existing file:


dd if=/dev/urandom of=random.dat count=100 # file: data
echo blah blah blah > random.text # file: ASCII text


Matlab's behavior is:


>> open nonexisting.file
Error using open (line 86)
File 'nonexisting.file' not found.
>> open random.dat
% try to display data in the variable editor
>> open random.text
% open file in Matlab's text editor


while Octave does nothing for a binary or non-existing file and opens the text file in the system's text editor (not Octave's).

Guillaume <gyom>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44309:  filetype.patch added by pantxo (3KiB - text/x-patch)
file #44307:  fileinfo.cc added by pantxo (1KiB - text/x-c++src)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by pantxo (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by gyom (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-06-06 pantxo Attached File- Added filetype.patch, #44309
    2018-06-06 pantxo StatusNeed Info In Progress
    2018-06-05 pantxo Attached File- Added fileinfo.cc, #44307
    2018-06-05 mtmiller StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code