bugGNU Octave - Bugs: bug #64756, add '-nocompression' option to...

 
 

bug #64756: add '-nocompression' option to save function for better Matlab compatibility

Submitter:  Bill Eaton <wpeaton>
Submitted:  Fri 06 Oct 2023 06:52:29 PM UTC
   
 
Category:  Octave Function Severity:  1 - Wish
Priority:  3 - Low Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * 8.2.0 Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 17 Oct 2023 04:52:08 PM UTC, comment #6: 

@wpeaton: you're absolved :-)

I didn't know, or realize, that Matlab compresses .mat files by default. That may not have always been the case. In the script library or collection (started in the late nineties) at my work we have old .m-files and function calling 'load' explicitly with the '-v7' flag; I suppose that hasn't been there for nothing. (Saving files on network drives goes a lot faster if they're compressed on the fly.)

BTW, AFAIK Octave's '-v7' flag invokes compression by default - provided Octave was built with zlib library support. That isn't stated as such in the help text, though.  .mat files saved with  -v6' and '-v7' differ quite a bit in size.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 16 Oct 2023 07:38:34 PM UTC, comment #5: 

Forgive me. I mistakenly thought that "-compress" was an option for MATLAB. Looking back now, I cannot find it anywhere in R2023B docs.

To restate the problem more accurately,
  Octave has '-z' or '-zip' options. default is no compression.
  MATLAB has '-nocompress' option. default is compression.

I agree that adding "-compression" as an alias for "-z" or "-zip"
probably doesn't add any value. Especially since it's not even a legitimate option in MATLAB.


Bill Eaton <wpeaton>
Tue 10 Oct 2023 04:19:24 PM UTC, comment #4: 

updating title to reflect the confirmed incompatibility. this would require modification to libinterp/corefcn/load-save.cc, so not just a simple m-file change, and we'd need to confirm whether using it would actually change any default compression behavior for all of the other possible uses of save.

the possibility of adding a matching -compression or -compress option as a -zip alias is still a simple possibility, but I'm not use adding it really adds any particular value.

Nicholas Jankowski <nrjank>
Group Member
Tue 10 Oct 2023 02:52:22 PM UTC, comment #3: 

forgot to copy:


>> save ('test.dat', 'a','-v7','-compression')
Error using save
Unknown command option.
-verbatim

in case the issue was option shortening

Nicholas Jankowski <nrjank>
Group Member
Tue 10 Oct 2023 02:50:43 PM UTC, comment #2: 

updating summary and category fields. this seems like a fairly straightforward compatibility change, but is this actually a matlab compatible change?

the online docs for current or `v7 era' matlab make no mention of a '-compress' option.  current matlab (>2019) online help shows a '-nocompression' option that only applies with to the v7 or 7.3 options (where matlab had introduced default compression). 'help save' in matlab tells you the option is valid for 'version 7 and above'.  When those v7/7.3 versions were new and before the '-nocompression' option was introduced, the help told you to save as v6 to disable compression.

if i try to use the '-compress' option in matlab 2023b:

>> a = magic (4);

>> save ('test.dat', 'a','-v7')

>> save ('test.dat', 'a','-v7','-nocompression')

>> save ('test.dat', 'a','-v7','-nocompress')
Error using save
Unknown command option.

>> save ('test.dat', 'a','-v7','-compress')
Error using save
Unknown command option.

>> save ('test.dat', 'a', '-nocompression')

>> save ('test.dat', 'a', '-nocompress')
Error using save
Unknown command option.

>> save ('test.dat', 'a', '-compress')
Error using save
Unknown command option.

>> save ('test.dat', 'a','-v6','-nocompression')
Error using save
Nocompression not supported for this MAT-file version. Use MAT-file
v7(default) or v7.3 instead.

>> save ('test.dat', 'a','-v6','-nocompress')
Error using save
Unknown command option.

>> save ('test.dat', 'a','-v6','-compression')
Error using save
Unknown command option.

>> save ('test.dat', 'a','-v6','-compress')
Error using save
Unknown command option.


So whether trying on the v7 formats, or default formats, the only supported option is -nocompression, and that is only supported on v7/7.3. any other option produces an error.

So, adding -compress isn't something that would need to be done for any compatibility concern. 


in Octave 8.3.0, i did notice the -nocompression option is unrecognized for -v7


>> a = magic(4)
a =

   16    2    3   13
    5   11   10    8
    9    7    6   12
    4   14   15    1

>> save('testdat.dat', 'a')
>> save('testdat.dat', 'a', '-v7')
>> save('testdat.dat', 'a', '-v7', '-nocompression')
error: save: Unrecognized option '-nocompression'


which would be nice to add for compatibility, but that's a separate issue.  Is our v7 uncompressed by default? And is that a known incompatibility?  if so, we could add '-nocompression' as a 'do nothing' option that is just there for code compatibility.

Nicholas Jankowski <nrjank>
Group Member
Sun 08 Oct 2023 08:53:30 PM UTC, comment #1: 

So you just want to have a '-compress' alias for Octave's '-zip' option? At first sight that looks easy enough. But I'm not familiar with the code so there may be some gotchas.

Yes it is known that Matlab uses gzip for '-v7'. So .mat files written with '-v7' can be used interchangeably by Octave and Matlab. I'm doing that all the time myself.
But for '-v7.3' Matlab is using HDF5, in a form not supported by Octave (yet), or maybe just partly.

FYI:
As to load/save there are longstanding wishes to simplify these commands (a.o., internally, and maybe drop support for some formats exclusively used by Octave), to support '-v7.3' and to be able to save classdef stuff. But these are still wishes ...

Philip Nienhuis <philipnienhuis>
Group Member
Fri 06 Oct 2023 06:52:29 PM UTC, original submission:  

For compression
  Octave uses '-z' or '-zip' options
  MATLAB uses '-compress' or '-nocompress' options

presumably Octave uses no compression by default. MATLAB compress options only apply to '-V7' or '-V7.3' formats.

I did a test where I save'd a file in Octave with '-zip':

====== begin script ==================
A = eye(5,5);
save('test.mat', A, '-V7', '-zip');
====== end script ==================

I asked a friend who has MATLAB to open and they were able to successfully open the file. This seems to indicate that MATLAB also uses gzip compression.

So my request is this: can Octave save function be modified to include 'compress' and '-nocompress' options to improve compatibility?

Bill Eaton <wpeaton>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by wpeaton (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2023-10-10 nrjank StatusNone Confirmed
        Summaryadd '-compress' option to save function for better Matlab compatibility add '-nocompression' option to save function for better Matlab compatibility
    2023-10-10 nrjank Severity3 - Normal 1 - Wish
        Priority5 - Normal 3 - Low
        Summarymake Octave save function more compatible with MATLAB add '-compress' option to save function for better Matlab compatibility
    2023-10-08 philipnienhuis Carbon-Copy- Added philipnienhuis

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code