bugGNU Octave - Bugs: bug #51741, zlib error when loading .mat file...

 
 

bug #51741: zlib error when loading .mat file on Windows 10 64bit

Submitter:  Xi Shen <davidshen84>
Submitted:  Sun 13 Aug 2017 01:07:40 AM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Works For Me Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 4.2.1 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 24 Jan 2019 11:39:03 PM UTC, comment #11: 

It works for me now.

Xi Shen <davidshen84>
Thu 24 Jan 2019 07:07:49 PM UTC, comment #10: 

Any further information on debugging this problem on the particular Windows system where it fails?

Mike Miller <mtmiller>
Group Member
Mon 28 Aug 2017 06:19:27 PM UTC, comment #9: 

From the mailing list, not posted to the tracker:

Hi Mike,

I tried to use absolute path, still no luck.

When I try to use fileread to read the file, octave simply hanged. But in the variable panel, it showed the size of the data is 1x7511764 characters. And 7511764 / (1024^2) = 7.16 MB which I think is the correct size.


Regards,
David

John W. Eaton <jwe>
Group administrator
Thu 17 Aug 2017 01:15:48 AM UTC, comment #8: 

This is a wild guess, but please make sure that you do not have another copy of the file somewhere on Octave's load path. Most Octave functions, including fileread and load, will find a file by name on the load path, so you may not be trying to use the copy of the file that you think you are. Or use the absolute path to be sure.

As long as that's not the simple solution, it looks like you're on the right path of debugging this. You may want to look at the size of the buffer that fileread returns, make sure it's the right number of bytes. Or try reading the file with fread in increasing blocks and see if you can figure out where it goes wrong, if there is some size threshold or a particular byte that is interpreted badly by Octave for some reason.

Mike Miller <mtmiller>
Group Member
Thu 17 Aug 2017 12:53:33 AM UTC, comment #7: 

Previously, when I say the md5 checksum of ex4data1.mat file on my affected computer was the same as Dmitri's, I was using an external tool to compute the checksum, not from Octave. Since we all believe the file is correct, I wonder if there's some thing wrong with reading the file. So I did

>> hash('md5', fileread('ex4data1.mat'))

ans = caeaea99f82a3faf059cbd32ca0f5111

And it looks bad...I checked from the external tool again, and it confirms that its md5 checksum is 7A879161A0E63D7156B33E11511FAD02.

Then I checked the md5 checksum of another file from Octave,

>> hash('md5', fileread('ex4weights.mat'))

ans = 8dddd0bb5ecf0633232c70e29aac8807

This time, the md5 checksum is the same as it is computed from external tool.

On my affected computer, it seems Octave has problem even reading the file, not to say decompress it. But why this particular file?

Xi Shen <davidshen84>
Wed 16 Aug 2017 04:02:44 PM UTC, comment #6: 

I spent about 30 minutes looking at this file to make sure there's nothing unusual about it. It seems like a correct mat file, and following along by hand it looks like Octave is reading the file correctly. If you want to pursue this bug, you'll want to debug the function read_mat5_binary_element using gdb, and print what your instance of Octave thinks it is trying to decompress.

The mat file contains two elements, both compressed. The file is in little-endian format. The byte offset of the first compressed element is 136, and its compressed size is 7511543 bytes. The byte offset of the second compressed element is 7511687 and its compressed size is 77 bytes.

I was able to manually decompress both elements at the command-line with


$ dd if=ex4data1.mat bs=1 skip=INDEX count=SIZE | pigz -dz | hexdump -C


and verify that the inner contents of the compressed elements look correct as well. The first element is named "X", type double, size 5000×400. The second element is named "y", type double, size 5000×1.

That's farther than your copy of Octave was able to go, since it errored out at the point of trying to decompress one of the compressed blocks. If you debug into this function, you should be able to print out the size of the block that it thinks it is decompressing, how large the uncompressed buffer is, and so on.

That's about as much as I can debug this at the moment, the file looks correct and Octave's algorithm for reading the file corresponds to what I can read out of the file manually, so you'll have to find where it's going wrong on your system.

Mike Miller <mtmiller>
Group Member
Tue 15 Aug 2017 05:10:21 PM UTC, comment #5: 

per email conversation,
OP verified that MD5sum on the affected computer is correct
(7a879161a0e63d7156b33e11511fad02  ex4data1.mat)
Also, quoting "Xi Shen":

<<<
I went on to rename .\bin\libz.dll to lib_.dll, and octave cannot start, complaining unable to find libz.dll. So I think octave is load the libz.dll at the correct path. The md5 checksum of the libz.dll is 893F556973A6028606F4988A96032516

>>>


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 14 Aug 2017 03:33:02 PM UTC, comment #4: 

Octave should use the libz.dll that you find in the bin folder where Octave is installed. You could try to rename it and check if Octave would still start. (It shouldn't.)

But more likely, the local copy of the file is broken on the affected PC: Have you tried to download the file again (possibly with another browser to make sure you don't get a cached version)?

Markus Mützel <mmuetzel>
Group administrator
Mon 14 Aug 2017 11:35:32 AM UTC, comment #3: 

I tried on another Windows 10 64 bit computer, and it works...

On the original computer, I also tried to move the .mat file to a directory which path does not contain space and non-ascii characters.

So I think it is the local configuration on this computer that causes the problem.

And idea how can I debug this? Like, how to find out which zlib it is loading?

Xi Shen <davidshen84>
Sun 13 Aug 2017 03:50:55 PM UTC, comment #2: 

I am able to load the file in Octave 4.2.1 on Windows 10 64bit.

>> clear all
>> load ex4data1.mat
>> who
Variables in the current scope:

X  y


Does your Windows user name contain any non-ASCII characters or do you try to load the file from a directory containing non-ASCII characters? Have you tried to download the file again (possibly with another browser to make sure you don't get a cached version)?

Markus Mützel <mmuetzel>
Group administrator
Sun 13 Aug 2017 04:17:59 AM UTC, comment #1: 

doug456 from #octave channel said the .mat loads fine in Ubuntu. So, it is a Windows compatibility issue.

Xi Shen <davidshen84>
Sun 13 Aug 2017 01:07:40 AM UTC, original submission:  

I have a MatLab .mat file. When I try to load it, I got this error:

    error: load: error uncompressing data element (data error from zlib)

I am able to work with file using SciPy. I was able to work with this file a few years ago on Windows 7.


Note: The file is over 7 MB large. You can get a copy from https://github.com/davidshen84/machine-learning/blob/coursera/coursera/ex4/ex4data1.mat. It is part of the Machine Learning course from Coursera.

Xi Shen <davidshen84>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by davidshen84 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-01-24 mtmiller Open/ClosedOpen Closed
    2019-01-24 mtmiller Carbon-CopyRemoved 80942 -
    2019-01-24 mtmiller Item GroupMatlab Compatibility Unexpected Error or Warning
    2017-08-15 mtmiller StatusNone Works For Me

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code