Mon 16 Jun 2014 09:11:14 PM UTC, comment #20:
I see I typed to fast and got row and column vectors reversed in the last post, but the answer still stands. You can use either one for export.
|
Mon 16 Jun 2014 09:10:03 PM UTC, comment #19:
Q1 : It's really up to you whether you prefer row (Nx1) or column (1xN) vectors. Octave/Matlab makes the distinction because you might want to form the Matrix/Vector product
which will only work if x is oriented correctly.
Q2 : Correct. We've worked around the 1-D case and it should be part of our 3.8.2 bug fix release in a few weeks so there is no longer any reason for you to hurry a patch into production.
|
Mon 16 Jun 2014 08:50:32 PM UTC, comment #18:
Author of R.matlab here:
Thanks for reporting on this out. I take it as R.matlab::writeMat() incorrectly writes R vectors as one-dimensional arrays (i.e. the dimension array subelement contains a single value) whereas the MAT file format requires at least two elements. I must admit that I've missed:
"Dimensions Array Subelement
This subelement specifies the size of each dimension of an n-dimensional array in an n-sized array of 32-bit values (miINT32). All numeric arrays have at least two dimensions. The Dimensions Array subelement is common to all MATLAB array types." (page 1-17 in http://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf)
Q1. Should R vectors (=one dimensional arrays) be written as nx1 or 1xn arrays?
Q2. Should I read this thread as you indeed updated Octave to workaround the one-dimensional case?
/Henrik
PS. It'll probably be a while before I fix this in R.matlab, because I have to careful about cases related to automatic translation/coercing of vectors-to-matrices in writeMat() and readMat().
|
Mon 16 Jun 2014 04:11:59 PM UTC, comment #17:
Rik,
Thank you.
I think the point is that even this "malformed" objects are read in by matlab, as you said. But they weren't loaded by GNU Octave.
I will tell the community of R (which seems to have problems loading empty objects as 0x0 matrices), but I guess their answer will be something like: "matlab and scipy load these files without problems."
Thanks again.
|
Mon 16 Jun 2014 03:56:26 PM UTC, comment #16:
This is really a problem with R and Python. Matlab objects are always at least 2-D. Even an empty object is implemented as a 2-D 0x0 object. Hence, when the file is read in by Matlab and written back out again it generates a proper scalar object which is 2-D 1x1.
We can help out our open source friends though since the code change is minimal. I made the change on the stable branch in this changeset (http://hg.savannah.gnu.org/hgweb/octave/rev/94d791a03737). It will be part of the 3.8.2 bug fix release.
|
Mon 16 Jun 2014 11:49:02 AM UTC, comment #15:
I implemented the suggestion from anonymous.
The attached patch should solve the problem.
Please check and close the bug.
Thanks
(file #31565)
|
Mon 16 Jun 2014 08:44:55 AM UTC, comment #14:
Juan Pablo,
a possible shorter version of your patch.
|
Mon 16 Jun 2014 03:02:05 AM UTC, comment #13:
To find out ndims directly from a dim_vector it is usually sufficient to take the length of the dim_vector. This will not tell you whether one of the dimensions is zero though so a 1x0 object still has 2 dimensions.
|
Mon 16 Jun 2014 02:13:45 AM UTC, comment #12:
The attached change set fixes the problem, but is not elegant.
I have tested it with row vectors, columns, matrices and NDarrays created in R.
Once the dim_vector is resized (line 670 ls-mat5.cc), the loop should use the information stored inside it (since when resized it is check that if dims < 2 then dim == 2), but I couldn't find the proper way of doing it.
How can one get ndims from dim_vector directly?
Thanks
(file #31561)
|
Mon 16 Jun 2014 01:36:19 AM UTC, comment #11:
OK, I think I found the problem.
I created a file as explained below using R.
As anonymous pointed out the tag for dimension of the data element is in 0x90
According to the format specification (note on page 1-17) the 04 means there is a single dimension (1x1 not 1x0). This is correctly calculated in line 669 of ls-mat5.cc. However in the following for loop the index_type dims is not fully fill and is left as 1x0.
This is a miss interpretation of the format specification. If the number of dimensions is 1 then the data should be seen as 1x1.
The fix of anonymous works cause ndims is 2 in that case and each value 1, so the resulting index_type dims is 1x1.
Is this correct?
I arrive to this conclusion by looking at the actual creation of the data container in octave
|
Mon 16 Jun 2014 01:02:23 AM UTC, comment #10:
Can you produce a version of the file that was written from Matlab and is readable by Octave?
It would then be easy to do a binary difference between the working and non-working files to find out which field code is being written differently by R.
|
Mon 16 Jun 2014 12:21:09 AM UTC, comment #9:
Well, python can also read it and matlab itself can also read it.
I think we better read these files!
To me it seems that read_mat5_tag in libinter/corefun/ls-mat5.cc is not implementing the mat-file format correctly. Although we haven't put the finger on it yet.
The format specification is here
http://www.mathworks.com/help/pdf_doc/matlab/matfile_format.pdf
page 1-8.
|
Mon 16 Jun 2014 12:09:10 AM UTC, comment #8:
According to later entries in the thread, Octave can read the files if they were created with Matlab. It's only R's version of matlab V5 that seems to be a problem.
|
Mon 16 Jun 2014 12:05:29 AM UTC, comment #7:
I am provinding the file created with R for easier debugging.
@Kai, could you provide the file created with matlab? Just save a variable "latency" with a small value in it.
(file #31560)
|
Sun 15 Jun 2014 11:46:19 PM UTC, comment #6:
This are the last bytes from the .mat file
The tag from position 0xd1d8 is 5,4,1,0
If this is changed to 5,8,1,1 'latency' can be read correctly.
I assume if there's only one value to read, the matrix size is 1x1
|
Sun 15 Jun 2014 11:18:19 PM UTC, comment #5:
Indeed, the following lines in R generate the problem
in R:
in Octave (dev)
The file test.m is incorrectly read in Octave. However python reads it without a problem.
In python:
+veratim+
from scipy.io import loadmat
loadmat("test.mat")
{'__globals__': [],
'__header__': 'MATLAB 5.0 MAT-file, Platform: unix, Software: R v3.0.2, Created on: Mon Jun 16 01:12:34 2014',
'__version__': '1.0',
'latency': array([ 0.])}
-verbatim-
|
Sun 15 Jun 2014 11:07:15 PM UTC, comment #4:
I did not created this file.
This are the files form the Kaggle constest.
Maybe the file was created with R?
I will check
|
Sun 15 Jun 2014 11:05:40 PM UTC, comment #3:
I can confirm the problem for the M-File you provided. But maybe your M-File is somehow broken. I loaded and saved your file with my MATLAB R2013b and Octave loaded the new one without problems.
The new document header becomes:
MATLAB 5.0 MAT-file, Platform: GLNXA64, Created on: Mon Jun 16 00:59:55 2014
What does "Software R" mean? That's the observed header difference. Do you use any plugins, toolboxes with your MATLAB that overwrites the save function?
|
Sun 15 Jun 2014 08:53:59 PM UTC, comment #2:
Trying to re-attach file
(file #31558)
|
Sun 15 Jun 2014 08:14:36 PM UTC, comment #1:
Using python module scipy.io, the function loadmat says that the data is has the header
'MATLAB 5.0 MAT-file, Platform: unix, Software: R v3.0.2, Created on: Mon Apr 7 22:05:57 2014'
It seems that octave can't read matfiles v5? I tried option -v4 but it fails. Options -v6 and -v7 read the file incorrectly.
|
Sun 15 Jun 2014 08:02:55 PM UTC, original submission:
A matfile saved with a version newer than r2010b of matlab can't be loaded properly in octave.
Try to load the attached file to observe the problem.
The variables "freq" and "latency" should contain a double (399.6098 and 0 respectively) but are loaded empty.
This problem is preventing several users to use Octave for a competition at Kaggle.
http://www.kaggle.com/c/seizure-detection/forums/t/8256/problems-with-freq-and-latency-in-octave-3-8-1/49137#post49137
|