bugGNU Octave - Bugs: bug #42112, Unable to load Java objects from...

 
 

bug #42112: Unable to load Java objects from MAT file

Submitter:  Ben Kurtz <bkurtz>
Submitted:  Mon 14 Apr 2014 05:29:39 AM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  None Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Thu 01 May 2014 10:28:56 PM UTC, comment #5: 

Just for fun I tried to see if I could load and save Java objects using Java's Serializable methods.  It worked fine.  It would be nice to have Octave implement this for load_binary/save_binary in ov-java.cc.  I used


x = javaObject ('java.lang.Double', pi)
fout = javaObject('java.io.FileOutputStream', '/tmp/blah.ser')
sout = javaObject ('java.io.ObjectOutputStream', fout)
sout.writeObject (x)
sout.close ()
fin = javaObject('java.io.FileInputStream', '/tmp/blah.ser')
sin = javaObject ('java.io.ObjectInputStream', fin)
xin = sin.readObject
xin
sin.close ()
fin = javaObject('java.io.FileInputStream', '/tmp/blah.ser')
sin = javaObject ('java.io.ObjectInputStream', fin)
xin2 = javaObject ('java.lang.Double', sin.readObject())
xin2.toString


The first time I read the object it just returned a double of 3.1416. I think this is probably caused by Octave auto-converting the result from a Java object to a double. The second time I called the Java constructor with the results of the readObject call and it returned a true Java object with the full value of pi.

Rik <rik5>
Group administrator
Thu 24 Apr 2014 05:22:35 PM UTC, comment #4: 

I'm glad to see that Matlab took a reasonable approach to this;  Calling the Java Objects own serialization method makes a lot of sense.

The function you probably want to modify is read_mat5_binary_element in ls-mat5.cc.  Near the top of the file there is an enum declaration with the various CLASSTYPES in a Matlab file.  In the function read_mat5_binary_element there is a case statement that keys off the CLASSTYPE.  My guess is that there is a special class for Java objects, although they may have used the MAT_FILE_OBJECT_CLASS type which is for any class object.

Rik <rik5>
Group administrator
Thu 24 Apr 2014 04:33:30 PM UTC, comment #3: 

That makes sense.  I've not had nearly as much time to work on this as I'd been hoping, but I have made some progress (namely, I've made considerable progress at dissecting the file format that MATLAB uses for java objects) but I haven't really gotten the chance to start writing code yet.  The good news is that it looks like matlab just asks the java object for a serialization of itself, and then stores that binary data in the .mat file in a particular way.

Also thanks for your explanation; I was having some trouble figuring out how exactly that was supposed to work (I think my issue was partly that I created the octave_java::load_binary function and it wasn't getting called).  Can you clarify a little more about how the save_binary/load_binary functions are supposed to interact with the functions in (for example) libinterp/corefcn/ls-mat5.cc?

And then, at some point I guess we'd have to figure out how/if it makes sense to save java objects to an ascii file...

Ben Kurtz <bkurtz>
Thu 24 Apr 2014 03:56:36 PM UTC, comment #2: 

This is probably a huge coding effort :(

First, I'm not sure it will be possible to save java objects in Matlab and then re-load them in Octave.  It really depends on how Matlab did the Serialization/Deserialization and whether there is any documentation or clues to their approach.

But, before that can happen Octave itself needs to be able to load/save Java objects which it can't.  See the following code:


octave:1> x = javaObject ('java.lang.Double', 1)
x =

<Java object: java.lang.Double>

octave:2> save -binary tst.save x
error: octave_base_value::save_binary(): wrong type argument 'octave_java'
warning: save: no such variable 'x'


I think the first place to start would be to get load/save working with Octave's own formats.

I can give you a few pointers about how to do this.  The key is that a Java object is an octave_value.  This is an overloaded class where the underlying value might be a double, uint8, range (such as 1:3:10), Java object, etc.  If you look in the directory libinterp/octave-value you will find all of the different octave_value classes.  The overall octave_value class (ov.h) defines various virtual methods like save_ascii or save_binary.  It is up to the individual class to override these with a function that does whatever is necessary for that particular object to save itself.

I'll take the example of a range because it is particularly simple.


x = 1:3:10
x =

    1    4    7   10

save -text tst.txt x


And the resulting file is:


# Created by Octave 4.1.0+, Thu Apr 24 08:51:01 2014 PDT <rik@tesseract>
# name: x
# type: range
# base, limit, increment
1 10 3


Now if you look in ov-range.h you will see there is a prototype for save_ascii.  In ov-range.cc you can find the actual function:


octave_range::save_ascii (std::ostream& os)
{
  Range r = range_value ();
  double base = r.base ();
  double limit = r.limit ();
  double inc = r.inc ();
  octave_idx_type len = r.nelem ();

  if (inc != 0)
    os << "# base, limit, increment\n";
  else
    os << "# base, length, increment\n";

  octave_write_double (os, base);
  os << " ";
  if (inc != 0)
    octave_write_double (os, limit);
  else
    os << len;
  os << " ";
  octave_write_double (os, inc);
  os << "\n";

  return true;
}


The range class has a pretty simple way of saving itself.  For Java objects, something more complex will have to be designed.  But the approach is the same.  Add a prototype for save_ascii, save_binary, etc. in ov-java.h and then an implementation in ov-java.cc.

Rik <rik5>
Group administrator
Mon 14 Apr 2014 07:52:54 PM UTC, comment #1: 

Thanks for your offer to try to fix this.

Michael Goffioul and Rik may be the ones who knows the most of Octave's Java support implementation, until one of them (or both) responds you could start looking in:

scripts/java/org/octave   for the various helpers

libinterp/octave-value/ov-java.*  (.cc & .h) for some binary stuff

and then of course you'd need to dive into the stuff that writes/reads .mat files:


>> which save
'save' is a built-in function from the file libinterp/corefcn/load-save.cc


Hopefully that should be enough to give you a basic start.

BTW see here for some other identified Octave <->Java deficiencies:
https://savannah.gnu.org/task/?12601
(just because some of these may be required to be able to fix this bug, who knows)

Philip Nienhuis <philipnienhuis>
Group Member
Mon 14 Apr 2014 05:29:39 AM UTC, original submission:  

I have a file created in matlab as follows

> x = java.io.File('/usr/local/bin');
> save('/tmp/file.mat')


When I try to load this file into octave, I get the following results:

octave:1> load('/tmp/file.mat')
octave:2> x
x =

  scalar structure containing the fields:

    java =

error: octave_base_value::print (): wrong type argument `<unknown type>'

    java.io.File =

error: octave_base_value::print (): wrong type argument `<unknown type>'

It seems that octave doesn't quite understand matlab's file format for embedding java objects in .mat files.  I am able to create java objects directly in octave, so I think it's not just a problem with my octave/java install.

I'd be happy to look into fixing this, but I'd very much appreciate if someone could give me a pointer in the right direction - octave's codebase is very large and last time I looked for this I was having a lot of trouble tracking down the right place to work.

Ben Kurtz <bkurtz>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #31167:  file.mat added by bkurtz (418B - application/x-matlab-workspace)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by bkurtz (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2015-08-04 rik5 Dependencies- bugs #45694 is dependent
    2014-04-24 rik5 CategoryOctave Package Interpreter
    2014-04-14 bkurtz Attached File- Added file.mat, #31167

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code