There is a GitHub repository that aims to provide an interface to the HDF5 library for GNU Octave. Link: https://github.com/pantxo/oct-hdf5 This is built as an Octave package.

Incomplete documentation link: https://pantxo.github.io/oct-hdf5/

I have not gone through the patches. Just informing about similar work.

Also see: https://octave.discourse.group/t/new-feature-hdf5-file-i-o/4323

It is a pity that this is hanging here without much attention for so long now (about 8 years).

I just had a very short look at the patch. I didn't check if it still applies.
If I understand correctly, it is/was already quite complete when it comes to implementing functions that are likely compatible to Matlab.

However, it adds those HDF5 functions in a new .oct file. Octave core already links to libhdf5, and as jwe mentioned pretty early on, it would be nice if the backbone of this implementation could be shared with the functions that load and save Octave values.
Because of that, it would make sense imho to move these functions to liboctinterp.

Is anyone still working on this?

Did this just die?  The nabble link is broken for me.

Attached is a patch relative to the default branch at July 18, hg id d1216d085280


(file #44573)

I'm more and more tempted to move to python/julia,
that seem better suited for research now.
Since you seem to be interested by this patch, please take it from here.

At the top of libinterp/dldfcn/module.mk, there is

## DO NOT EDIT -- generated from module-files by config-module.awk

Is libinterp/dldfcn/module.mk really to be edited ?

Probably, since oct-qhull.h is present in module.mk, but not in module-files.

With the updated patch attached, it seems to work again:

>> m = rand(2, 3)
m =

   0.94236   0.96434   0.71092
   0.23191   0.50169   0.92243

>> h5create("try_w.hdf5", "/m", size(m))
>> h5write("try_w.hdf5", "/m", m)
>> h5read("try_w.hdf5", "/m")
ans =

   0.94236   0.96434   0.71092
   0.23191   0.50169   0.92243

(file #42815)

I've updated the patch so that it applies cleanly to 4.3.0+ bu the build breaks at h5read:

:
  CXX      libinterp/dldfcn/libinterp_dldfcn_h5read_la-h5read.lo
In file included from ./config.h:3541:0,
                 from ../dev_VE/libinterp/dldfcn/h5read.h:25,
                 from ../dev_VE/libinterp/dldfcn/h5read.cc:43:
./oct-conf-post.h:58:1: error: redefinition of 'template<class T> void octave_unused_parameter(const T&)'
 octave_unused_parameter (const T&)
 ^
In file included from ./config.h:3541:0,
                 from ../dev_VE/libinterp/dldfcn/h5read.cc:25:
./oct-conf-post.h:58:1: note: 'template<class T> void octave_unused_parameter(const T&)' previously declared here
 octave_unused_parameter (const T&)
 ^
../dev_VE/libinterp/dldfcn/h5read.cc: In function 'bool any_int_leq_zero(const Matrix&)':
../dev_VE/libinterp/dldfcn/h5read.cc:50:35: warning: 'octave_idx_type Array<T>::length() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:354): use 'numel' instead [-Wdeprecated-declarations]
   for (int i = 0; i < mat.length (); i++)
                                   ^
../dev_VE/libinterp/dldfcn/h5read.cc: In function 'int check_vec(const octave_value&, Matrix&, const char*, bool)':
../dev_VE/libinterp/dldfcn/h5read.cc:77:38: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
       for (int i = 0; i < mat.nelem (); i++)
                                      ^
In file included from ../dev_VE/libinterp/corefcn/defun-dld.h:32:0,
                 from ../dev_VE/libinterp/corefcn/oct.h:32,
                 from ../dev_VE/libinterp/dldfcn/h5read.cc:28:
../dev_VE/libinterp/dldfcn/h5read.cc: At global scope:
../dev_VE/libinterp/dldfcn/h5read.cc:200:29: warning: unused parameter 'nargout' [-Wunused-parameter]
 DEFUN_DLD (h5readatt, args, nargout,
                             ^
../dev_VE/libinterp/corefcn/defun-int.h:92:49: note: in definition of macro 'DECLARE_FUNX'
   name (const octave_value_list& args_name, int nargout_name)
                                                 ^
../dev_VE/libinterp/corefcn/defun-dld.h:48:3: note: in expansion of macro 'DECLARE_FUN'
   DECLARE_FUN (name, args_name, nargout_name)
   ^
../dev_VE/libinterp/dldfcn/h5read.cc:200:1: note: in expansion of macro 'DEFUN_DLD'
 DEFUN_DLD (h5readatt, args, nargout,
 ^
../dev_VE/libinterp/dldfcn/h5read.cc: In constructor 'H5File::H5File(const char*, bool)':
../dev_VE/libinterp/dldfcn/h5read.cc:509:25: warning: 'file_stat' is deprecated (declared at ../dev_VE/liboctave/system/file-stat.h:332): use 'octave::sys::file_stat' instead [-Wdeprecated-declarations]
   file_stat fs (filename);
                         ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'hsize_t* H5File::alloc_hsize(const T&, int, bool)':
../dev_VE/libinterp/dldfcn/h5read.cc:575:61: warning: use of old-style cast [-Wold-style-cast]
   hsize_t *hsize = (hsize_t*)malloc (rank * sizeof (hsize_t));
                                                             ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'int H5File::open_dset(const char*)':
../dev_VE/libinterp/dldfcn/h5read.cc:614:53: warning: use of old-style cast [-Wold-style-cast]
   h5_dims = (hsize_t*)malloc (rank * sizeof(hsize_t));
                                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:615:56: warning: use of old-style cast [-Wold-style-cast]
   h5_maxdims = (hsize_t*)malloc (rank * sizeof(hsize_t));
                                                        ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'octave_value H5File::read_dset_hyperslab(const char*, const Matrix&, const Matrix&, const Matrix&, const Matrix&, int)':
../dev_VE/libinterp/dldfcn/h5read.cc:670:20: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (start.nelem () != rank)
                    ^
../dev_VE/libinterp/dldfcn/h5read.cc:675:20: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (count.nelem () != rank)
                    ^
../dev_VE/libinterp/dldfcn/h5read.cc:684:22: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (_stride.nelem () != rank)
                      ^
../dev_VE/libinterp/dldfcn/h5read.cc:692:21: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (_block.nelem () != rank)
                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:709:37: warning: use of old-style cast [-Wold-style-cast]
                  i+1, (int)_stride(i), (int)_block(i));
                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:709:53: warning: use of old-style cast [-Wold-style-cast]
                  i+1, (int)_stride(i), (int)_block(i));
                                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:720:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (h5_dims[rank-i-1] < end)
                             ^
../dev_VE/libinterp/dldfcn/h5read.cc:723:85: warning: use of old-style cast [-Wold-style-cast]
                  " are required for requested hyperslab", i+1, (int)h5_dims[rank-i-1],
                                                                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'octave_value H5File::read_dset()':
../dev_VE/libinterp/dldfcn/h5read.cc:753:8: warning: unused variable 'is_cmplx' [-Wunused-variable]
   bool is_cmplx = false;
        ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'void H5File::write_dset(const char*, octave_value)':
../dev_VE/libinterp/dldfcn/h5read.cc:898:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
   for (int i=1; i < path.length (); i++)
                   ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'void H5File::write_dset_hyperslab(const char*, octave_value, const Matrix&, const Matrix&, const Matrix&, const Matrix&, int)':
../dev_VE/libinterp/dldfcn/h5read.cc:1037:20: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (start.nelem () != rank)
                    ^
../dev_VE/libinterp/dldfcn/h5read.cc:1042:20: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (count.nelem () != rank)
                    ^
../dev_VE/libinterp/dldfcn/h5read.cc:1050:22: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (_stride.nelem () != rank)
                      ^
../dev_VE/libinterp/dldfcn/h5read.cc:1058:21: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   if (_block.nelem () != rank)
                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:1071:37: warning: use of old-style cast [-Wold-style-cast]
                  i+1, (int)_stride(i), (int)_block(i));
                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:1071:53: warning: use of old-style cast [-Wold-style-cast]
                  i+1, (int)_stride(i), (int)_block(i));
                                                     ^
../dev_VE/libinterp/dldfcn/h5read.cc:1078:32: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (h5_maxdims[rank-i-1] < end)
                                ^
../dev_VE/libinterp/dldfcn/h5read.cc:1082:57: warning: use of old-style cast [-Wold-style-cast]
                  i+1, dsetname, (int)h5_maxdims[rank-i-1], end);
                                                         ^
../dev_VE/libinterp/dldfcn/h5read.cc:1089:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
       if (h5_dims[rank-i-1] < end)
                             ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'octave_value H5File::read_att(const char*, const char*)':
../dev_VE/libinterp/dldfcn/h5read.cc:1188:56: warning: use of old-style cast [-Wold-style-cast]
       char* buf = (char*)calloc (totsize, sizeof (char));
                                                        ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'void H5File::write_att(const char*, const char*, const octave_value&)':
../dev_VE/libinterp/dldfcn/h5read.cc:1330:54: warning: use of old-style cast [-Wold-style-cast]
       buf = (void *) attvalue.string_value ().c_str ();
                                                      ^
../dev_VE/libinterp/dldfcn/h5read.cc:1330:54: warning: cast from type 'const char*' to type 'void*' casts away qualifiers [-Wcast-qual]
../dev_VE/libinterp/dldfcn/h5read.cc:1338:23: warning: use of old-style cast [-Wold-style-cast]
       buf = (void *) &attval_int;
                       ^
../dev_VE/libinterp/dldfcn/h5read.cc:1345:23: warning: use of old-style cast [-Wold-style-cast]
       buf = (void *) &attval_double;
                       ^
../dev_VE/libinterp/dldfcn/h5read.cc:1272:12: warning: unused variable 'dims' [-Wunused-variable]
   hsize_t *dims;
            ^
../dev_VE/libinterp/dldfcn/h5read.cc: In member function 'void H5File::create_dset(const char*, const Matrix&, const char*, const Matrix&)':
../dev_VE/libinterp/dldfcn/h5read.cc:1409:45: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
   dspace_id = H5Screate_simple (size.nelem (), dims, maxdims);
                                             ^
../dev_VE/libinterp/dldfcn/h5read.cc:1429:47: warning: 'octave_idx_type Array<T>::nelem() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:360): use 'numel' instead [-Wdeprecated-declarations]
       if (H5Pset_chunk (crp_list, size.nelem (), dims_chunk) < 0)
                                               ^
../dev_VE/libinterp/dldfcn/h5read.cc: In instantiation of 'hsize_t* H5File::alloc_hsize(const T&, int, bool) [with T = Matrix; hsize_t = long long unsigned int]':
../dev_VE/libinterp/dldfcn/h5read.cc:729:66:   required from here
../dev_VE/libinterp/dldfcn/h5read.cc:574:26: warning: 'octave_idx_type Array<T>::length() const [with T = double; octave_idx_type = int]' is deprecated (declared at ../dev_VE/liboctave/array/Array.h:354): use 'numel' instead [-Wdeprecated-declarations]
   int rank = dim.length ();
                          ^
Makefile:18461: recipe for target 'libinterp/dldfcn/libinterp_dldfcn_h5read_la-h5read.lo' failed
make[2]: *** [libinterp/dldfcn/libinterp_dldfcn_h5read_la-h5read.lo] Error 1

Updated patch attached

(file #39154)

This is a key pain point for scientific and engineering users. HDF5 is a very popular format, growing in popularity.

This hdf5 package should be as high a priority as having imread() working.

Stefan, thanks for your contribution in providing this patch and helping to test and move this forward. It is definitely appreciated and will hopefully become part of Octave core in a future version.

Just be aware that when this patch was first posted we were weeks away from the first release candidate for version 4.0, and it was therefore highly unlikely to be included in version 4.0, it has more to do with timing than lack of interest.

Hi,

> I haven't been able to read/write hdf5 files yet.

Did it work in the meantime?

>I did hit an issue but I'm not sure it is related:
>With your patch I cannot install the OF communications package >anymore, it complains about undeclared HID_t. That is somewhere >in the hdf5 include tree.

It would be helpful to have more details, if this problem was not just spurious, but is really related.

• Do you install with `pkg install -forge communications signal` ?
• does the same problem arise, if compiled without the patch but with the normal hdf5 support?

after a few minors the issue reported in comment #9 does not appear any more. I don't know exactly why, but it seems to behave well now.

Hence, I won't change anything in the patch for the next days and weeks, unless someone discovers a bug. Concerning the integration with existing hdf5 routines in octave, my opinion is (because at the moment I am trying to make it work both as a package and as a octave-core file) that one should add the patch first with separate mechanics, and subsequently simplify (because then I could drop the pkg) - but I'm not a maintainer.

cheers

(file #33245)

Removed a permute statement which consumed a lot of memory and made the behaviour differ from load('-hdf5',..) and matlab.
The current patch also uses two minor utility functions from ls-hdf5 , but no others (yet).

(file #33172)

Are you saying that we should improve our current ls-hdf5 using the code from this patch? Or that our ls-hdf5 is really meant to be used by our load/save functions so we should leave the code from this patch in a separate file?

Two fixes made a new patch necessary.

Can somebody compare this to Matlab's behaviour, please? In particular: whether the data order is correct, and especially if vectors come out the same. I don't have a very new version of Matlab.

(file #33140)

Hello everyone,

I would like to propose the integration some HDF5 related functions into octave core, maybe even for the new 4.0?

This patch defines the functions h5read, h5readatt, h5write, h5writeatt, h5create, see help strings.
This patch is equivalent to

https://github.com/stegro/hdf5oct/releases/tag/0.4.0

and should already integrate into Octave's build, if I did no mistake.
Finally, I was told to put everything into one rather long file, hope that's the way you want it.

NOTE: The original author of hdf5oct published his h5read function under the LGPL (I have not contacted him). Thorsten Liebig (openEMS) published his h5readatt under the GPL but agrees to any license, if only the stuff makes it into Octave, he says. I changed them both a lot and added new features. Is it necessary to/should one change the hdf5oct license to GPL in order to integrate this patch?

have fun, and thanks again for the hints on #octave

Stefan