bugGNU Octave - Bugs: bug #38928, configure -> hdf5 doesn't...


bug #38928: configure -> hdf5 doesn't handle path through

Submitted by:  None
Submitted on:  Tue 07 May 2013 01:57:51 PM UTC  
Category: Configuration and Build SystemSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Build Failure
Status: ConfirmedAssigned to: None
Originator Name: Clemens1509Originator Email: -unavailable-
Open/Closed: OpenRelease: dev
Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):

You are not logged in

Please log in, so followups can be emailed to you.


(Jump to the original submission Jump to the original submission)

Tue 23 Sep 2014 04:30:16 AM UTC, comment #18:

Per the discussion at OctConf 2014, the --with-XXX-includedir and --with-XXX-libdir are going to be removed. To build, follow Mike's suggestion and add any necessary directories to either CPPFLAGS or LDFLAGS before compiling.

Rik <rik5>
Project Administrator
Tue 09 Sep 2014 05:31:00 PM UTC, comment #17:

As a workaround, here is how I build Octave with HDF5 in Debian unstable (which has been moved to non-standard include and lib directories):

This is similar to what I originally suggested in comment #2,
essentially using the global CPPFLAGS and LDFLAGS instead of the --with-hdf5-* options.

I think the best approach is to try to remove HDF5 from the Octave public API as much as possible, which in turn means we wouldn't have to add HDF5 flags all over the place. Since that's a big separate issue on its own, I'll open a separate bug report about that.

Mike Miller <mtmiller>
Project Member
Wed 06 Aug 2014 06:07:01 PM UTC, comment #16:

If you see a way to hide the hid_t type, then yeah, I'm all for that.

I'm also in favor of cleaning up the set of installed header files.

And cleaning up the config.h that we install (I think there is another bug report about that mess).

John W. Eaton <jwe>
Project Administrator
Wed 06 Aug 2014 06:02:30 PM UTC, comment #15:

Yes, adding either HDF5_CPPFLAGS or ALL_CPPFLAGS gets past that one rule. There are other specific .df rules in libinterp//module.mk that also need HDF5_CPPFLAGS added. And there are sources, e.g. builtins.cc, octave.cc, libgui/src, libgui/graphics, that also need it. I can try to iteratively go through and add it to the minimal number of rules that need it.

At the extreme opposite end, would it be a terrible thing to define our own "hid_t" type, it's only an int? Or to be really safe, our own opaque handle type that would map to an hid_t type? This would potentially remove the need for HAVE_HDF5 and hdf5.h to be in any public header files. From an encapsulation point of view this would be a win, it would be far better for users to not even have to see this conditional and depend on these third-party header files.

Of course now that I'm chasing which files include other files, this implies massively cleaning up the set of files installed in the include directory, I believe there are many files installed that don't need to be, for example ls-.h, ov-.cc. The ls-hdf5.h header file contains calls to HDF5 functions, but it is never needed by any external user of Octave.

Mike Miller <mtmiller>
Project Member
Wed 06 Aug 2014 05:48:08 PM UTC, comment #14:

Hopefully Mike can try some of the permutations. If you look at the .deps/.Plo files though you see that we are generating dependencies on hdf5.h for huge numbers of files which inherit through the ov.h header files.

For example, libinterp/operators/.deps/*.Plo has 127 files that depend on hdf5.h, but these don't really "depend" in the sense of actually using function prototypes from hdf5.h.

Rik <rik5>
Project Administrator
Wed 06 Aug 2014 05:39:54 PM UTC, comment #13:

Or, those rules should be using ALL_CPPFLAGS instead of just CPPFLAGS.

John W. Eaton <jwe>
Project Administrator
Wed 06 Aug 2014 05:36:44 PM UTC, comment #12:

I don't think the default pattern rule needs all the extra CPPFLAGS variables.

There are lines like

in the dldfcn/module.mk file. Can't something similar be used for those files that need the extra LLVM_CPPFLAGS or HDF5_CPPFLAGS?

John W. Eaton <jwe>
Project Administrator
Wed 06 Aug 2014 04:59:16 PM UTC, comment #11:

Oy. This looks like it might be a bigger problem than I thought. It appears that maybe the rule for generating .df files from .cc files should have every single configurable CPPFLAGS variable in it.

The code in libinterp/Makefile.am is:

How far does the build proceed if you add $(HDF5_CPPFLAGS) after $(LLVM_CPPFLAGS)?

Rik <rik5>
Project Administrator
Wed 06 Aug 2014 01:43:51 AM UTC, comment #10:

Sure, a clean build with

fails in many, many places.

I get the same missing header file error with generation of the .df files in corefcn, octave-value, with libinterp/octave.cc, libinterp/builtins.cc, libinterp/corefcn/oct-errno.cc, just to name a few.

I think the problem is basically because hdf5.h is included in the public API and leaks into all ov*.h header files. After a successful (patched) build, I count almost 400 files that depend on hdf5.h, looking in the generated .Plo and .Po files. So it may have to be added to essentially all CPPFLAGS in libinterp and libgui.

Mike Miller <mtmiller>
Project Member
Tue 05 Aug 2014 10:58:38 PM UTC, comment #9:

@Mike: Could you post the error log for the build around the problem in libinterp/corefcn? Also, what is the value of HDF5_CPPFLAGS in libinterp/Makefile? Did you re-run bootstrap and configure after updating to the new Debian locations for HDF5 files?

Rik <rik5>
Project Administrator
Tue 05 Aug 2014 10:26:37 PM UTC, comment #8:

I am now able to test this and it is not fully resolved with the changes on default.

Debian has new HDF5 packages coming down the pike that move the header files and libraries into namespaced subdirectories. Building default against these new packages fails in libinterp/corefcn with "hdf5.h: No such file or directory" so we are still missing some places where the flags are needed.

The Debian HDF5 maintainer has proposed a couple of patches to Octave to solve this issue, attached here. The first patch adds HDF5_CPPFLAGS to everything in libinterp and libgui/src, I had to add also to libgui/graphics (original patch was based on 3.8.1). With this I get a successful build. The second patch adds HDF5 flags to mkoctfile, I'm not sure if this is necessary for building Forge packages with mkoctfile or what, I haven't fully tested that path yet.

(file #31844, file #31845)

Mike Miller <mtmiller>
Project Member
Tue 14 Jan 2014 06:19:40 PM UTC, comment #7:

No response so I'll assume the patch worked. Closing report.

Rik <rik5>
Project Administrator
Wed 06 Nov 2013 12:08:34 AM UTC, comment #6:

I added the HDF5_CPPFLAGS where I think they were necessary. Hopefully the original reporter, Clemens, can verify that everything builds correctly now. Use the development branch code with a changeset >= 17858:25601fe8c6f7.

Rik <rik5>
Project Administrator
Wed 30 Oct 2013 12:41:19 AM UTC, comment #5:

Yes, the HDF5_CPPFLAGS and possibly HDF5_LDFLAGS need to be added to more modules, probably both corefcn and one or more dldfcns, possibly also the GUI, not sure how many modules touch HDF5.

Unfortunately I don't have a way to test this at the moment, but hopefully Clemens still can to verify a fix.

Mike Miller <mtmiller>
Project Member
Tue 29 Oct 2013 11:53:45 PM UTC, comment #4:

Mike, was there something we needed to change in the build system for this?

It looks like maybe the module.mk for libinterp/corefcn needs to be changed to add HDF5_CPPFLAGS.

This line is adding some special CPPFLAGS, maybe it needs more?

Rik <rik5>
Project Administrator
Mon 13 May 2013 12:05:16 PM UTC, comment #3:

ok - that workaround works. Maybe let the bug open until fixed.

Clemens <clemens1509>
Thu 09 May 2013 02:00:32 AM UTC, comment #2:

Ok, your 2nd config.log attachment looks correct to me. I think there might be a bug in the way HDF5_CPPFLAGS is not used everywhere it is needed.

Can you run configure the way I suggested and report back whether that builds successfully, in order to confirm that the problem is with the Makefile rules and not with your particular setup?

Mike Miller <mtmiller>
Project Member
Tue 07 May 2013 04:49:15 PM UTC, comment #1:

The attachment doesn't look like a complete config.log, it was interrupted. Can you attach one that completed when you got a successful configure where you were able to run make?

Can you also try configuring Octave the following way instead, this is the technique I use and it works for me regularly:

If this works and the --with-hdf5 options didn't, then there may indeed be a minor bug in one of the make files.

Mike Miller <mtmiller>
Project Member
Tue 07 May 2013 01:57:51 PM UTC, original submission:

configure looks like:
./configure --prefix=/home//dev/octave-3.6.4/install --with-hdf5-includedir=/home//dev/hdf5-1.8.10/install/include/ --with-hdf5-libdir=/home//dev/octave-3.6.4/../hdf5-1.8.10/install/lib/



(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):

Attached Files
file #31844:  hdf5-flags.patch added by mtmiller (1kB - text/x-patch)
file #31845:  hdf5-mkoctfile.patch added by mtmiller (2kB - text/x-patch)
file #28045:  config2.log.gz added by None (116kB - application/x-gzip-compressed)
file #28039:  config.log added by None (30kB - text/x-log)


   bug dependencies.

   bug dependencies.


Carbon-Copy List
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by clemens1509 (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by None (Submitted the item)

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.


    Please enter the title of George Orwell's famous dystopian book (it's a date):



    Follow 15 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Mon 15 Sep 2014 03:07:01 PM UTCmtmillerDependencies-=>bugs #43226 is dependent
    Tue 09 Sep 2014 05:40:28 PM UTCmtmillerDependencies-=>Depends on bugs #43180
    Tue 05 Aug 2014 10:26:37 PM UTCmtmillerAttached File-=>Added hdf5-flags.patch, #31844
      Attached File-=>Added hdf5-mkoctfile.patch, #31845
    Tue 14 Jan 2014 06:19:40 PM UTCrik5StatusReady For Test=>Fixed
    Mon 11 Nov 2013 07:29:51 PM UTCrik5Summaryconfugure -> hdf5 doesn\'t handle path through=>configure -> hdf5 doesn't handle path through
    Wed 06 Nov 2013 12:08:34 AM UTCrik5StatusConfirmed=>Ready For Test
    Mon 13 May 2013 12:38:06 PM UTCmtmillerStatusNeed Info=>Confirmed
    Wed 08 May 2013 06:06:55 AM UTCNoneAttached File-=>Added config2.log.gz, #28045
    Tue 07 May 2013 04:49:15 PM UTCmtmillerStatusNone=>Need Info
    Tue 07 May 2013 01:57:51 PM UTCNoneAttached File-=>Added config.log, #28039

    Back to the top

    Powered by Savane 3.1-cleanup