bugGNU Octave - Bugs: bug #45578, build libgnu using non-recursive...

 
 

bug #45578: build libgnu using non-recursive automake rules

Submitter:  John Donoghue <lostbard>
Submitted:  Sun 19 Jul 2015 02:10:47 PM UTC
   
 
Category:  Configuration and Build System Severity:  2 - Minor
Priority:  1 - Later Item Group:  Build Failure
Status:  Postponed Assigned to:  None
Originator Name:  JohnD Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 09 Mar 2018 10:01:54 PM UTC, comment #14: 

Yeah I think the risk of a "build failure" like the one that was occurring before is now small to nil.

It would still be nice to have a full non-recursive build with proper dependencies on libgnu, but that's more of a build optimization at this point.

 Mike Miller <mtmiller>
Group Member
Thu 08 Mar 2018 10:19:08 PM UTC, comment #13: 

No, this is pretty minor.  The main people it could occasionally hit are developers.  If anything goes really wrong I just wipe things out and bootstrap over anyways.  I lowered both the severity and the priority.

 Rik <rik5>
Group administrator
Thu 08 Mar 2018 09:42:53 PM UTC, comment #12: 

Does this still need to be classified as an "important" issue?  The build seems to be working pretty well now.  It would be nice if the gnulib part was not done with a recursive make invocation, but are there any significant problems with the current arrangement?

 John W. Eaton <jwe>
Group administrator
Fri 08 Jul 2016 02:18:36 AM UTC, comment #11: 

I pushed a minor fixup

http://hg.savannah.gnu.org/hgweb/octave/rev/d9643bc3792a

I may look again at the partial work to try to build libgnu as an include file rather than a subdirectory make file.

This won't improve the build much over what we have now other than "cleanliness".

The overall issue of dependencies on generated files, say a header file that is built from a shell script, still remains.

However, I am able to do a full build of 'make dist' after doing some sneaky things to build all of the $(BUILT_SOURCES) first. So this proves to me that, if we were to have a way to add a dependency on $(BUILT_SOURCES), everything else actually does build with the right dependencies now.

To be specific, the following does work for me:


make maintainer-clean ## just to start from a clean slate
../configure
printf 'asdf:\n\t$(MAKE) $(BUILT_SOURCES)\n-include Makefile\n' | make -f /dev/stdin asdf
cd libgnu
printf 'asdf:\n\t$(MAKE) $(BUILT_SOURCES)\n-include Makefile\n' | make -f /dev/stdin asdf
cd ..
make dist


Now the 'dist' target, which does not depend on 'all' or 'check' or $(BUILT_SOURCES), builds successfully, including all of the things needed to actually build and run Octave to generate the images for the docs.

After building 'dist' I can 'make all' and show that only a handful of files are built that weren't needed for 'dist', including the -tst files, the .qm files under libgui, and the .gdbinit files useful for ./run-octave -g.

However, beyond that hack, I don't know how to actually make every compilation depend on $(BUILT_SOURCES), the automake manual talks about this and a few different approaches, but it looks like we may be approaching the best we can do.

 Mike Miller <mtmiller>
Group Member
Tue 01 Mar 2016 05:57:03 PM UTC, comment #10: 

I added a partial fix for libgnu not being built early enough in this cset (http://hg.savannah.gnu.org/hgweb/octave/rev/ad1dbbd23f1d).  It is really only a band-aid that works when running 'make all', as is the case when MXE is used, but does nothing to put the real list of dependencies into the Makefile.

 Rik <rik5>
Group administrator
Thu 11 Feb 2016 10:49:11 PM UTC, comment #9: 

Should we apply the patch from comment #6 at least temporarily until there is a clean way to handle the dependency on libgnu?

 Rik <rik5>
Group administrator
Tue 01 Sep 2015 03:33:48 AM UTC, comment #8: 

I would definitely prefer to switch to a non-recursive build for libgnu.  I had something that was working but wasn't really satisfied with the way it was done.  There are better ways, but they will require more work.  For now, we can probably just use a modification of what was done for bison's build system.

 John W. Eaton <jwe>
Group administrator
Tue 01 Sep 2015 01:11:48 AM UTC, comment #7: 

I think Automake always does a make in SUBDIRS first, followed by making the "all" targets in the current directory. So a normal "make all" just works, at least most of the time.

Another thing that is implied are the files in BUILT_SOURCES, which are made before any other of the targets for "make all". Some of these are in the source tarball, some are not.

I haven't had a lot of time to look at the build system lately, but jwe and I were looking at some diffs to make libgnu non-recursively. If we need a workaround in the meantime, we could try adding a dependency rule on libgnu/libgnu.la as you suggested. I don't know if it's realistic to try to capture the dependencies for all of the built source files.

 Mike Miller <mtmiller>
Group Member
Sun 30 Aug 2015 11:12:23 PM UTC, comment #6: 

@Mike: I think this has uncovered something rather fundamental.  The old build system was serialized by directory and in each directory the dependencies were correctly specified for make.  But now that all files in all directories can potentially be made in parallel there are a lot of dependencies that have yet to be specified.

The old SUBDIRS variable (in octave-stable) was


SUBDIRS = libgnu liboctave libinterp libgui


which built libgnu before everything.

Now it is just


SUBDIRS = libgnu


which seems to allow things that don't "depend" on libgnu like libinterp to run whenever they can be scheduled.  But of course, libinterp does depend on gnulib.  If I use "grep gnulib::" in libinterp I find 82 instances such as this one in rand.cc:


corefcn/rand.cc:                                      gnulib::floor (rvec[i] * (n - i));


gnulib is segregated with it's own system for constructing a Makefile, etc.  I think it will be easier for maintenance purposes to preserve that separation and just add some sort of ordering rule or dependency to create libgnu before the rest of the build.  My attempt, added to Makefile.am, is shown below.


 SUBDIRS = libgnu

+# All of build depends on having libgnu.
+# Add the library to BUILT_SOURCES so it is created early in the build process
+
+BUILT_SOURCES += libgnu/libgnu.la
+
+libgnu/libgnu.la:
+       cd libgnu; $(MAKE) all
+


It doesn't work perfectly however.  I think because BUILT_SOURCES may only apply when something like 'make all' is used.

The example you discovered, 'make src/octave-gui', is an excellent test bench.  I used 'make dist' to create a tar ball and then unpacked it into a temporary source directory.  I then switched to a build directory and configured using the temporary source directory.  When I try to create src/octave-gui I repeatedly ran into unmet dependencies.  See below the errors, and what command I used to overcome them.


In file included from ../octave-4.1.0+/liboctave/util/lo-regexp.h:32:0,
                 from ../octave-4.1.0+/libinterp/corefcn/symtab.h:34,
                 from ../octave-4.1.0+/libinterp/corefcn/load-save.h:30,
                 from ../octave-4.1.0+/libgui/src/files-dock-widget.cc:48:
../octave-4.1.0+/liboctave/array/Matrix.h:34:20: fatal error: mx-ops.h: No such file or directory

make liboctave/operators/mx-ops.h

--------------------------------------------------------------------------------

../octave-4.1.0+/libgui/src/m-editor/file-editor-tab.cc:61:21: fatal error: version.h: No such file or directory

make libinterp/version.h

--------------------------------------------------------------------------------

In file included from ../octave-4.1.0+/liboctave/array/Matrix.h:34:0,
                 from ../octave-4.1.0+/liboctave/util/lo-regexp.h:32,
                 from ../octave-4.1.0+/libinterp/corefcn/symtab.h:34,
                 from ../octave-4.1.0+/libinterp/corefcn/load-save.h:30,
                 from ../octave-4.1.0+/libgui/src/files-dock-widget.cc:48:
./liboctave/operators/mx-ops.h:4:23: fatal error: mx-cdm-dm.h: No such file or directory
compilation terminated.
make: *** [libgui/src/libgui_src_libgui_src_la-files-dock-widget.lo] Error 1
make: *** Waiting for unfinished jobs....
mv -f libgui/src/m-editor/.deps/libgui_src_libgui_src_la-file-editor-tab.Tpo libgui/src/m-editor/.deps/libgui_src_libgui_src_la-file-editor-tab.Plo

make liboctave/operators/mx-cdm-dm.h

--------------------------------------------------------------------------------

../octave-4.1.0+/libgui/src/settings-dialog.cc:31:32: fatal error: ui-settings-dialog.h: No such file or directory
compilation terminated.

make ./libgui/src/ui-settings-dialog.h

--------------------------------------------------------------------------------
../octave-4.1.0+/libgui/src/thread-manager.cc: In function 'void block_or_unblock_signal(int, int)':
../octave-4.1.0+/libgui/src/thread-manager.cc:115:3: error: 'gnulib' has not been declared
../octave-4.1.0+/libgui/src/thread-manager.cc:117:3: error: 'gnulib' has not been declared
make: *** [libgui/src/libgui_src_libgui_src_la-thread-manager.lo] Error 1
make: *** Waiting for unfinished jobs....

make -C libgnu all
--------------------------------------------------------------------------------

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I../octave-4.1.0+ -I/usr/include/freetype2 -I/usr/include/qt4 -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4/QtNetwork -I/usr/include/qt4/QtOpenGL -Ilibgui/graphics -I../octave-4.1.0+/libgui/graphics -Isrc -I../octave-4.1.0+/libgui/src -I../octave-4.1.0+/liboctave/cruft/misc -I../octave-4.1.0+/liboctave/array -I./liboctave/numeric -I../octave-4.1.0+/liboctave/numeric -I./liboctave/operators -I../octave-4.1.0+/liboctave/operators -I../octave-4.1.0+/liboctave/system -I../octave-4.1.0+/liboctave/util -I./libinterp -I../octave-4.1.0+/libinterp -I./libinterp/parse-tree -I../octave-4.1.0+/libinterp/parse-tree -I./libinterp/corefcn -I../octave-4.1.0+/libinterp/corefcn -I../octave-4.1.0+/libinterp/octave-value -pthread -fopenmp -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual -O2 -pipe -MT libgui/graphics/libgui_graphics_libgui_graphics_la-__init_qt__.lo -MD -MP -MF libgui/graphics/.deps/libgui_graphics_libgui_graphics_la-__init_qt__.Tpo -c ../octave-4.1.0+/libgui/graphics/__init_qt__.cc  -fPIC -DPIC -o libgui/graphics/.libs/libgui_graphics_libgui_graphics_la-__init_qt__.o
../octave-4.1.0+/libgui/graphics/__init_qt__.cc:34:22: fatal error: graphics.h: No such file or directory
compilation terminated.
make: *** [libgui/graphics/libgui_graphics_libgui_graphics_la-__init_qt__.lo] Error 1
make: *** Waiting for unfinished jobs....

make libinterp/corefcn/graphics.h

--------------------------------------------------------------------------------

../octave-4.1.0+/libgui/graphics/annotation-dialog.cc:28:34: fatal error: ui-annotation-dialog.h: No such file or directory
compilation terminated.
make: *** [libgui/graphics/libgui_graphics_libgui_graphics_la-annotation-dialog.lo] Error 1
make: *** Waiting for unfinished jobs....

make libgui/graphics/ui-annotation-dialog.h

--------------------------------------------------------------------------------


Some of them are rather obvious.  The code libgui/src/m-editor/file-editor-tab.cc includes version.h, but version.h is a generated file in libinterp and there is no dependency written in the libgui/module.mk file.  Is there an automated way to discover these dependencies?  The source code will obviously change and having to manually track these dependencies across directories seems awkward.



(file #34771, file #34772)

 Rik <rik5>
Group administrator
Tue 28 Jul 2015 08:33:51 PM UTC, comment #5: 

Leaving open for now as in mxe, we are just getting around the problem by doing an explicit build of libgnu, rather than an actual fix of the underlying issue

 John Donoghue <lostbard>
Group Member
Tue 28 Jul 2015 08:12:31 PM UTC, comment #4: 

It works now. Thanks!

Avinoam Kalma <avinoam>
Group Member
Thu 23 Jul 2015 04:39:37 AM UTC, comment #3: 

This build failure can definitely also be demonstrated outside of mxe-octave. I think the top-level Makefile has no rule telling it what steps depend on libgnu now, other than "make all". So it's currently a race condition that libgnu gets built before other steps that need it.

E.g. the following fails if I try to build a specific target rather than "all":


$ make src/octave-gui
…
make: *** No rule to make target 'libgnu/libgnu.la', needed by 'liboctave/liboctave.la'.  Stop.


IMHO the fix will be to merge libgnu into the top-level Makefile as an include as well, but I think that might need some work.

Mike Miller <mtmiller>
Group Member
Tue 21 Jul 2015 01:28:21 AM UTC, comment #2: 

that would be
make -C libgnu

 John Donoghue <lostbard>
Group Member
Tue 21 Jul 2015 01:27:50 AM UTC, comment #1: 

running make -C linbnu first, before running make for the toplevel circumvents the issue.

It then fails later trying to build the documentation (as it does have a native octave built)

 John Donoghue <lostbard>
Group Member
Sun 19 Jul 2015 02:10:47 PM UTC, original submission:  

ATtempting to build a current defaut branch octave using mxe-octave.


For octave:

hg tip
changeset:   20368:2f9119bb3fe5
bookmark:    @
tag:         tip
user:        Rik <rik@octave.org>
date:        Sat Jul 18 08:56:51 2015 -0700
summary:     build: Add EXTRA_DIST files frob libgui/ and test/ to tarball.

Failure is at:

rm -f libinterp/corefcn/data.df-t libinterp/corefcn/data.df-t1 libinterp/corefcn/data.df && \
        x86_64-w64-mingw32-g++ -E -DHAVE_CONFIG_H -I. -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+  \
          -I./liboctave -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/cruft/misc -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/array -I./liboctave/numeric -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/numeric -I./liboctave/operators -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/operators -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/system -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/liboctave/util -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/octave-value -Ilibinterp -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/operators -Ilibinterp/parse-tree -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/parse-tree -Ilibinterp/corefcn -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/corefcn -I./libgnu -I/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libgnu   -I/home/jade/mxe-octave/usr/x86_64-w64-mingw32/include \
          -pthread -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual \
          -DMAKE_BUILTINS /home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/corefcn/data.cc > libinterp/corefcn/data.df-t1 && \
        /home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/mkdefs /home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp /home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/corefcn/data.cc < libinterp/corefcn/data.df-t1 > libinterp/corefcn/data.df-t && \
        rm -f libinterp/corefcn/data.df-t1 && \
        mv libinterp/corefcn/data.df-t libinterp/corefcn/data.df
/home/jade/mxe-octave/tmp-default-octave/octave-4.1.0+/libinterp/corefcn/data.cc:31:23: fatal error: sys/times.h: No such file or directory
 #include <sys/times.h>
                       ^
compilation terminated.
make[2]: *** [libinterp/corefcn/data.df] Error 1



config.log reported that sys/select.h and sys/times.h was not found which is correct for mingw, so I am assuming gnulib will then replace them, however it looks like libgnu is not yet entered to build the files at this point.


John Donoghue <lostbard>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #34771:  build_libgnu.cset added by rik5 (1005B - application/octet-stream)
file #34772:  bad_build.notes added by rik5 (4KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one
bugsBuild error with parallel builds (bug #46964, Duplicate)

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by rik5
  • -email is unavailable- added by avinoam (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by lostbard (Submitted the item)

    •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-07-12 mtmiller StatusConfirmed Postponed
        Summarybuild failure due to no dependency on libgnu build libgnu using non-recursive automake rules
    2018-07-12 rik5 Priority3 - Low 1 - Later
    2018-03-08 rik5 Severity4 - Important 2 - Minor
        Priority5 - Normal 3 - Low
    2016-07-01 rik5 Severity3 - Normal 4 - Important
    2016-02-01 rik5 Dependencies- bugs #46964 is dependent
    2015-08-30 rik5 Attached File- Added build_libgnu.cset, #34771
        Attached File- Added bad_build.notes, #34772
    2015-08-30 rik5 Carbon-Copy- Added jwe
    2015-07-23 mtmiller StatusNone Confirmed
        Operating SystemGNU/Linux Any
        Summarymxe-octave default-octave build failure build failure due to no dependency on libgnu

    Back to the top

    Copyright © 2025 Free Software Foundation, Inc.
    Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
    The Levitating, Meditating, Flute-playing Gnu logo is a GNU GPL'ed image provided by the Nevrax Design Team.
    Page source code

    Powered by Savane 3.14-8e77.
    Corresponding source code