bugGNU Octave - Bugs: bug #53627, build: oct files should not need...

 
 

bug #53627: build: oct files should not need to link with -loctinterp -loctave

Submitter:  Mike Miller <mtmiller>
Submitted:  Thu 12 Apr 2018 06:52:15 PM UTC
   
 
Category:  Configuration and Build System Severity:  2 - Minor
Priority:  5 - Normal Item Group:  Feature Request
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 02 May 2021 11:20:57 AM UTC, comment #31: 

I pushed a modified version of Andrew's patch in comment #24 here:
https://hg.savannah.gnu.org/hgweb/octave/rev/b4cb9d04f3cf

Main changes to his original patch are:
- Update the help text for "--enable-no-undefined".
- No longer default to "-enable-link-all-dependencies" on macOS.
- Add path to Octave libraries in linker flags used by mkoctfile if built with "-enable-link-all-dependencies".

Markus Mützel <mmuetzel>
Group administrator
Fri 28 Dec 2018 08:20:35 PM UTC, comment #30: 

I'll just close this as fixed for now, since it seems to me like the original issue is addressed on GNU/Linux systems, and Windows or Cygwin systems can work around this by using --enable-link-all-dependencies. Whether this should be done automatically on those systems or not can be the subject of a separate bug report. And the same goes for macOS, if there are build issues related to linking all dependencies, that can be the subject of another bug report.

Mike Miller <mtmiller>
Group Member
Thu 27 Dec 2018 04:25:38 PM UTC, comment #29: 

it seems the new switch

  --enable-link-all-dependencies

is overcoming the problem. Building still running

marco atzeri <matzeri>
Thu 27 Dec 2018 11:48:18 AM UTC, comment #28: 

Currently the build on Cygwin is broken on both
stable and default.

config.status: executing liboctave/mk-version-h.sh commands
  CXXLD    libinterp/dldfcn/__delaunayn__.la
libinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `free_qhull_memory':
/pub/devel/octave/hg/octave-build/../octave/libinterp/dldfcn/__delaunayn__.cc:84: undefined reference to `warning(char const*, ...)'

as "warning" is in liboctinterp and it is not built at all
as prerequisite for all the oct files.


marco atzeri <matzeri>
Fri 14 Dec 2018 06:15:56 AM UTC, comment #27: 

I think what is going wrong on macos is related to bug #55033
is this correct?


Carlo de Falco <cdf>
Group Member
Fri 14 Dec 2018 01:57:17 AM UTC, comment #26: 

The original issue I reported is fixed on GNU/Linux systems. So from my point of view it can be closed fixed.

But starting with comment #8, something seems to have been broken on macOS, and is probably still broken. Probably should have been moved to a new bug report initially.

Mike Miller <mtmiller>
Group Member
Thu 13 Dec 2018 08:25:49 PM UTC, comment #25: 

This bug is marked as "Ready for Test".  Can it be closed now?

Rik <rik5>
Group administrator
Fri 03 Aug 2018 07:17:37 PM UTC, comment #24: 

Here's a revised attempt at this, which sets -no-undefined conditionally based on which OS it's running on, with it on by default only on Windows/BeOS/OS2/AIX. file #44700:  macos-bundle-undefined-dynamic_lookup-v3.patch

I've added "-bind_on_load" because I think that will produce better fail-fast behavior when loading octfiles with undefined symbols that cannot be resolved.

> What happens if you omit that option and do have undefined symbols? Does the program fail to load and run, or fail if it attempts to call an undefined function?


On macOS, with -no-undefined omitted, mkoctfile can compile files that have undefined symbols, but when they're called, it will produce an unresolved-symbol error (without crashing Octave). Also, you can build Octave without passing --enable-link-all-dependencies to configure.


$ cat helloworld_with_undef.cc
#include <octave/oct.h>

// Intentionally undefined function
int undefined_function (void);

DEFUN_DLD (helloworld_with_undef, args, nargout,
           "Hello World Help String")
{
  int nargin = args.length ();

  octave_stdout << "Hello World has "
                << nargin << " input arguments and "
                << nargout << " output arguments.\n";

  undefined_function ();

  return octave_value_list ();
}



octave:4> mkoctfile helloworld_with_undef.cc
octave:5> helloworld_with_undef
error: /Users/janke/tmp/octave/helloworld_with_undef.oct: failed to load: dlopen(/Users/janke/tmp/octave/helloworld_with_undef.oct, 10): Symbol not found: __Z18undefined_functionv
  Referenced from: /Users/janke/tmp/octave/helloworld_with_undef.oct
  Expected in: flat namespace
 in /Users/janke/tmp/octave/helloworld_with_undef.oct
octave:5>


I expect stuff in libinterp/dldfcn would behave similarly, but I'm not sure how to test that.

Andrew Janke <apjanke>
Fri 03 Aug 2018 07:05:59 PM UTC, comment #23: 


> Have you looked at the work that was done for bug #41574?


I looked through #41574, and I don't think anything from there needs to be pulled in here. Commented over there on the current state of things.

Andrew Janke <apjanke>
Thu 19 Jul 2018 10:04:15 PM UTC, comment #22: 

This is all very confusing and very operating system dependent.

I would think we want to follow the libtool default behavior, at least on as many operating systems as it is possible to do so. On Windows clearly it is not, DLLs need to be built against the stub library that contains the definitions for them to use. But on most other operating systems, it is a choice, and I think the choice should be to default to allowing undefined symbols.

Glancing through the libtool code, there is a set of linker flags for allowing undefined symbols, another set of flags for no undefined symbols, as well as whether one behavior or the other is unsupported.

On GNU/Linux, I think all of the flags are empty, so the linker is called the same way with or without the -no-undefined option.

I think on Unix operating systems we should not be using -no-undefined. It seems to only be required on Windows/MinGW/Cygwin, BeOS, OS/2, and AIX v3.

I think libtool will either error if you try to build for Windows without this option, or maybe build a static library instead of a shared object and print a warning.

Mike Miller <mtmiller>
Group Member
Thu 19 Jul 2018 08:26:15 PM UTC, comment #21: 

The -no-undefined option is needed when building Windows DLL files.

What happens if you omit that option and do have undefined symbols?  Does the program fail to load and run, or fail if it attempts to call an undefined function?

John W. Eaton <jwe>
Group administrator
Thu 19 Jul 2018 08:02:33 PM UTC, comment #20: 

I never realized that Octave was linking with -no-undefined by default, that was added before my time here. Is there a reason for that? I would think that option shouldn't normally be used on any system.

Mike Miller <mtmiller>
Group Member
Thu 19 Jul 2018 01:52:01 PM UTC, comment #19: 

Have you looked at the work that was done for bug #41574?

John W. Eaton <jwe>
Group administrator
Thu 19 Jul 2018 03:52:57 AM UTC, comment #18: 

I've attached file #44589:  macos-bundle-undefined-dynamic_lookup-v2.patch, which is my latest attempt to get `-undefined dynamic_lookup` working. It's getting closer, but not there yet.

With this, the build succeeds. Though I have to do an explicit `--disable-no-undefined` with `configure`.

The oct files it produces do not have the liboct/libinterp linkage.


$ otool -L __control_helper_functions__.oct
__control_helper_functions__.oct:
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)


And they work when run.


octave:4> pkg load control
octave:5> bode(ss(-1, 1, 1, 0))
octave:6>


However, the test suite crashes, and early.


octave:1> __run_test_suite__

Integrated test scripts:

  liboctave/array/Array.cc-tst ................................ PASS     21/21
  liboctave/array/CMatrix.cc-tst ..............................fatal: caught signal Segmentation fault: 11 -- stopping myself...
[1]    72183 segmentation fault  ./bin/octave


Mike, I'll give that "-Xlinker -undefined -Xlinker dynamic_lookup" a try if I have some time in the next couple days. And getting that libtool.m4 logic to activate may be the best way, but I'll need to do some work to figure out how to do that. But it's time to call it a night for now.

Andrew Janke <apjanke>
Thu 19 Jul 2018 03:37:17 AM UTC, comment #17: 

Yes, try "-Xlinker -undefined -Xlinker dynamic_lookup".

I see some logic in GNU libtool to automatically use these options, or other variants of them, depending on the name and the version of the operating system:


    case $host_os in
    rhapsody* | darwin1.[[012]])
      _lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
    darwin1.*)
      _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
    darwin*) # darwin 5.x on
      # if running on 10.5 or later, the deployment target defaults
      # to the OS version, if on x86, and 10.4, the deployment
      # target defaults to 10.4. Don't you love it?
      case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
        10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
          _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
        10.[[012]][[,.]]*)
          _lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
        10.*)
          _lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
      esac
    ;;
  esac


Mike Miller <mtmiller>
Group Member
Thu 19 Jul 2018 02:38:23 AM UTC, comment #16: 

I've attached a changeset, file #44587:  macos-bundle-undefined-dynamic_lookup.patch, that attempts to enable `-undefined dynamic_lookup` for octfiles and dynamic libraries by adding it to DL_LDFLAGS and MKOCTFILE_DL_LDFLAGS in acinclude.m4.

This isn't sufficient, though. When I do the build with this, it's still erroring out with undefined symbols.


libtool: link: g++ -std=gnu++11  -o libinterp/dldfcn/.libs/__delaunayn__.so -bundle  libinterp/dldfcn/.libs/__delaunayn___la-__delaunayn__.o   -lqhull -lm  -pthread -g -O2   -pthread
Undefined symbols for architecture x86_64:
  "print_usage()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "check_version(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:


The `-undefined dynamic_lookup` option is absent from the `g++` call there. Sounds like we need to tell `libtool` to use that option. Does anybody know how to do that?

Andrew Janke <apjanke>
Wed 18 Jul 2018 11:59:43 PM UTC, comment #15: 

And here's the Apple documentation for it:

See "Loading Plug-in Code With Bundles" under:
https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/MachOTopics/1-Articles/loading_code.html#//apple_ref/doc/uid/TP40001830

And this mirror of the "OS X ABI Dynamic Loader Reference":
https://github.com/jeremysears/osx-abi-dyamic-loader-reference

It doesn't really document the `-bundle` and `-undefined` `ld` options.

The O'Reilly book "Mac OS X for Unix Geeks" also discusses this in section 5.3. I don't see a legit publicly-available copy of that, though.

Andrew Janke <apjanke>
Wed 18 Jul 2018 11:29:53 PM UTC, comment #14: 

I don't really know how Python does things, but I think the macOS equivalent for plugin-style loadable modules is building them as "dynamically loaded modules" or "Mach-O bundles" (totally different from "app bundles") instead of a "shared library". 

The `-bundle` option to macOS's `ld` looks relevant, along with `-undefined dynamic_lookup`.

This is hard to Google because "bundle" seems to mean two different things in the context of macOS: a Mach-O plugin-style loadable module like we want here, and an "app bundle" directory structure, which is the more common usage.

In Python distutils' tests, there's a reference to using `-bundle -undefined dynamic_lookup`, but I can't find it in the main code. From `tests/test_unixccompiler.py`:


    @unittest.skipUnless(sys.platform == 'darwin', 'test only relevant for OS X')
    def test_osx_cc_overrides_ldshared(self):
        # Issue #18080:
        # ensure that setting CC env variable also changes default linker
        def gcv(v):
            if v == 'LDSHARED':
                return 'gcc-4.2 -bundle -undefined dynamic_lookup '
            return 'gcc-4.2'


It looks like octave is already making some use of this. In `configure`:


    *-*-darwin*)
      DL_LDFLAGS="-bundle -bundle_loader ${ac_top_build_prefix}libinterp/octave ${LDFLAGS}"
                  MKOCTFILE_DL_LDFLAGS='-bundle -bundle_loader ${bindir}/octave-${version}'"${EXEEXT}"
      SH_LDFLAGS="-dynamiclib -single_module ${LDFLAGS}"


and in the output from my broken Octave build below:


/bin/sh ./libtool  --tag=CXX   --mode=link g++ -std=gnu++11 -fPIC -D_THREAD_SAFE -pthread -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual  -g -O2 -avoid-version -module -no-undefined     -o libinterp/dldfcn/__delaunayn__.la -rpath /tmp/test-octave/lib/octave/5.0.0 libinterp/dldfcn/__delaunayn___la-__delaunayn__.lo   -lm
libtool: link: g++ -std=gnu++11  -o libinterp/dldfcn/.libs/__delaunayn__.so -bundle libinterp/dldfcn/.libs/__delaunayn___la-__delaunayn__.o   -lm -pthread -g -O2   -pthread


I can see that it's passing `-bundle`. It's also passing `-no-undefined`. Maybe that needs to be `-undefined dynamic_lookup` instead of `-no-undefined`? `-undefined dynamic_lookup` seems to be the magic sauce for having the linker allow unresolved symbols at compilation/initial link time. That's what the Tim Smith blog post links below indicates, for Python modules. It also mentions crashes when not using this, which sounds like the behavior we're seeing with octfiles and Homebrewed Octave on macOS.

> The best solution is to provide -bundle -undefined dynamic_lookup,
> which has the same behavior as the Linux linker.


Based on my reading of that and other Google results for Python modules, then yes, a single compiled extension is portable between two different instances of the same version of Python.

Some references:

https://stackoverflow.com/questions/2339679/what-are-the-differences-between-so-and-dylib-on-osx

https://blog.tim-smith.us/2015/09/python-extension-modules-os-x/

https://cmake.org/pipermail/cmake/2005-March/006249.html

https://books.google.com/books?id=H3JmBgAAQBAJ&pg=PA197&lpg=PA197

From `man ld` on macOS 10.13:


   Libraries
     A static library (aka static archive) is a collection of .o files with a table of contents that lists the global sym-
     bols in the .o files.  ld will only pull .o files out of a static library if needed to resolve some symbol reference.
     Unlike traditional linkers, ld will continually search a static library while linking. There is no need to specify a
     static library multiple times on the command line.

     A dynamic library (aka dylib or framework) is a final linked image.  Putting a dynamic library on the command line
     causes two things: 1) The generated final linked image will have encoded that it depends on that dynamic library. 2)
     Exported symbols from the dynamic library are used to resolve references.

[...]
     -undefined treatment
                 Specifies how undefined symbols are to be treated. Options are: error, warning, suppress, or
                 dynamic_lookup.  The default is error.


Looks like Lua does the same thing for its dynamic modules:
http://lua-users.org/lists/lua-l/2005-03/msg00349.html


Andrew Janke <apjanke>
Wed 18 Jul 2018 09:51:30 PM UTC, comment #13: 

The linker errors here


  CXXLD    libinterp/dldfcn/__delaunayn__.la
Undefined symbols for architecture x86_64:
  "print_usage()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "check_version(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      _G__delaunayn__ in __delaunayn___la-__delaunayn__.o
  "error(char const*, ...)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
...


seem to indicate that the macOS linker requires some resolution of undefined symbols even in a plugin-type shared object.

On GNU/Linux, the advice for building plugin-style loadable modules is to avoid linking them with libraries that the executable will already have opened in the application process space.

For example, when building Python extensions in C, the .so files are not linked against libpython, even though the extension will be calling Python C API functions. At build time, these are unresolved symbols, but the linker does not care.

I guess we need to know what the equivalent build process should be on macOS and we'll do it.

How are C Python extensions built on macOS? Is a single compiled extension portable between two different instances of the same version of Python?

Mike Miller <mtmiller>
Group Member
Wed 18 Jul 2018 09:19:14 PM UTC, comment #12: 

While we're on this subject: it would be nice if oct files also did not link to liboctinterp and liboctave on macOS. The presence of those dylib links means (I think) that a given octfile is only compatible with an Octave that is installed in the same location as the original Octave that produced those octfiles.

For example, this is what I see for the octfiles produced by the control package:



$ pwd
/Users/janke/octave/control-3.1.0/x86_64-apple-darwin17.6.0-api-v52
$ otool -L __control_helper_functions__.oct
__control_helper_functions__.oct:
        /usr/local/opt/octave/lib/octave/4.4.0/liboctinterp.5.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/local/opt/octave/lib/octave/4.4.0/liboctave.5.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)


This has shown up as an issue for a user who's interested in using both a Mac-style Octave.app and a Homebrew-installed octave on the same system: http://lists.gnu.org/archive/html/octave-maintainers/2018-07/msg00101.html. It's of interest to me also, because I would like to be able to install multiple versions and variants of Octave side-by-side on the same machine for testing.

Is there anything I can do to help figure out how to do this on macOS?

Andrew Janke <apjanke>
Wed 18 Jul 2018 09:14:51 PM UTC, comment #11: 

That other build failure is fixed now.

I have updated to the latest default branch in the Hg repo, and am still getting a link-related build failure.


  CXXLD    libinterp/dldfcn/__delaunayn__.la
Undefined symbols for architecture x86_64:
  "print_usage()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "check_version(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      _G__delaunayn__ in __delaunayn___la-__delaunayn__.o
  "error(char const*, ...)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "warning(char const*, ...)", referenced from:
      free_qhull_memory() in __delaunayn___la-__delaunayn__.o
  "MatrixType::MatrixType()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "MatrixType::~MatrixType()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "dim_vector::nil_rep()", referenced from:
      octave_value_list::octave_value_list() in __delaunayn___la-__delaunayn__.o
  "octave_value::nil_rep()", referenced from:
      octave_value_list::elem(long long) in __delaunayn___la-__delaunayn__.o
      Array<octave_value>::ArrayRep::ArrayRep(octave_value*, long long) in __delaunayn___la-__delaunayn__.o
  "octave_value::octave_value(Matrix const&, MatrixType const&)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "octave_value::octave_value(RowVector const&)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "octave_value::octave_value(double)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "octave_dld_function::create(octave_value_list (*)(octave_value_list const&, int), octave::dynamic_library const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      _G__delaunayn__ in __delaunayn___la-__delaunayn__.o
  "Array<octave_value>::resize(dim_vector const&, octave_value const&)", referenced from:
      octave_value_list::resize(long long, octave_value const&) in __delaunayn___la-__delaunayn__.o
  "Array<octave_value>::nil_rep()", referenced from:
      octave_value_list::octave_value_list() in __delaunayn___la-__delaunayn__.o
  "Array<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::nil_rep()", referenced from:
      octave_value_list::octave_value_list() in __delaunayn___la-__delaunayn__.o
  "Array<double>::fortran_vec()", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "dim_vector::safe_numel() const", referenced from:
      Array<double>::Array(dim_vector const&) in __delaunayn___la-__delaunayn__.o
  "dim_vector::redim(int) const", referenced from:
      Array<double>::as_matrix() const in __delaunayn___la-__delaunayn__.o
  "Array<octave_value>::resize_fill_value() const", referenced from:
      vtable for Array<octave_value> in __delaunayn___la-__delaunayn__.o
  "Array<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::resize_fill_value() const", referenced from:
      vtable for string_vector in __delaunayn___la-__delaunayn__.o
      vtable for Array<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > in __delaunayn___la-__delaunayn__.o
  "Array<double>::resize_fill_value() const", referenced from:
      vtable for MArray<double> in __delaunayn___la-__delaunayn__.o
      vtable for Array<double> in __delaunayn___la-__delaunayn__.o
      vtable for Matrix in __delaunayn___la-__delaunayn__.o
      vtable for NDArray in __delaunayn___la-__delaunayn__.o
      vtable for RowVector in __delaunayn___la-__delaunayn__.o
  "Array<double>::transpose() const", referenced from:
      Matrix::transpose() const in __delaunayn___la-__delaunayn__.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libinterp/dldfcn/__delaunayn__.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2


I'm attaching bug53627_example01.tgz with details of the files produced in my build.

(file #44584)

Andrew Janke <apjanke>
Sat 14 Jul 2018 06:24:30 AM UTC, comment #10: 

I'm getting a different, apparently unrelated build failure on macOS now, so I can't confirm the fix.



  YACC     libinterp/corefcn/oct-tex-parser.h
  YACC     libinterp/parse-tree/oct-parse.h
  GEN      test/conv.tst
  GEN      test/sparse.tst
/Users/janke/local/repos/octave-BUILD/libinterp/corefcn/oct-tex-parser.yy:53.9-16: syntax error, unexpected identifier, expecting string
make: *** [libinterp/corefcn/oct-tex-parser.h] Error 1
make: *** Waiting for unfinished jobs....
/Users/janke/local/repos/octave-BUILD/libinterp/parse-tree/oct-parse.yy:130.9-16: syntax error, unexpected identifier, expecting string
make: *** [libinterp/parse-tree/oct-parse.h] Error 1


Will look in to this when I have time.

Andrew Janke <apjanke>
Wed 11 Jul 2018 11:51:23 PM UTC, comment #9: 

Is this still a problem for MacOS builds?  Or can we close this report?

Rik <rik5>
Group administrator
Sun 01 Jul 2018 08:13:20 AM UTC, comment #8: 

I think this may have broken the build on Mac, too.

I am building Octave on macOS 10.13.5. I can build the 4.4.0 release okay, but building from the tip of the Hg repo is broken with the following build error.


$ make V=1
rm -f bits/stl_algo.h
/Applications/Xcode-9.4.app/Contents/Developer/usr/bin/make all-recursive
Making all in libgnu
/Applications/Xcode-9.4.app/Contents/Developer/usr/bin/make all-recursive
make[4]: Nothing to be done for `all-am'.
/bin/sh ./libtool  --tag=CXX   --mode=link g++ -std=gnu++11 -fPIC -D_THREAD_SAFE -pthread -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual  -g -O2 -avoid-version -module -no-undefined     -o libinterp/dldfcn/__delaunayn__.la -rpath /tmp/test-octave/lib/octave/5.0.0 libinterp/dldfcn/__delaunayn___la-__delaunayn__.lo   -lm
libtool: link: g++ -std=gnu++11  -o libinterp/dldfcn/.libs/__delaunayn__.so -bundle libinterp/dldfcn/.libs/__delaunayn___la-__delaunayn__.o   -lm -pthread -g -O2   -pthread
Undefined symbols for architecture x86_64:
  "check_version(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      _G__delaunayn__ in __delaunayn___la-__delaunayn__.o
  "err_disabled_feature(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      F__delaunayn__(octave_value_list const&, int) in __delaunayn___la-__delaunayn__.o
  "octave_dld_function::create(octave_value_list (*)(octave_value_list const&, int), octave::dynamic_library const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
      _G__delaunayn__ in __delaunayn___la-__delaunayn__.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libinterp/dldfcn/__delaunayn__.la] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2


I'm configuring using the following:


../octave/configure --prefix=/tmp/test-octave --disable-readline --without-fltk --without-x --without-qhull


Switching to shared libs doesn't fix it; and doing --enable-link-all-dependencies breaks the build with a different error.

Maybe this needs to be conditionalized for macOS as well as Windows?

Andrew Janke <apjanke>
Thu 14 Jun 2018 08:33:40 PM UTC, comment #7: 
John W. Eaton <jwe>
Group administrator
Wed 13 Jun 2018 08:31:49 PM UTC, comment #6: 

Same issue is seen when cross compiling any of the octave forge packages .oct files using mkoctfile in windows, so I guess it does need be conditional for windows.

John Donoghue <lostbard>
Group Member
Wed 13 Jun 2018 08:14:45 PM UTC, comment #5: 

Yes, I assume so.  If there is no better solution, then I can make the change conditional so it works as before on Windows systems.

John W. Eaton <jwe>
Group administrator
Wed 13 Jun 2018 11:43:10 AM UTC, comment #4: 

Im assing this is directlt related to this change.

In mxe with latest dev (1c4ffd4f9404: create valid tmp scope for who -file )



/bin/sh ./libtool  --tag=CXX   --mode=compile x86_64-w64-mingw32-g++ -DHAVE_CONFIG_H -I. -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0  -Iliboctave -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/array -Iliboctave/numeric -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/numeric -Iliboctave/operators -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/operators -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/system -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/util -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/octave-value -Ilibinterp -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/operators -Ilibinterp/parse-tree -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/parse-tree -Ilibinterp/corefcn -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/corefcn -I/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/wrappers  -I/home/johnd/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/GraphicsMagick -I/home/johnd/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/libxml2 -I/home/johnd/mxe-octave/build-64/usr/x86_64-w64-mingw32/include  -I/home/johnd/mxe-octave/build-64/usr/x86_64-w64-mingw32/include  -fopenmp -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual  -g -O2 -MT libinterp/dldfcn/libinterp_dldfcn___delaunayn___la-__delaunayn__.lo -MD -MP -MF libinterp/dldfcn/.deps/libinterp_dldfcn___delaunayn___la-__delaunayn__.Tpo -c -o libinterp/dldfcn/libinterp_dldfcn___delaunayn___la-__delaunayn__.lo `test -f 'libinterp/dldfcn/__delaunayn__.cc' || echo '/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/'`libinterp/dldfcn/__delaunayn__.cc
ylibinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `octave_value_list::resize(long long, octave_value const&)':
/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/octave-value/ovl.h:102: undefined reference to `Array<octave_value>::resize(dim_vector const&, octave_value const&)'
libinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `octave_value::octave_value()':
/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/octave-value/ov.h:175: undefined reference to `octave_value::nil_rep()'
libinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `dim_vector::dim_vector()':
/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/array/dim-vector.h:255: undefined reference to `dim_vector::nil_rep()'
libinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `F__delaunayn__(octave_value_list const&, int)':
/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/libinterp/dldfcn/__delaunayn__.cc:152: undefined reference to `error(char const*, ...)'
libinterp/dldfcn/.libs/libinterp_dldfcn___delaunayn___la-__delaunayn__.o: In function `MArray<double>::transpose() const':
/home/johnd/mxe-octave/build-64/tmp-default-octave/octave-5.0.0/liboctave/array/MArray.h:104: undefined reference to `Array<double>::transpose() const'
.
.
.


John Donoghue <lostbard>
Group Member
Mon 11 Jun 2018 07:14:48 PM UTC, comment #3: 

I pushed the following changeset:

http://hg.savannah.gnu.org/hgweb/octave/rev/fa66d81d0956

If this causes trouble on Mac or Windows systems, then we can undo whatever is needed and make it conditional.  It would also be unfortunate, since it will make building and testing MEX files in the build directory a little more complicated.

John W. Eaton <jwe>
Group administrator
Mon 11 Jun 2018 06:10:19 PM UTC, comment #2: 

If we remove "-loctinterp -loctave" from the link command, then I think we can also eliminate the DEFAULT_LFLAGS variable.  We can still accept LFLAGS from the environment, but there is no need for a default.

Since I'm looking at a way to build MEX files for testing and it would be most convenient if we didn't have to worry about linking with liboctave and liboctinterp, I can look at making these changes now.

John W. Eaton <jwe>
Group administrator
Wed 23 May 2018 09:20:58 PM UTC, comment #1: 

What is the minimum unit of change here?

It seems like we could start with modifying the mkoctfile command to remave '-loctinterp -loctave'.  Is that expected to be safe on all platforms, or just Linux?

The next step would be to remove LIBADD from the .oct files, and then test on Linux, macOS, and Windows.

Final step would be to eliminate DLD_LIBOCTINTERP_LIBADD as well.

Does that sound right?

Rik <rik5>
Group administrator
Thu 12 Apr 2018 06:52:15 PM UTC, original submission:  

This applies equally to the .oct files that are built as part of Octave, as well as the mkoctfile rules for building external .oct files.

On GNU/Linux, a shared object that is a plugin or loadable module doesn't need to link with the base libraries of the program that is loading it. Since Octave is doing the loading, all .oct files will have access to the functions in Octave's executable image.

I propose that we drop DLD_LIBOCTINTERP_LIBADD and remove both liboctinterp.la and liboctave.la from the LIBADD of each of the .oct files built in libinterp/dldfcn. I also propose that "-loctinterp -loctave" be removed from the mkoctfile build command.

But this will definitely need some testing on macOS and Windows to see whether this is safe or needs to be made conditional.

Mike Miller <mtmiller>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44700:  macos-bundle-undefined-dynamic_lookup-v3.patch added by apjanke (3KiB - application/octet-stream - Revised version with -no-undefined conditionally set based on OS)
file #44587:  macos-bundle-undefined-dynamic_lookup.patch added by apjanke (1KiB - application/octet-stream - patch to enable -undefined dynamic_lookup on macOS)
file #44584:  bug53627_example01.tgz added by apjanke (237KiB - application/gzip - files from build failure)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by matzeri (Posted a comment)
  • -email is unavailable- added by cdf (Posted a comment)
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by lostbard (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-28 mtmiller StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2018-12-14 mtmiller Carbon-CopyRemoved 80942 -
    2018-08-03 apjanke Attached File- Added macos-bundle-undefined-dynamic_lookup-v3.patch, #44700
    2018-07-19 apjanke Attached File- Added macos-bundle-undefined-dynamic_lookup-v2.patch, #44589
    2018-07-19 apjanke Attached File- Added macos-bundle-undefined-dynamic_lookup.patch, #44587
    2018-07-18 apjanke Attached File- Added bug53627_example01.tgz, #44584
    2018-07-17 rik5 Dependencies- Depends on bugs #54324
    2018-06-14 jwe StatusPostponed Ready For Test

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code