bugGNU Octave - Bugs: bug #50398, jit no longer compiles with Octave...

 
 

bug #50398: jit no longer compiles with Octave 4.2.1

Submitter:  Ceral Paquet <octavebugs>
Submitted:  Sat 25 Feb 2017 01:05:52 PM UTC
   
 
Category:  Configuration and Build System Severity:  2 - Minor
Priority:  3 - Low Item Group:  Regression
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 4.2.1 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 14 Dec 2017 04:34:09 PM UTC, comment #16: 

The workaround is simple: don't compile with the JIT compiler. It never grew out of the toy implementation and doesn't compile anything but toy examples.

Jordi Gutiérrez Hermoso <jordigh>
Group Member
Thu 14 Dec 2017 03:23:51 PM UTC, comment #15: 

I have this problem when compiling Octave 4.2.1 from source on CentOS 7.4. I use the following configure options:


./configure LLVM_CONFIG=/usr/bin/llvm-config --prefix=/usr/local/octave --enable-threads=posix --enable-jit --enable-address-sanitizer-flags --disable-java --with-openssl --disable-float-truncate --enable-bounds-check


My LLVM version is 3.4.2, so according to the previous comment it should build without an error, but it doesn't.

I have the following error:


../libinterp/corefcn/jit-typeinfo.cc:216:1: error: ‘octave_jit_octave’ has not been declared
 octave_jit_octave::err_nan_to_logical_conversion (void)
 ^
../libinterp/corefcn/jit-typeinfo.cc: In function ‘void err_nan_to_logical_conversion()’:
../libinterp/corefcn/jit-typeinfo.cc:216:1: error: redefinition of ‘void err_nan_to_logical_conversion()’
In file included from ../liboctave/array/Array-util.h:31:0,
                 from ../liboctave/operators/mx-inlines.cc:43,
                 from ../liboctave/array/MArray.h:30,
                 from ../liboctave/array/dNDArray.h:28,
                 from ../liboctave/array/dMatrix.h:28,
                 from ../liboctave/array/Range.h:30,
                 from ../libinterp/corefcn/jit-typeinfo.h:35,
                 from ../libinterp/corefcn/jit-typeinfo.cc:35:
../liboctave/util/lo-array-errwarn.h:164:1: error: ‘void err_nan_to_logical_conversion()’ previously defined here
 err_nan_to_logical_conversion (void)
 ^
../libinterp/corefcn/jit-typeinfo.cc: In constructor ‘jit_typeinfo::jit_typeinfo(llvm::Module*, llvm::ExecutionEngine*)’:
../libinterp/corefcn/jit-typeinfo.cc:1508:32: error: ‘octave_jit_octave’ has not been declared
     = create_external (JIT_FN (octave_jit_octave::err_nan_to_logical_conversion), 0);
                                ^
../libinterp/corefcn/jit-typeinfo.cc:1040:29: note: in definition of macro ‘JIT_FN’
 #define JIT_FN(fn) engine, &fn, #fn


It would be good to have this bug re-opened or at list get a link where it is explained how to workaround this issue. Thank you!

Yury <scrutari>
Thu 02 Mar 2017 08:01:29 PM UTC, comment #14: 

Verified that a normal build with llvm3.4 works and that the JIT is functional within Octave.  Closing report.

Rik <rik5>
Group administrator
Thu 02 Mar 2017 07:52:40 PM UTC, comment #13: 

Sorry for the fuss.  It was a dual installation of llvm 3.4 and 3.6 that was the problem.  It was picking up bits from both installs that was the issue.

Rik <rik5>
Group administrator
Thu 02 Mar 2017 07:51:41 PM UTC, comment #12: 

The functions DO seem to be in the library


root:~# nm -D /usr/lib/llvm-3.4/lib/libLLVM-3.4.so | c++filt | grep verifyModule
0000000000b32d60 T LLVMVerifyModule
0000000000558e40 T llvm::verifyModule(llvm::Module const&, llvm::VerifierFailureAction, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)
root:~# nm -D /usr/lib/llvm-3.4/lib/libLLVM-3.4.so | c++filt | grep verifyFunction
0000000000b32e00 T LLVMVerifyFunction
0000000000558af0 T llvm::verifyFunction(llvm::Function const&, llvm::VerifierFailureAction)


I tried setting the environment variables as you suggested, but it still fails to link.


libtool: link: g++ -std=gnu++11  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbeginS.o  -Wl,--whole-archive libgui/qterminal/.libs/libqterminal.a libgui/src/.libs/libgui-src.a libgui/graphics/.libs/libgui-graphics.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/home/rik/wip/Projects_Mine/octave-stable/libinterp/.libs -Wl,-rpath -Wl,/home/rik/wip/Projects_Mine/octave-stable/liboctave/.libs -Wl,-rpath -Wl,/home/rik/local/lib/octave/4.2.1 -L/usr/lib/llvm-3.4/lib libinterp/.libs/liboctinterp.so liboctave/.libs/liboctave.so -lQt5Network -lQt5OpenGL -lQt5PrintSupport -lQt5Widgets -lQt5Gui -lQt5Core -lqt5scintilla2 -lGL -lGLU -lutil -lLLVM-3.4 -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. -lstdc++ -lm -lgomp -lpthread -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/5/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o  -pthread -fopenmp -O2   -pthread -fopenmp -Wl,-soname -Wl,liboctgui.so.2 -o libgui/.libs/liboctgui.so.2.0.0
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyModule(llvm::Module const&, llvm::raw_ostream*)'
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyFunction(llvm::Function const&, llvm::raw_ostream*)


I looked and "-lLLVM-3.4" is in the link command above.

Also, this is a NEEDED library.


readelf -d libinterp/.libs/liboctinterp.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [liboctave.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libhdf5_serial.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libGraphicsMagick++.so.12]
 0x0000000000000001 (NEEDED)             Shared library: [libGraphicsMagick.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libGL.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libGLU.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfontconfig.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfreetype.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgl2ps.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libLLVM-3.4.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgomp.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]



Rik <rik5>
Group administrator
Thu 02 Mar 2017 07:26:28 PM UTC, comment #11: 

The functions llvm::verifyFunction and llvm::verifyModule are declared in the public header file "llvm/IR/Verifier.h", so I don't know why they would not be declared in the library as regular public symbols.

Running "nm -D /usr/lib/llvm-3.4/lib/libLLVM-3.4.so | c++filt | grep verify" should help clarify whether the symbols are exported by that library properly.

Mike Miller <mtmiller>
Group Member
Thu 02 Mar 2017 07:19:27 PM UTC, comment #10: 

Ok, if you want to try to force linking, try


setenv LDFLAGS "-L/usr/lib/llvm-3.4/lib"
setenv LIBS "-lLLVM-3.4"


This should put "-lLLVM-3.4" at the end of the command, with "-lutil -lm", which is where it needs to be to be useful.

Still, can you also verify that liboctinterp.so has libLLVM-3.4.so.1 in its header? It should show up in this list in your build directory:


$ readelf -d libinterp/.libs/liboctinterp.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [liboctave.so.4]
 0x0000000000000001 (NEEDED)             Shared library: [libhdf5_serial.so.100]
 0x0000000000000001 (NEEDED)             Shared library: [libGraphicsMagick++-Q16.so.12]
 0x0000000000000001 (NEEDED)             Shared library: [libGraphicsMagick-Q16.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3_threads.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3f_threads.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libfftw3f.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libGL.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libGLU.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfontconfig.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libfreetype.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgl2ps.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [liblapack.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libutil.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgomp.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]


Mike Miller <mtmiller>
Group Member
Thu 02 Mar 2017 07:12:25 PM UTC, comment #9: 

I cleared things with 'make distclean' and then re-configured after setting my environment variables to


setenv CFLAGS "-O2 -pipe"
setenv CXXFLAGS "${CFLAGS} -I/usr/lib/llvm-3.4/include"
setenv FFLAGS "${CFLAGS}"
setenv LDFLAGS "-L/usr/lib/llvm-3.4/lib -lLLVM-3.4"


As you can see below, the link still fails despite the fact that the library and directory are included in the path.


/bin/bash ./libtool  --tag=CXX   --mode=link g++ -std=gnu++11 -fPIC -pthread -fopenmp -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual -O2 -pipe -I/usr/lib/llvm-3.4/include -O2 -pipe -no-undefined   -L/usr/lib/llvm-3.4/lib -lLLVM-3.4 -o src/octave-cli src/src_octave_cli-main-cli.o src/src_octave_cli-octave-build-info.o libinterp/liboctinterp.la liboctave/liboctave.la libgnu/libgnu.la  -lutil -lm
libtool: link: g++ -std=gnu++11 -fPIC -pthread -fopenmp -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual -O2 -pipe -I/usr/lib/llvm-3.4/include -O2 -pipe -o src/.libs/octave-cli src/src_octave_cli-main-cli.o src/src_octave_cli-octave-build-info.o  -L/usr/lib/llvm-3.4/lib libinterp/.libs/liboctinterp.so liboctave/.libs/liboctave.so libgnu/.libs/libgnu.a -lLLVM-3.4 -lutil -lm -fopenmp -pthread -Wl,-rpath -Wl,/home/rik/local/lib/octave/4.2.1
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyModule(llvm::Module const&, llvm::raw_ostream*)'
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyFunction(llvm::Function const&, llvm::raw_ostream*)'
collect2: error: ld returned 1 exit status


Is the verifyModule only included if I use '-g' somehow?


Rik <rik5>
Group administrator
Thu 02 Mar 2017 06:40:26 PM UTC, comment #8: 

I think that's right. In Debian's llvm-3.8-dev package, both shared and static libraries are included. Probably because the API is so unstable, they intentionally give users the option to link with either the shared library or the modular static libraries.

Octave should prefer the shared library, LLVM_LIBS should be set to "-lLLVM-3.4", and liboctinterp.so should be pulling it in when octave-cli is linked.

Mike Miller <mtmiller>
Group Member
Thu 02 Mar 2017 06:14:13 PM UTC, comment #7: 

I re-titled the bug report to track the new issue.

I installed all of what appeared to be llvm3.4 packages.


root:~# dpkg -l | grep llvm | grep 3.4
ii  libllvm3.4:amd64                              1:3.4.2-13ubuntu2.1                        amd64        Modular compiler and toolchain technologies, runtime library
ii  llvm-3.4                                      1:3.4.2-13ubuntu2.1                        amd64        Modular compiler and toolchain technologies
ii  llvm-3.4-dev                                  1:3.4.2-13ubuntu2.1                        amd64        Modular compiler and toolchain technologies, libraries and headers
ii  llvm-3.4-runtime                              1:3.4.2-13ubuntu2.1                        amd64        Modular compiler and toolchain technologies, IR interpreter
ii  llvm-3.4-tools                                1:3.4.2-13ubuntu2.1                        amd64        Modular compiler and toolchain technologies, tools


If I look in /usr/lib/llvm-3.4/lib I find libLLVM-3.4.so, but there are also 122 .a files.  I'm not sure why that should be. 

Rik <rik5>
Group administrator
Thu 02 Mar 2017 04:57:47 PM UTC, comment #6: 

May as well pursue the llvm build problems here if you think it's worth it.

The error is strange because liboctinterp is supposedly linked against LLVM_LIBS.

Did you install llvm 3.4 as static or shared libraries? The liboctinterp.la file should list a reference to llvm in the dependency_libs variable, and liboctinterp.so should list it in its header as a NEEDED library.

Mike Miller <mtmiller>
Group Member
Thu 02 Mar 2017 04:30:08 PM UTC, comment #5: 

@Mike: I committed your patch to the stable branch here (http://hg.savannah.gnu.org/hgweb/octave/rev/c21939b04931).  I was able to install llvm3.4 and verify that without the patch I get the octave_jit_octave error, and with the patch it compiles fine.

It still doesn't work completely.  I now have a fail at the link stage, but that is a different error.


libtool: link: g++ -std=gnu++11 -fPIC -pthread -fopenmp -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual -O2 -pipe -o src/.libs/octave-cli src/src_octave_cli-main-cli.o src/src_octave_cli-octave-build-info.o  libinterp/.libs/liboctinterp.so liboctave/.libs/liboctave.so libgnu/.libs/libgnu.a -lutil -lm -fopenmp -pthread -Wl,-rpath -Wl,/home/rik/local/lib/octave/4.2.1
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyModule(llvm::Module const&, llvm::raw_ostream*)'
libinterp/.libs/liboctinterp.so: undefined reference to `llvm::verifyFunction(llvm::Function const&, llvm::raw_ostream*)'
collect2: error: ld returned 1 exit status
Makefile:14681: recipe for target 'src/octave-cli' failed
make[2]: *** [src/octave-cli] Error 1


Probably Octave is not using the correct LDFLAGS because the two mentioned functioned are available and declared in the header files.

Do we want to re-purpose this bug report to JIT not compiling on 4.2.1 or should we open a new one?

Rik <rik5>
Group administrator
Wed 01 Mar 2017 09:27:04 PM UTC, comment #4: 

Ceral - does this patch work for you?

(file #39873)

Mike Miller <mtmiller>
Group Member
Wed 01 Mar 2017 09:24:25 PM UTC, comment #3: 

I think this change

https://hg.savannah.gnu.org/hgweb/octave/diff/d0562b3159c7/libinterp/corefcn/jit-typeinfo.cc

was an overzealous search and replace. I think the parts that renamed "octave_jit_err_nan_to_logical_conversion" to "octave_jit_octave::err_nan_to_logical_conversion" should be undone. Unfortunately I don't have an ancient version of llvm to try to compile the stable branch against.

Mike Miller <mtmiller>
Group Member
Sat 25 Feb 2017 11:02:28 PM UTC, comment #2: 

If you are using a version of llvm > 3.4, then this is probably a duplicate of bug #41061 (failure to compile jit with llvm3.5).

Rik <rik5>
Group administrator
Sat 25 Feb 2017 10:47:38 PM UTC, comment #1: 

Does this work for 4.2.0 (mine doesn't)?

What version of llvm do you have installed?

Can you post more of the compilation log made with


make -j1 V=1


for extra verbosity?

Rik <rik5>
Group administrator
Sat 25 Feb 2017 01:05:52 PM UTC, original submission:  

I get an error during compilation of 4.2.1:

./configure --enable-jit
make

error: ‘octave_jit_octave’ has not been declared


Ceral Paquet <octavebugs>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #39873:  jit.diff added by mtmiller (828B - text/x-patch)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jordigh (Posted a comment)
  • -email is unavailable- added by scrutari (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by octavebugs (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2017-03-02 rik5 StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2017-03-02 rik5 Severity3 - Normal 2 - Minor
        Priority5 - Normal 3 - Low
        StatusNeed Info Confirmed
    2017-03-02 rik5 Summaryerror: ‘octave_jit_octave’ has not been declared jit no longer compiles with Octave 4.2.1
    2017-03-02 rik5 CategoryNone Configuration and Build System
        Item GroupNone Regression
    2017-03-01 mtmiller Attached File- Added jit.diff, #39873
    2017-02-25 rik5 StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-02a9.
    Corresponding source code