bugGNU Octave - Bugs: bug #40111, support runtime selection of Java...

 
 

bug #40111: support runtime selection of Java with JAVA_HOME on Linux and Unix systems

Submitted by:  Mike Miller <mtmiller>
Submitted on:  Thu 26 Sep 2013 01:07:40 PM UTC  
 
Category:  Configuration and Build System Severity:  2 - Minor
Priority:  5 - Normal Item Group:  Feature Request
Status:  Fixed Assigned to:  John W. Eaton <jwe>
Originator Name:  Open/Closed:  Closed
Release:  dev Operating System:  GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 19 Aug 2018 06:37:00 PM UTC, comment #36:

Thanks Mike for fixing the logic.

I pushed a documentation change here:
http://hg.savannah.gnu.org/hgweb/octave/rev/98242f7e8a0b

Proof-reading is very welcome.

Markus Mützel <mmuetzel>
Project Member
Wed 15 Aug 2018 09:24:37 PM UTC, comment #35:

Documentation contribution would be most welcome!

Rik <rik5>
Project Administrator
Tue 14 Aug 2018 03:57:56 PM UTC, comment #34:

No, the use of JAVA_HOME has not been documented yet. Rik's earlier suggestion was

2) Documentation should be added to section A4.3 describing how to take advantage of this new feature.

Mike Miller <mtmiller>
Project Administrator
Tue 14 Aug 2018 03:54:19 PM UTC, comment #33:

Good to know, but is it properly documented? Esp. that selecting a Java JRE tru a JAVA_HOME environment setting only works once per Octave session before any Java method has been executed?

I'm away from my box until early Sept. otherwise I could come up with a cset.

Philip Nienhuis <philipnienhuis>
Project Member
Sun 12 Aug 2018 04:53:29 PM UTC, comment #32:

Looks like it's working perfectly, thanks for testing.

Mike Miller <mtmiller>
Project Administrator
Sun 12 Aug 2018 09:13:09 AM UTC, comment #31:

An update after some more experiments:

  • AFAICS the new Java JR selection works fine. After restarting my laptop dev Octave (64b) could find the 64bit Java w/o problems on Win 7. Older 32bit Octave and 64bit Octave also still find matching Java.
  • On Windows, the path to the JRE should have forward slashes as file separators. Backslashes in the JAVA_HOME setting don't work. Theoretically this may interfere with other Windows programs that use a JAVA_HOME setting and that do expect backslashes, although I'm not aware of any such programs.
  • Selecting Java JRE only works the first time, before any Java call is invoked. I think that's something to watch out for; it may provoke bug reports from unwary users.

See here after starting Octave:

and here:

  • On 64bit Windows 10 (multiboot on the same laptop) 32-bit Octave-4.4.0 can't find 32bit Java at all. I suppose that's a Windows 10 thing. On 64b Windows 7 it all works fine.

(I didn't notice before because I never run 32bit Octave these days)

  • On wWindows 10, 64-bit dev Octave works fine incl. JAVA JRE selection (again, before any Java method invocation).
  • I cannot reproduce that pointing 64bit Octave to 32bit JRE mixes up Octave (note difference between ../Program Files (X86)/.. = 32 bit and ..Program Files/.. = 64bit JRE):

... so Octave seems to cling on to a working JRE it found before.

Philip Nienhuis <philipnienhuis>
Project Member
Sat 11 Aug 2018 06:21:01 PM UTC, comment #30:

After trying the fix I'm a bit confused.
Sorry for the long story below.

On my desktop with only 64-bit Java installation (10.0.1) it works now with a 64-bit Octave ("--enable-windows-64" configure option).

On my laptop with both a 32- and a 64-bit Java JRE it is very different:

  • It didn't work at first (with "--enable-64" configure option). An older 64bit octave binary (June 12) I built myself works fine with Java however. (That older binary is obviously from before the recent round of Java fixes).

(FYI I simply tried the "javaclasspath" command.)

  • I first uninstalled both Javas, then reinstalled 64bit Java 8 JRE update 162 (that I had lying around)

==> dev Octave runs fine with it, the older binary too. So apparently the initial 64bit Java installation was clobbered up.

  • Then I also installed the 32bit Java 8 JRE update 162 (was lying around as well)

==> dev octave doesn't run with it anymore, says "..could not find library .. blah blah ...".
==> binary from June 12 still works

So I downloaded the 32bit 4.4.0 installer from ftp.gnu.org
==> works fine with Java.

The Java call you mentioned only returns the Java version but not the Java "bit width".
So I simply renamed the Java JRE install directories by prepending them in turn with an underscore to verify that the older 32bit Octave automatically invoked the 32bit Java but couldn't work with the 64bit one, and the older 64bit Octave automatically invoked the 64bit one but couldn't work with the 32bit Java.

Experimenting with JAVA_HOME did help however. It made the dev Octave invoke 64-bit Java.
Pointing JAVA_HOME to the 32-bit Java also worked but of course I got "library open failed" (due to no matching jvm.dll).
Then, pointing JAVA_HOME back to the proper place showed that Octave got mixed up: "could not find library or dependencies".
That suggests that apparently Octave keeps leaning on the last location where it finds a (any) jvm.dll; that's fine at first sight but IMO it should also check that that jvm.dll really works.

FYI:
In the Windows registry I only see the reference to the 64-bit Java installation (..\Program Files\.. rather than ..\Program Files (X86)\..).
And system("java -version") invokes the 64-bit Java executable. Could be due to the search path, however, maybe the 32bit java is after the 64bit one.
All in all I'm a bit surprised that the older 32-bit octave does find the proper Java "bit width" installation, as it isn't referred to in the registry - AFAICS that is.

My conclusions;

  • JAVA_HOME selection works on Windows but IMO also needs to check that the jvm.dll is a proper one.
  • (that's where I get confused) the 'auto-selection' of the proper Java (32 or 64 bit) is now somehow broken, or at least fragile (it did work initially but not later on). I'm unsure if the 64bit indexing ("--enable-64" vs. '--enable-windows-64") makes a difference.

I'm too unfamiliar with the deeper Java intricacies, be it on windows or Linux, to dig further and/or help out.

Philip Nienhuis <philipnienhuis>
Project Member
Fri 10 Aug 2018 10:15:37 PM UTC, comment #29:

I pushed a change to fix the problems I am seeing with this feature on the default branch:

https://hg.savannah.gnu.org/hgweb/octave/rev/374edade7b1b

Now the following work correctly

  • if JAVA_HOME is not set, the compiled-in default is used
  • if JAVA_HOME is not set on Windows, the registry is used
  • if JAVA_HOME is set, but points to a non-existent directory, a directory that is not a Java root, or the wrong architecture (e.g. a 32-bit Java on x86_64), Octave issues an error message and Java is disabled
  • if Java wasn't found when Octave starts, the user can use setenv inside of Octave without quitting to try different directories until one is found that works

Tests on Windows and macOS are highly encouraged!

Philip - rest assured that if JAVA_HOME is not set, which is the default situation as you say on Windows, Octave will continue to look up the correct Java to use in the registry, and that logic does not depend on any heuristics, guessing, or attempting to use bin/client vs bin/server.

The only time Octave tries to use ${DIR}/bin/client vs ${DIR}/bin/server is if the user chooses to set JAVA_HOME in the system environment or in their working environment inside of Octave before loading Java.

If you want to test this on Windows, you would install another copy of Java, that is of the correct bit width, etc, in e.g. C:\javatest\some\other\java, start Octave, and test Java functions with and without JAVA_HOME set in the environment. On my Debian system I used "javaMethod('getProperty', 'java.lang.System', 'java.version')" to verify that it was loading different instances of Java with different values of JAVA_HOME.

Mike Miller <mtmiller>
Project Administrator
Wed 08 Aug 2018 09:44:44 AM UTC, comment #28:

@Mike comment #24:
if bin/client is found, it should prefer that to bin/server
At one time I had a 32-bit and a 64 bit Octave plus 32-bit and 64-bit Java installed, and AFAIR both Octaves accessed the proper (bitwidth) Java installation w/o any issue or JAVA_HOME environment settings (the latter don't exist on Windows by default).
Hopefully linking preferrably to ./bin/client doesn't break that (..)
(IIRC some bitwidth/Java discussion has happened a few years back, also related to ./client vs. ./server, there may be some clues in that discussion)

And that's not just Octave; I have 32-bit and 64-bit Java installed as I have a mixture of 32-bit and 64-bit SW that uses Java like GIS clients, LibreOffice, Freemat at one time,etc. AFAICS all that SW automagically links to the "proper" Java installation.
(BTW not that I'm a Java fan, but mentioned SW Just Works and fulfills our needs)

Philip Nienhuis <philipnienhuis>
Project Member
Tue 07 Aug 2018 08:39:15 PM UTC, comment #27:

@Mike: I was referring to Windows (without writing so - sorry): On Windows 32bit, the JVM should be installed in "C:\Program Files".
On Windows 64bit, the 64bit Version should be installed in "C:\Program Files", the 32bit version should be in "C:\Program Files (x86)".

Is there a reason to prefer the one or the other path on Linux? Is it ok to prefer the one in client?

Markus Mützel <mmuetzel>
Project Member
Tue 07 Aug 2018 08:21:12 PM UTC, comment #26:

Markus - I don't know about Windows, but at least on GNU/Linux, 32-bit x86 Java runtimes older than version 8 may contain both jre/lib/i386/client/libjvm.so and jre/lib/i386/server/libjvm.so, and in that case this selection logic prefers to use the client instance.

Mike Miller <mtmiller>
Project Administrator
Tue 07 Aug 2018 07:51:54 PM UTC, comment #25:

@Philip: That's probably point 1) in Rik's comment #20.

If I understand correctly, there should only be either "bin/client" or "bin/server" in any given JAVA_PATH. It should be up to the user to set the JAVA_PATH correctly.

Markus Mützel <mmuetzel>
Project Member
Tue 07 Aug 2018 07:42:48 PM UTC, comment #24:

Philip - yes, this is related.

Primarily, you are suffering the same bug as I am, regardless of having one or two different JVMs installed. The string "linked in or loaded libraries" means that the logic is falling through incorrectly when JAVA_HOME isn't set. That should be easy to fix.

As to using bin/client or bin/server, at the moment I believe that if bin/client is found, it should prefer that to bin/server, as the current code does it.

If you are running a 64-bit Octave and setting JAVA_HOME to C:\Program Files\java\jre<version>, then that ought to fail because you set the wrong JAVA_HOME.

Do you have any examples of a single JAVA_HOME (not two different Java installations) where both bin/client and bin/server are present, and bin/client shouldn't be preferred?

Mike Miller <mtmiller>
Project Administrator
Tue 07 Aug 2018 07:35:00 PM UTC, comment #23:

I don't know if it's related to comment #22,bt on my Win7 PC octave doesn't find Java anymore.

Then, looking at the cset http://hg.savannah.gnu.org/hgweb/octave/rev/3ddd74bd7b11 I see in L1.13:
:
+ std::string subdirs[] = {"bin/client", "bin/server"};
:

As at least on Windows it is perfectly possible to have both 32bit and 64bit Java installed, where the 32bit jvm.dll lives in
C:/Program files (X86)/java/jre<version>/bin/client
and the 64bit one in
C:/Program Files/java/jre<version>/bin/server
I wonder if the various versions won't be mixed up and Java detection would fail. Shouldn't there be a sort of preselection based on whether Octave is 32 or 64 bit?

Philip Nienhuis <philipnienhuis>
Project Member
Mon 06 Aug 2018 09:55:12 PM UTC, comment #22:

I think that the logic for JAVA_HOME being not set or empty was working correctly in Andrew's original patch. I must have messed up when I tried to streamline the patch for including the logic for Windows. Sorry.
I won't be close to a PC during the next week. But I hope the issues won't be too hard to resolve or can wait until later.

Markus Mützel <mmuetzel>
Project Member
Mon 06 Aug 2018 06:42:22 PM UTC, comment #21:

Agree on all 4 points.

I am not working on a fix for 1 and 4 at the moment, but I may take a stab at it later in the day, the logic seems pretty straightforward.

Mike Miller <mtmiller>
Project Administrator
Mon 06 Aug 2018 06:30:01 PM UTC, comment #20:

Marking as "In Progress".

1) The issue of this patch not working when JAVA_HOME is undefined needs to be resolved.

2) Documentation should be added to section A4.3 describing how to take advantage of this new feature.

3) If it is not too cumbersome, it would be nice to support Java8 and older on non-x86 machines. However, I don't use such systems so this might be left as an exercise for someone who is affected and motivated enough to provide a patch.

4) My preference is that selection should fail when JAVA_HOME points to a non-existent directory.

Rik <rik5>
Project Administrator
Mon 06 Aug 2018 06:18:53 PM UTC, comment #19:

Regarding comment #18, I believe Octave should honor the JAVA_HOME override. If it points to a non-existent directory then the responsibility lies on the programmer to fix it, not Octave to try and figure out a suitable workaround such as the compiled-in path in JAVA_LDPATH.

Rik <rik5>
Project Administrator
Mon 06 Aug 2018 06:02:22 PM UTC, comment #18:

Also if I set JAVA_HOME=/tmp, the code uses the compiled-in JAVA_LDPATH. What should be the behavior if JAVA_HOME is set, but is set to a nonexistent directory or to a directory that is not actually a Java directory?

Mike Miller <mtmiller>
Project Administrator
Mon 06 Aug 2018 05:59:42 PM UTC, comment #17:

Aside from my previous comments, this change does not work for me when JAVA_HOME is not defined in the environment (the fall-back case).

The reason is that the string variable jvm_lib_path is initialized to "linked in or loaded libraries". The test for whether the string is empty always fails so the compiled-in JAVA_LDPATH is never used.

Mike Miller <mtmiller>
Project Administrator
Mon 06 Aug 2018 05:25:27 PM UTC, comment #16:

Here is an m-file function that should return the proper architecture name to use in Java file paths of the form "${JAVA_HOME}/jre/lib/${ARCH}/server/libjvm.so". If someone finds this useful and worth preserving, it can be ported to C++ if needed.

(file #44733)

Mike Miller <mtmiller>
Project Administrator
Mon 06 Aug 2018 04:09:15 PM UTC, comment #15:

This change only works on x86 platforms or platforms with Java 9 or later.

I'm pretty confident that many systems now do have Java 9 or later.

Are we ok with intentionally excluding systems such as arm or powerpc that have Java 8 or older? That was the purpose of the 'arch_name' variable in my pseudocode suggestion in comment #10.

Mike Miller <mtmiller>
Project Administrator
Mon 06 Aug 2018 03:33:21 AM UTC, comment #14:

Probably in Section A.4.3 which is about passing parameters to the JVM. This section could be generalized to document how to pick a JVM and then how to pass parameters to it.

Rik <rik5>
Project Administrator
Sun 05 Aug 2018 10:45:09 PM UTC, comment #13:

Maybe we should also document that JAVA_HOME can be used to select a JVM at load time.
Where is the correct place for this?

Markus Mützel <mmuetzel>
Project Member
Sun 05 Aug 2018 10:38:41 PM UTC, comment #12:

Andrew: I modified the step-02 of your changesets in bug #54219 to also work on Windows and pushed it to default here:
http://hg.savannah.gnu.org/hgweb/octave/rev/3ddd74bd7b11

With this we have a common way to select between multiple installed JVM on all supported platforms. Thanks.

I'll leave the other two MacOS specific parts in that bug report to someone who has a background in Mac development.

Markus Mützel <mmuetzel>
Project Member
Mon 02 Jul 2018 12:22:30 AM UTC, comment #11:

HI folks. Would you mind looking at https://savannah.gnu.org/bugs/index.php?54219 - it adds dynamic JVM detection for macOS and Linux (despite the issue name just saying macOS). It folds in some of the stuff discussed here, adding $JAVA_HOME support.

Andrew Janke <apjanke>
Thu 29 Mar 2018 09:08:40 PM UTC, comment #10:

I think it might be nice / easier to maintain if most of the filesystem logic were written in a .m file function. ov-java.cc could export a simple DEFUN that would accept the full file name of libjvm.so or jvm.dll to use.

See also bug #53389 for the same feature on Windows, would override the Windows registry setting.

The logic for finding the JRE should be something like

Mike Miller <mtmiller>
Project Administrator
Tue 09 Feb 2016 09:18:38 PM UTC, comment #9:

Developers will still need a Java build system including a javac compiler and access to header files like jvm.h. The build system creates a Java library octave.jar which is distributed.

But at runtime the user should only need a JVM and the octave.jar file.

Rik <rik5>
Project Administrator
Tue 09 Feb 2016 07:58:49 PM UTC, comment #8:

I plan to look at this problem soon.

I'm sure I don't understand all the details, but my intent is to fix Octave so that finding the JVM library is done entirely at runtime. I think the only thing we need at build time is a valid jvm.h file.

John W. Eaton <jwe>
Project AdministratorIn charge of this item.
Wed 04 Feb 2015 06:23:45 AM UTC, comment #7:

What about revising Pascal's patch in comment #4 to use the JAVA_HOME varibale?

Rik <rik5>
Project Administrator
Wed 04 Feb 2015 03:08:53 AM UTC, comment #6:

Hi Pascal, that is certainly one workaround for this problem, but ultimately I think we'd want to dynamically find the libjvm.so, and preferably using the semi-standardized JAVA_HOME environment variable.

On Debian we build with the path /usr/lib/jvm/default-java, for example

This makes it work with whatever the distro selects as the default Java, or in your case, the current version of Java 8, but it doesn't let a user pick a different Java runtime, for whatever reason they may need to do so.

Mike Miller <mtmiller>
Project Administrator
Tue 03 Feb 2015 05:22:27 PM UTC, comment #5:

I compiled octave-3.8.2 on Fedora 21 x86_64. I think I solved the Java issue by not using the full path name of libjvm.so, but a link which will stay constant but be upgraded to the right location with java update.

The technique is to use (under Fedora):

./configure --with-java-includedir=/usr/lib/jvm/java-1.8.0-openjdk/include --with-java-libdir=/usr/lib/jvm/java-1.8.0-openjdk/jre/lib/amd64/server (other options ...)

Regards

Pascal

Pascal Dupuis <cdemills>
Project Member
Wed 17 Dec 2014 04:45:04 PM UTC, comment #4:

This bug is still present in Fedora 21: after an upgrade of JDK, the java lib cant't be found anymore, up to the next recompilation.

I enclose a small patch adding a fallback mechanism. If loading the jvm lib from the static path fails, try a dynamic path built from the environment variable JAVA_JVM_LIB.

Regards

Pascal

(file #32686)

Pascal Dupuis <cdemills>
Project Member
Mon 30 Sep 2013 06:01:24 PM UTC, comment #3:

The examples are meant to illustrate the logic for finding the jvm, not for copying into configure.ac.

Whether configure.ac is fine as it is?
Yes I'd agree with you, except maybe for Mac OSX; the examples contain some sections devoted to OSX as well.

Philip Nienhuis <philipnienhuis>
Project Member
Mon 30 Sep 2013 12:14:28 AM UTC, comment #2:

Your examples are all for configure scripting. I think Java detection in the configure script is fine as it is. This bug is about overriding the Java path at runtime by setting JAVA_HOME in the shell environment or in octaverc.

Mike Miller <mtmiller>
Project Administrator
Fri 27 Sep 2013 02:06:13 PM UTC, comment #1:

For the automatic way, here are some links (from this thread:
http://lists.debian.org/debian-java/2009/04/msg00005.html):

http://gitweb.scilab.org/?p=scilab;a=blob;f=scilab/m4/java.m4;h=ace058ee88c6c9a876e22162bf4e62f5a6c2107c;hb=HEAD

http://gitweb.scilab.org/?p=scilab;a=blob;f=scilab/bin/scilab;h=65fd00adf4e1e38eeb2359f97e10bdb0a5c5ad41;hb=HEAD

(From what I can see in Octave's configure.ac there's already quite a bit of these implemented.)

I haven't looked whether these configure macros also detect the JVM on 64 bit systemes (where they reside in
$JAVA_HOME/jre/lib/<arch>/server
...rather than
$JAVA_HOME/jre/lib/<arch>/client)

(and my 'configure proficiency' is lacking anyway)

Philip Nienhuis <philipnienhuis>
Project Member
Thu 26 Sep 2013 01:07:40 PM UTC, original submission:

We currently hardcode the path to the Java runtime (specifically the libjvm.so library) in the Octave interpreter at compile-time. The library is not actually used at compile-time, since it is dlopen'd only when Octave needs it. It would be better if there was an ability for the user to override this path in the Octave interpreter at runtime.

The easiest and most obvious solution would be to keep the hardcoded path as it is now and allow the user's JAVA_HOME environment variable to override it.

More ambitious would be to do some kind of automatic determination of the Java path at runtime, but that's already hard enough as it is in the configure script.

Mike Miller <mtmiller>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #44733:  javaarchname.m added by mtmiller (1KiB - text/x-objcsrc)
file #32686:  ov-java.pat added by cdemills (993B - application/octet-stream)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by apjanke (Posted a comment)
  • -email is unavailable- added by apjanke (Adding myself to the Cc list)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by cdemills (Updated the item)
  • -email is unavailable- added by mtmiller (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-08-14 rik5 StatusReady For Test => Fixed
        Open/ClosedOpen => Closed
    2018-08-10 mtmiller StatusIn Progress => Ready For Test
    2018-08-06 rik5 StatusReady For Test => In Progress
    2018-08-06 mtmiller Attached File- => Added javaarchname.m, #44733
    2018-08-05 mmuetzel Dependencies- => bugs #53389 is dependent
    2018-08-05 mmuetzel StatusNone => Ready For Test
    2018-07-01 apjanke Carbon-Copy- => Added apjanke
    2018-03-29 mtmiller SummarySupport runtime selection of Java version on Linux and Unix systems => support runtime selection of Java with JAVA_HOME on Linux and Unix systems
    2016-02-18 jwe Assigned toNone => jwe
    2014-12-17 cdemills Attached File- => Added ov-java.pat, #32686
    2014-06-11 mtmiller Dependencies- => bugs #42540 is dependent

    Back to the top


    Powered by Savane 3.3