bugGNU Octave - Bugs: bug #53520, Windows builds made with Java 8...

 
 

bug #53520: Windows builds made with Java 8 JDK on Linux cannot find Java 9 JRE on Windows 10

Submitter:  Philip Nienhuis <philipnienhuis>
Submitted:  Fri 30 Mar 2018 07:49:54 AM UTC
   
 
Category:  Libraries Severity:  4 - Important
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Fixed Assigned to:  None
Originator Name:  Philip Nienhuis Open/Closed:  * Closed
Release:  * dev Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 11 Apr 2018 05:12:27 PM UTC, comment #37: 

@Philip: Did your test for bug #53572 also test this bug?  Can this report be closed?

Rik <rik5>
Group administrator
Fri 06 Apr 2018 07:11:28 PM UTC, comment #36: 

I pushed the patch here: http://hg.savannah.gnu.org/hgweb/octave/rev/d61f42a392b7

Marking as ready for test

Markus Mützel <mmuetzel>
Group administrator
Fri 06 Apr 2018 02:34:28 PM UTC, comment #35: 

@Markus: Well any straw to hang is good ATM :-)
But yes I remember that I saw a VC++ runtime being installed when installing Java 9. I'll give that a try.

It could also be Windows 10 itself; that has all kinds of obscure policies that are changed behind our backs with any major feature upgrade. Then again on my desktop's Win10 Java 8 works, while on my laptop's Win10 it doesn't work now but used to work some time ago. 
All of this is one reason I rarely use Win10. Currently I only have it around as reference OS to test Octave on.

Back to topic:
Agreed that the patch should be applied.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 06 Apr 2018 02:15:06 PM UTC, comment #34: 

@Philip: That might be a long shot but Java 8 was built against Visual C++ 2008 while Java 9 and 10 are built against Visual C++ 2013. Maybe it might help un-installing and re-installing the Visual C++ 2013 redistributables for example from here: https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads

Markus Mützel <mmuetzel>
Group administrator
Fri 06 Apr 2018 12:48:34 PM UTC, comment #33: 

In my opinion the patch from comment #2 should be applied before the 4.4 release. Without it, Java >=9 doesn't work on Windows at all. With it applied, at least some systems seem to behave.
I just tried on my work PC (also a Windows 10 64bit): Octave finds the installed Java 9 and can used it (with that patch).

After applying that patch, we can leave this report open to investigate why Java still cannot be used on other systems.

Markus Mützel <mmuetzel>
Group administrator
Fri 06 Apr 2018 08:45:02 AM UTC, comment #32: 

(Thx Rik, on my other build box mxe-octave was fully up-to-date but I have the same issue there as regards Java.)

I tried with Java 10: same issue
Logged in as Administrator; same issues.
Checked permissions of the entire Java setup: no clues there.
On my laptop, under Windows 10, Java doesn't cooperate with Octave at all, be it Java 8, 9 or 10.

However;
On my Windows 7 box Java 10 seems to work, it can run javaObject both with Markus' build and my own build; spreadsheet I/O doesn't fully work but that is due to bug #352510 that I'll pick up now.

On Windows 7 the NSIS installer still complains when installing Octave with the patch from comment 32; so mxe-octave needs another patch for its installer part. I'll open a separate bug report for that.
Otherwise Octave can work fine with Java > 8 on Windows 7.

Considering that

  • it works for Markus on Windows 10,
  • it work for me on Windows 7,
  • my (Linux) development setup seems fine

I tend to suggest leaving this bug for the moment and set status to "postponed".

OK?

(Adapting title to reflect that it refers to Windows 10)

Philip Nienhuis <philipnienhuis>
Group Member
Thu 05 Apr 2018 10:09:34 PM UTC, comment #31: 

The vbs scripts have been updated.  I think you need to do an 'hg pull' in your MXE directory.  See this changeset


changeset:   4644:c3950e2b4066
user:        Rik <rik@octave.org>
date:        Tue Mar 27 09:53:22 2018 -0700
summary:     Fix vbs scripts to launch with '--gui' option (bug #53369).



Rik <rik5>
Group administrator
Thu 05 Apr 2018 08:36:06 PM UTC, comment #30: 

I just saw that it isn't just Octave; the NSIS installer doesn't detect Java 9 either (and Octave proper isn't anywhere unpacked yet at that moment).

Philip Nienhuis <philipnienhuis>
Group Member
Thu 05 Apr 2018 08:30:09 PM UTC, comment #29: 

Yes, thanks, got it, d/led & unzipped it, & ran it.
Result: on my system it doesn't detect Java 9 :-(
GUI or CLI made no difference.

So now the questions are:
1) What is so special about my Win10 system?
2) Are there any chances that other Windows users will run into the same or similar problem?

Disabling the fireall (ZoneAlarm, which also blocks access to / invoking unknown / not-yet-whitelisted applications) made no difference.

Anyway I'll try with Java 10 (didn't know that was already out). but I have little faith that it'll work better.

OTOH: good to know that here's nothing wrong (as far as this bug matters) with my build system :-)
BTW AFAIK octave had better be started with octave.vbs rather than octave.bat. I just noted that on my mxe builds the --gui flag hasn't been added yet to octave.vbs; I have a permanent desktop icon pointing to octave.vbs in dist/octave/ on my linux home drive and IIRC I added --gui to that by hand.


Philip Nienhuis <philipnienhuis>
Group Member
Thu 05 Apr 2018 07:52:15 PM UTC, comment #28: 

@Philip:Did you receive the email with the link to the zip? I downloaded and installed your version. Weird thing is that it crashes when I try to start it with octave.bat and doesn't seem to do anything with the .vbs.
However, when I start "octave-cli.exe", Java 10 loads correctly:

octave:1> version -java
ans = Java 10+46 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
octave:2> javaObject ("java.lang.Double", 9.0)
ans =

<Java object: java.lang.Double>


Still unsure whether my Windows or Java is somehow special. At least we can probably tell that it doesn't depend on the build environment.

Markus Mützel <mmuetzel>
Group administrator
Thu 05 Apr 2018 05:58:12 PM UTC, comment #27: 

@Markus:
wetransfer is used all the time at my work and in the branch I work for. I'd be happy to try your binary. Note that I use 64bit Octave (--enable-windows-64) and also 64bit Java.
My SF email address to send it to must be in many files in the io and mapping package or probably even in commits in the core repo.

In return I've uploaded a Windows binary (built yesterday) incl. your patch to my Stack account, I hope you and/or someone else running Windows w. Java 9 would like to try it just to be sure that this whole bug isn't caused by local things in my (Linux) development boxes (can never be ruled out).
See here:
https://prn183.stackstorage.com/s/ReMaTomrHKjyMef
(valid until next Sunday April 8)

Philip Nienhuis <philipnienhuis>
Group Member
Thu 05 Apr 2018 05:07:38 PM UTC, comment #26: 

I tested with Java 9 and 10 on Windows 10 64bit 1709. Java in Octave is working with both versions for me (and also with Java 8).

@Philip: Do you prefer any provider who accepts sending the ~450MB .zip-file? With a quick Google search, I found https://wetransfer.com/ that allows sending files up to 2 GB to an email address(?). Would that be ok for you? Which email address could I use?

Markus Mützel <mmuetzel>
Group administrator
Thu 05 Apr 2018 05:02:36 PM UTC, comment #25: 

@ comment #24 and comment #19 (sharing binary): that's what occurred to me as well.

@Markus can you upload the windows binary installer (or zip) somewhere? I'd like to test it; if it works we have some more clues.
OTOH I can upload my most recent Windows binary as well.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 05 Apr 2018 04:06:08 PM UTC, comment #24: 

Is anyone else testing with Java 9 or 10 on Windows? Does it work for them? Is Philip the only one seeing this error?

If someone has this working with Java 9, can someone share their mxe-octave build with Philip to see if there is something about the cross-build environment? I really doubt that the host version of Java matters when cross-building with mxe-octave. The JNI header file is pulled from the OpenJDK web site during the cross-build.

Mike Miller <mtmiller>
Group Member
Thu 05 Apr 2018 04:01:12 PM UTC, comment #23: 

Yes, the absolute path is shown in the error message that Philip posted, repeated here:


>> javaObject ("java.lang.Double", 9.0)
error: javaObject: could not find library or dependencies: C:\Program Files\Java\jre-9.0.4\bin\server\jvm.dll


The path does include a space, but there shouldn't be any shell escaping issues here because the string is passed directly to LoadLibrary. And because it does work with Java 8 for Philip.

I think the oct_shlib code already does some magic with directories when loading DLLs on Windows:


    std::string dir = sys::file_ops::dirname (f);

    set_dll_directory (dir);

    handle = LoadLibrary (file.c_str ());

    set_dll_directory ();


Mike Miller <mtmiller>
Group Member
Thu 05 Apr 2018 03:48:11 PM UTC, comment #22: 

Do we pass the absolute path to the DLL in LoadLibrary?  If not, maybe we need to be parsing the path to jvm.dll and adding the base directory the the equivalent of LD_LIBRARY_PATH.

What if jvm.dll is calling out for other dlls or supported files?  Again, maybe adding the base directory to the LD_LIBRARY_PATH would help with that.

Rik <rik5>
Group administrator
Thu 05 Apr 2018 03:35:12 PM UTC, comment #21: 

Also the error at the point that Philip hit indicates that Octave is simply calling 'LoadLibrary' and that call is returning a null pointer. That means it has nothing to do with octave.jar, or even if any of the expected functions are in the jvm.dll, simply that the dll can't be loaded into the executable image. The error message indicates that the Windows API error code set by LoadLibrary was ERROR_DLL_NOT_FOUND.

Mike Miller <mtmiller>
Group Member
Thu 05 Apr 2018 03:29:32 PM UTC, comment #20: 

It shouldn't be relevant.  octave.jar is built in to the tarball, but we specify using only java instructions available from 1.6 on.


"$(JAVAC)" -source 1.6 -target 1.6


See scripts/java/module.mk.

Rik <rik5>
Group administrator
Thu 05 Apr 2018 01:52:42 PM UTC, comment #19: 

For my tests I cross-built Octave on an Ubuntu Linux (Bionic Beaver) with Java 9. Maybe that makes the difference...
Does MXE Octave build octave.jar? Which Java does it use? Can this be relevant at all?

Markus Mützel <mmuetzel>
Group administrator
Thu 05 Apr 2018 12:09:22 PM UTC, comment #18: 

Mike,
(Sorry I'm a bit terse, busy time at work and privately - all nice things but there's not so much time for Octave these weeks)

I tested with JAVA_VERSION not set (comment #3) and set (comment #11).
With or w/o JAVA_VERSION set, I get the very same error message about Octave not being able to invoke the jvm.dll library or dependencies; so you and I come to the same conclusion.
The patch does what it was designed for, sure, but it looks like that design aim isn't sufficient.

And yes, (again:) jvm.dll is at exactly the position indicated in the registry.
Maybe it needs helper executables / libraries / functions that should be Java-9 based or -compatible.

What I can try is to search for an older or newer Java 9/10?/.... JRE (for Windows) than 9.0.4 and then see if it still doesn't work.

What I'm not going to do is to update Java to 9 on my Linux box. While that may help (theoretically) to have Java-9 based code to invoke the JVM (hopefully backwards compatible) I'm afraid that if I would, it would lead to dependency issues within the rpm installed packages database of my Linux installation. I hit that before and ended up reinstalling Linux, so I'd rather stay with the official distro-supplied packages and tools.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 10:53:30 PM UTC, comment #17: 

Thanks, I missed the screenshots.

What I can't tell from your output is if you tested with this patch with JAVA_VERSION not set and with Java 9 installed?

I see that you tested with JAVA_VERSION set to "9.0.4" with Java 8 installed, which correctly failed, and you tested with JAVA_VERSION set to "9.0.4" with Java 9 installed.

If you test with Java 9 installed with no JAVA_VERSION set, and you get the same error message

error: javaObject: could not find library or dependencies: C:\Program Files\Java\jre-9.0.4\bin\server\jvm.dll

with or without JAVA_VERSION set, then that at least means this patch is working correctly for what it was intended to do.

What may not be working is that once Octave finds the path to the jvm.dll, it can't load it.

Does that file actually exist at that location on your system when you install Java 9?

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 10:30:13 PM UTC, comment #16: 

@Mike:

>> winqueryreg ("HKLM", 'SOFTWARE\JavaSoft\JRE', 'CurrentVersion')
ans = 9.0.4


and the rest of the answer is in the pictures in comment #11.
Indeed the key names are as stated in comment #4, for older and newer Java.

The jvm.dll is in the location indicated in the registry, so the code found the correct key value:

>> winqueryreg ("HKLM", 'SOFTWARE\JavaSoft\JRE\9.0.4', 'runTimelib')
ans = C:\Program Files\Java\jre-9.0.4\bin\server\jvm.dll


Note: I tried with --enable-64 and --enable-windows-64 mxe-octave configure options.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 09:38:24 PM UTC, comment #15: 

I think the output in comment #11 may mean that it did find the correct registry key using 'CurrentVersion', when JAVA_VERSION was not set, but that the jvm.dll shared library does not exist at the location that the registry is pointing to.

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 09:34:01 PM UTC, comment #14: 

If I'm reading the patch correctly, it should be checking for both '\software\javasoft\jre' and '\software\javasoft\java runtime environment\', regardless of what the version number is. In my experience that is the more robust approach, check all possible paths with all possible version numbers, rather than trying to encode information about which path goes with which version.

The error message is indicating that both checks failed, first it tried the path containing '\jre\', then it tried '\java runtime environment\', and both did not exist.

Philip - you never answered comment #4 as far as I can see, what is the proper registry key path on your system with Java 9 installed?

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 09:23:32 PM UTC, comment #13: 

Thinking about the uploaded pics, it's intruiging that apparently the JAVA_VERSION environment setting only changes the version but not the rest of the registry key name:

in comment #12 in the verbatim block, the registry key name of the Java 8 setting is shown (...\java runtime environment\...) so somehow the code picks that up, instead of ...\jre\... for Java 9.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 07:48:46 PM UTC, comment #12: 

After reinstalling Java 8, I get this in Octave;

>> __have_feature__ ("JAVA")
ans = 1
>> javaObject ("java.lang.Double", 9.0)
error: unable to find Java Runtime Environment: software\javasoft\java runtime environment\9.0.4::RuntimeLib
>> getenv ("JAVA_VERSION")
ans = 9.0.4
>> setenv ("JAVA_VERSION", "")
>> getenv ("JAVA_VERSION")
ans =
>> javaObject ("java.lang.Double", 9.0)
ans =

<Java object: java.lang.Double>

>>


so the JAVA_VERSION environment setting does do something.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 07:41:40 PM UTC, comment #11: 

Setting JAVA_VERSION doesn't help either.
On my Windows 10 box:

>> system ("java -version")
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
ans = 0
>> chk_spreadsheet_support ("", 3)

(OCT interface... OK, included in io package)

Checking Excel/ActiveX/COM... OK.

1. Checking Octave's Java support... OK.
2. Checking Java dependencies...
  Checking Java JRE presence.... OK, found one.
No Java support found.
No Java support found.
ans =  1
>> getenv ("JAVA_VERSION")
ans = 9.0.4
>> javaObject ("java.lang.Double", 9.0)
error: javaObject: could not find library or dependencies: C:\Program Files\Java\jre-9.0.4\bin\server\jvm.dll


(NB: step #1 in chk_spreadsheet_support.m's output is checking if Octave was compiled with Java support, i.e., if '__have_feature__ ("JAVA")' returns 1.)

Attached are two screen shots of the registry key values on my Win10 box; AfAIK they are identical to what they would be on Win7, if not I'll report back later.

BTW: As to the bug report title, I'm unsure if a mxe-octave crossbuilt Octave built on Linux with Java 9 would find any Java on a Windows box. Currently the title just mentions my situation.



Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 05:59:52 PM UTC, comment #10: 

@Mike: That's fine, we can just use JAVA_VERSION for the release.  But something needs to be done to partially resolve this ahead of 4.4.

Rik <rik5>
Group administrator
Wed 04 Apr 2018 05:56:17 PM UTC, comment #9: 

Rik - personally I think the JAVA_HOME solution will be a lot more work and there is not enough time before this release. JAVA_VERSION has been in place in the Windows Java setup code since Octave 3.8 when the Java code was first merged into Octave, so it's nothing new.

If someone is going to pursue the JAVA_HOME solution on Windows, please take a look at my notes and proof-of-concept on bug #40111, and discuss on bug #53389. It should be possible to write some platform-independent m-file code that explores the directory structure under JAVA_HOME to find the shared library that Octave loads.

But I do agree we should resolve this bug for 4.4, by looking for different registry key layouts for different possible Java versions.

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 05:39:59 PM UTC, comment #8: 

I'll try tonight or tomorrow night.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 04 Apr 2018 04:34:54 PM UTC, comment #7: 

Adding bug #53389 as a dependency.  Maybe instead of JAVA_VERSION we should be using the JAVA_HOME variable and simply go wherever that points?

Rik <rik5>
Group administrator
Wed 04 Apr 2018 04:24:04 PM UTC, comment #6: 

Updating the Severity level to Important.  We should get this fixed before the 4.4 release.

@Philip: Can you see about Markus's questions in comment #4?  The answers will help determine how to modify this patch.

Rik <rik5>
Group administrator
Mon 02 Apr 2018 04:36:55 PM UTC, comment #5: 

I think a patch like the one Markus proposed is the way to go, if it can be made to work on enough platforms including Philip's.

Rik <rik5>
Group administrator
Mon 02 Apr 2018 04:18:48 PM UTC, comment #4: 

@Philip: Does setting JAVA_VERSION make any difference?
Could you please check with the Registry Editor what the respective key names are? For me they are:

For Java 5 and 8:
HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment

For Java 9 and 10:
HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\JRE

What does the following command return for you?

winqueryreg ("HKLM", 'SOFTWARE\JavaSoft\JRE', "CurrentVersion")


Markus Mützel <mmuetzel>
Group administrator
Sun 01 Apr 2018 05:01:17 PM UTC, comment #3: 

@Markus:
Unfortunately the patch from comment #2 doesn't help with Java-9.0.4 64bit.
Reinstalling Java 8 (64bit) makes javaclasspath work again.

My system: Win7 64 bit, my Octave build is with 64-bit indexing.

On Windows 10 that same Octave build can find neither Java version, but that may be another issue.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 30 Mar 2018 03:25:39 PM UTC, comment #2: 

The attached patch changes the logic that searches the Java Runtime Library on Windows to prefer the new registry key (>= 9) over the old one.
You can select the JRE version with the environment variable JAVA_VERSION. I.e. the following manually selects the JRE 1.8 when both JRE 1.8 and JRE 9 are installed:

set JAVA_VERSION=1.8
octave


To revert to JRE 9 in the same command prompt, either set the environment variable to empty before starting Octave again:

SET JAVA_VERSION=


or explicitly select the installed version. For me that is:

set JAVA_VERSION=9.0.4



(file #43727)

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Mar 2018 12:51:59 PM UTC, comment #1: 

It looks like the registry key name has changed in JRE 9. [1]

Should we first check the new one and only use the old one in case we weren't successful there?

[1] https://docs.oracle.com/javase/9/install/installation-jdk-and-jre-microsoft-windows-platforms.htm#JSJIG-GUID-47C269A3-5220-412F-9E31-4B8C37A82BFB

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Mar 2018 07:49:54 AM UTC, original submission:  

While investigating bug #53510 I found that after removing the Java 8 JRE and installing a Java 9 JRE, Octave cannot find the installed Java anymore. Reinstalling Java 8 makes Octave find Java again.
The Octave version I run on Windows was cross-built with a Java 8 JDK on Mageia-6 Linux.

Most easily reproduced / detected by simply running the commands:

>> system ("java -version")
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
ans = 0

>> javaclasspath
error: unable to find Java Runtime Environment: software\javasoft\java runtime environment::CurrentVersion
error: called from
    javaclasspath at line 64 column 16


but trying any javaObject or javaMethod call will show this as well.

Apparently the way the Java JRE is detected on Windows has changed between Java 8 and Java 9.

This may become a problem for Octave Windows cross-built on Linux with e.g., a Java 8 JDK and installed on Windows with a Java 9 JRE, or vice versa.

Philip Nienhuis <philipnienhuis>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #43727:  bug53520_java9.patch added by mmuetzel (3KiB - application/octet-stream)

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by philipnienhuis (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 11 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-04-16 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2018-04-06 mmuetzel StatusPatch Submitted Ready For Test
    2018-04-06 philipnienhuis SummaryWindows builds made with Java 8 JDK on Linux cannot find Java 9 JRE on Windows Windows builds made with Java 8 JDK on Linux cannot find Java 9 JRE on Windows 10
    2018-04-04 philipnienhuis Attached File- Added Win10-regkey_HKLM_javasoft_JRE_904.PNG, #43798
        Attached File- Added Win10-regkey_HKLM_javasoft_JRE_904_MSI.PNG, #43799
    2018-04-04 rik5 Dependencies- Depends on bugs #53389
    2018-04-04 rik5 Severity3 - Normal 4 - Important
    2018-04-02 rik5 StatusNone Patch Submitted
    2018-03-30 mmuetzel Attached File- Added bug53520_java9.patch, #43727
    2018-03-30 philipnienhuis Dependencies- bugs #53510 is dependent

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code