bugGNU Octave - Bugs: bug #58004, Java xerces library makes Octave...

 
 

bug #58004: Java xerces library makes Octave crash

Submitter:  Philip Nienhuis <philipnienhuis>
Submitted:  Tue 17 Mar 2020 02:13:51 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Confirmed Assigned to:  None
Originator Name:  Philip Nienhuis Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 26 Oct 2020 07:07:05 AM UTC, comment #41: 

Unless there is someone really working on the Java interface, this bug will probably not be fixed.  Calling Java functions from a particular xerces library is not a main feature of Octave and I suggest to postpone it for Octave 7.

Marking "dev".

Kai Torben Ohlhus <siko1056>
Group Member
Thu 01 Oct 2020 11:03:01 AM UTC, comment #40: 

Thank you Kai.
To be precise, I wasn't the first observer (TTBOMK that was Olaf, in the package release tracker) but rather the one who formally reported this bug.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 01 Oct 2020 07:28:44 AM UTC, comment #39: 

On openSUSE 15.2 and Octave 6 the code of comment #30


__octave_config_info__ ("hg_id")
ans = 697562e2b09a

usejava ("jvm")
javaaddpath ("/usr/share/java/xerces-j2.jar")
figure ();
line ();
print ('line.svg');

for i = 1:100000
  parser = javaObject ("org.apache.xerces.parsers.DOMParser");
endfor


makes Octave crash reliably, when I run the whole code a second time.

As this bug is independent of the IO package. I re-title this item to relieve Philip, who was unfortunately only the first observer of this deeper rooted problem.

Kai Torben Ohlhus <siko1056>
Group Member
Fri 04 Sep 2020 09:00:42 PM UTC, comment #38: 

(just revisiting this bug)

AFAICS it's only a some-flavors-of-linux problem.

On Windows I haven't ever seen this issue, nor on my Mageia-7 linux systems.

But it is hampering reintroducing the xmlread/xmlwrite functions in the io package, as the octave-forge admin's system is affected.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 22 Jul 2020 12:01:21 AM UTC, comment #37: 

have there been any issues on windows, or is it just a (some flavors of) linux problem?

Nicholas Jankowski <nrjank>
Group Member
Sat 13 Jun 2020 09:54:16 PM UTC, comment #36: 

I can crash octave-cli with fltk on CentOS.

So, on CentOS 8.1 it crashes in all configurations cli/gui fltk/qt.

On Fedora 32 I cannot crash gui or cli w/ fltk.
(But perhaps I shall try harder.)

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 13 Jun 2020 09:43:56 PM UTC, comment #35: 

I cannot crash octave-gui on Fedora (with neither fltk nor qt).
octave-gui crashes on CentOS with either fltk or qt.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 13 Jun 2020 09:15:40 PM UTC, comment #34: 

Same question for fltk or qt toolkit in CLI or GUI?
(sorry for asking, I can't reproduce the segfault at all, and I was wondering why the fltk toolkit is required in the xmlread demo in the first place)

Would you agree that with respect to the xmlread code proper, it's clear enough by now that it's not to blame?
I suppose it may be wise to come up with some other another xmlread demo, or disable the demo for now. But I think I can put xmlread.m back in the io package.

Philip Nienhuis <philipnienhuis>
Group Member
Sat 13 Jun 2020 05:06:09 PM UTC, comment #33: 

Same here (no crash with cli and fltk).

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 13 Jun 2020 03:08:47 PM UTC, comment #32: 

For me, using the same build of Octave, it doesn't appear to be crashing with octave-cli and using the fltk toolkit.

John W. Eaton <jwe>
Group administrator
Sat 13 Jun 2020 02:20:28 PM UTC, comment #31: 

Does running from the CLI or the GUI make a difference?

Philip Nienhuis <philipnienhuis>
Group Member
Sat 13 Jun 2020 11:27:08 AM UTC, comment #30: 

You could try "make install V=1" and see whether strip is ever called or if install is given the -s option, but Octave's Makefile shouldn't be set up to strip by default.

The behavior I observe varies.  I've been able to capture stack traces in which the call stack of the Octave interpreter thread appears to be completely invalid.  I'm attaching an example.  The first file is the output of "thread apply all bt" after starting Octave and executing the commands


usejava ("jvm");
javaaddpath ("/usr/share/java/xercesImpl.jar")
figure ();
line ();
print ('line.svg');


The second is after a segfault happens when running


for i = 1:100000
  parser = javaObject ("org.apache.xerces.parsers.DOMParser");
end


Comparing the stack traces for threads 1-7, they all look about the same except for thread 6, which gdb can no longer make any sense of.  That's the one that was running the Octave interpreter.  Maybe I'm chasing the wrong thing here, but how can we find what is executing when that happens?

(file #49249, file #49250)

John W. Eaton <jwe>
Group administrator
Sat 13 Jun 2020 09:38:01 AM UTC, comment #29: 

Hmm... It looks like I get more verbose bt by running octave in build tree. Does "make install" strip some symbols?

Anyway, here is full bt from 10f4ac250b89 (stable) on CentOS 8.
(attached bt_10f4ac250b89.txt.gz)

Dmitri.
--


(file #49248)

Dmitri A. Sergatskov <dasergatskov>
Sat 13 Jun 2020 09:15:55 AM UTC, comment #28: 

I alse reproduced it on current dev on Fedora. bt looks more interesting:


(gdb) thread apply all bt

Thread 30 (Thread 0x7ff6054ce700 (LWP 3582895)):
#0  0x00007ff622042e92 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007ff5efb1c53b in os::PlatformEvent::park() () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#2  0x00007ff5efad1c15 in Monitor::ILock(Thread*) () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#3  0x00007ff5efad23ea in Monitor::lock_without_safepoint_check() () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#4  0x00007ff5efbb0af9 in SafepointSynchronize::block(JavaThread*) [clone .part.0] () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#5  0x00007ff5efcc88f8 in JavaThread::check_safepoint_and_suspend_for_native_trans(JavaThread*) () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#6  0x00007ff5ef8a72cd in ThreadStateTransition::transition_from_native(JavaThread*, JavaThreadState) [clone .constprop.0] () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#7  0x00007ff5ef8b7972 in jni_FindClass () at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/jre/lib/amd64/server/libjvm.so
#8  0x00007ff627181439 in JNIEnv_::FindClass(char const*) (this=0x7ff5f446fa60, name=0x7ff6278239ef "java/lang/Object") at /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.252.b09-1.fc32.x86_64/include/jni.h:794
#9  0x00007ff62717bcf1 in unbox(JNIEnv*, octave_value_list const&, jobjectArray_ref&, jobjectArray_ref&) (jni_env=0x7ff5f446fa60, args=..., jobjs=..., jclss=...) at ../libinterp/octave-value/ov-java.cc:1881
#10 0x00007ff62717e752 in octave_java::do_javaObject(void*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, octave_value_list const&) (jni_env_arg=0x7ff5f446fa60, name="org.apache.xerces.parsers.DOMParser", args=...) at ../libinterp/octave-value/ov-java.cc:2577
#11 0x00007ff62717fa78 in FjavaObject(octave_value_list const&, int) (args=...) at ../libinterp/octave-value/ov-java.cc:3079
#12 0x00007ff6270f4f59 in octave_builtin::execute(octave::tree_evaluator&, int, octave_value_list const&) (this=0x7ff5f4028e20, tw=..., nargout=1, args=...) at ../libinterp/octave-value/ov-builtin.cc:59
#13 0x00007ff6271532ca in octave_function::call(octave::tree_evaluator&, int, octave_value_list const&) (this=0x7ff5f4028e20, tw=..., nargout=1, args=...) at ../libinterp/octave-value/ov-fcn.cc:57
#14 0x00007ff6272b2e85 in octave::tree_index_expression::evaluate_n(octave::tree_evaluator&, int) (this=0x7ff5f4d43b30, tw=..., nargout=1) at ../libinterp/parse-tree/pt-idx.cc:521
#15 0x00007ff6272b4747 in octave::tree_index_expression::evaluate(octave::tree_evaluator&, int) (this=0x7ff5f4d43b30, tw=..., nargout=1) at ../libinterp/parse-tree/pt-idx.h:109
#16 0x00007ff627281d22 in octave::tree_simple_assignment::evaluate(octave::tree_evaluator&, int) (this=0x7ff5f499a6f0, tw=...) at ../libinterp/parse-tree/pt-assign.cc:101
#17 0x00007ff6272973a4 in octave::tree_evaluator::visit_statement(octave::tree_statement&) (this=0x7ff5f4004030, stmt=...) at ../libinterp/parse-tree/pt-eval.cc:3075
#18 0x00007ff6272be156 in octave::tree_statement::accept(octave::tree_walker&) (this=0x7ff5f443c0b0, tw=...) at ../libinterp/parse-tree/pt-stmt.h:124
#19 0x00007ff6272975d2 in octave::tree_evaluator::visit_statement_list(octave::tree_statement_list&) (this=0x7ff5f4004030, lst=...) at ../libinterp/parse-tree/pt-eval.cc:3134
#20 0x00007ff6270a55bc in octave::tree_statement_list::accept(octave::tree_walker&) (this=0x7ff5f4479420, tw=...) at ../libinterp/parse-tree/pt-stmt.h:201
#21 0x00007ff627295094 in octave::tree_evaluator::visit_simple_for_command(octave::tree_simple_for_command&) (this=0x7ff5f4004030, cmd=...) at ../libinterp/parse-tree/pt-eval.cc:2517
#22 0x00007ff6272b7c0c in octave::tree_simple_for_command::accept(octave::tree_walker&) (this=0x7ff5f4455650, tw=...) at ../libinterp/parse-tree/pt-loop.h:222
#23 0x00007ff6272972ee in octave::tree_evaluator::visit_statement(octave::tree_statement&) (this=0x7ff5f4004030, stmt=...) at ../libinterp/parse-tree/pt-eval.cc:3053
#24 0x00007ff6272be156 in octave::tree_statement::accept(octave::tree_walker&) (this=0x7ff5f44556b0, tw=...) at ../libinterp/parse-tree/pt-stmt.h:124
#25 0x00007ff6272975d2 in octave::tree_evaluator::visit_statement_list(octave::tree_statement_list&) (this=0x7ff5f4004030, lst=...) at ../libinterp/parse-tree/pt-eval.cc:3134
#26 0x00007ff6270a55bc in octave::tree_statement_list::accept(octave::tree_walker&) (this=0x7ff5f43c3910, tw=...) at ../libinterp/parse-tree/pt-stmt.h:201
#27 0x00007ff62728d4e4 in octave::tree_evaluator::eval(std::shared_ptr<octave::tree_statement_list>&, bool) (this=0x7ff5f4004030, stmt_list=std::shared_ptr<octave::tree_statement_list> (use count 2, weak count 0) = {...}, interactive=true) at ../libinterp/parse-tree/pt-eval.cc:403
#28 0x00007ff62763bc45 in octave::interpreter::main_loop() (this=0x7ff5f40032a0) at ../libinterp/corefcn/interpreter.cc:1243
#29 0x00007ff627638650 in octave::interpreter::execute() (this=0x7ff5f40032a0) at ../libinterp/corefcn/interpreter.cc:806
#30 0x00007ff627e350b4 in octave::interpreter_qobject::execute() (this=0x645a20) at ../libgui/src/interpreter-qobject.cc:87
#31 0x00007ff627f3ff3a in octave::interpreter_qobject::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (_o=0x645a20, _c=QMetaObject::InvokeMetaMethod, _id=3, _a=0x79cbe0) at libgui/src/moc-interpreter-qobject.cc:96
#32 0x00007ff6250aa352 in QObject::event(QEvent*) (this=0x645a20, e=<optimized out>) at kernel/qobject.cpp:1260
#33 0x00007ff62598ae76 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /lib64/libQt5Widgets.so.5
#34 0x00007ff627ea7425 in octave::octave_qapplication::notify(QObject*, QEvent*) (this=0x4eb680, receiver=0x645a20, ev=0x8cd470) at ../libgui/src/octave-qobject.cc:132
#35 0x00007ff6250838a0 in QCoreApplication::notifyInternal2(QObject*, QEvent*) (receiver=0x645a20, event=0x8cd470) at ../../include/QtCore/../../src/corelib/kernel/qobject.h:143
#36 0x00007ff625086b43 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (receiver=0x0, event_type=0, data=0x688880) at kernel/qcoreapplication.cpp:1840
--Type <RET> for more, q to quit, c to continue without paging--c
#37 0x00007ff6250cf187 in postEventSourceDispatch(GSource*, GSourceFunc, gpointer) (s=0x7ff5f4002ed0) at kernel/qeventdispatcher_glib.cpp:277
#38 0x00007ff6212be7af in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#39 0x00007ff6212beb38 in g_main_context_iterate.constprop () at /lib64/libglib-2.0.so.0
#40 0x00007ff6212bec03 in g_main_context_iteration () at /lib64/libglib-2.0.so.0
#41 0x00007ff6250cec06 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (this=0x7ff5f4000b60, flags=...) at kernel/qeventdispatcher_glib.cpp:423
#42 0x00007ff62508221b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (this=this@entry=0x7ff6054beba0, flags=..., flags@entry=...) at ../../include/QtCore/../../src/corelib/global/qflags.h:140
#43 0x00007ff624ef6577 in QThread::exec() (this=<optimized out>) at ../../include/QtCore/../../src/corelib/global/qflags.h:120
#44 0x00007ff624ef7846 in QThreadPrivate::start(void*) () at thread/qthread_unix.cpp:360
#45 0x00007ff62203c432 in start_thread () at /lib64/libpthread.so.0
#46 0x00007ff621f6a9d3 in clone () at /lib64/libc.so.6

<....>


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Fri 12 Jun 2020 10:42:02 PM UTC, comment #27: 

I do not see any change after adding parser.reset().

What I noticed that if I run the loop before print()
(to at least i=1000), then do print and repeat the loop --
it does not crash any more until I restart octave.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Fri 12 Jun 2020 10:34:03 PM UTC, comment #26: 

On Windows with dev Octave (tip of June 10) it just runs fine all along to i = 100000.
Same on my Mageia 7 Linux system (which runs this twice as fast BTW).

Any changes on your Linux systems when adding

parser.reset ();

after creating the parser object?

Philip Nienhuis <philipnienhuis>
Group Member
Fri 12 Jun 2020 10:03:05 PM UTC, comment #25: 

It crashed on mine after i=7027 (with high reproducibility at exactly that same number)

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Fri 12 Jun 2020 09:43:54 PM UTC, comment #24: 

This is a weird one, for sure.

On my Debian system using a current build of the stable branch, I can generate a segfault with the following script (no io package needed, so I doubt that is the source of the problem):


usejava ("jvm");
javaaddpath ("/usr/share/java/xercesImpl.jar")
figure ();
line ();
print ('line.svg');

for i = 1:100000
  i
  parser = javaObject ("org.apache.xerces.parsers.DOMParser");
end


I usually see a crash for me well before 5,000 trips through the loop.

If I comment out the call to the print function, then it runs through all 100,000 trips through the loop.  Executing the script repeatedly also works.

Note that we don't even have to parse a file, just create the parse object!

And somehow it is interacting with the print function?  What?!

Can anyone else duplicate this result?

John W. Eaton <jwe>
Group administrator
Fri 12 Jun 2020 08:16:30 PM UTC, comment #23: 

BTW -- I also noticed that if octave hangs (rather than crashes)
it runs with 200% cpu load (means two threads are running).
Which kind of suggest there is a thread synchronization issue.

Dmitri.
--



Dmitri A. Sergatskov <dasergatskov>
Fri 12 Jun 2020 08:09:20 PM UTC, comment #22: 

I suspect there is nothing wrong with this package, but rather
it exposes some bugs with octave's java interface. We have a few
very strange java-related bugs that are not resolved.
May be https://savannah.gnu.org/bugs/?func=detailitem&item_id=55395 is related? (Reading a large xml file trashes stack?)

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 11 Jun 2020 10:07:58 PM UTC, comment #21: 

I tried 10 sec delay between print/fileread/xmlread and it still
crashes (on the second invocation):


 cat t1.m
sombrero;
print ('sombrero.svg');
pause(10);
str = fileread ('sombrero.svg');
pause(10);
dom = xmlread ('sombrero.svg');



octave:1> pkg load io
octave:2> javaaddpath ("./xercesImpl.jar" )
octave:3> javaaddpath ("./xercesImpl.jar" )
octave:4> javaaddpath ("./xml-apis.jar" )
octave:5> tic; t1; toc
Elapsed time is 21.3784 seconds.
octave:6> tic; t1; toc
fatal: caught signal Segmentation fault -- stopping myself...
Segmentation fault (core dumped)


Dmitri.
--


Dmitri A. Sergatskov <dasergatskov>
Thu 11 Jun 2020 09:54:55 PM UTC, comment #20: 

Thanks for the tests, Mike.

First off, let's not forget that this is all about a mere demo that occasionally causes a segfault. We haven't seen any bug reports about functioning of xmlread.m itself, have we?

If a time delay fixes the issue I'd say it is the easiest solution.

Mike, your hint that this issue might be related Java I/O may have some merit. My experience is that it is rather XML files that have these issues. But those are usually read using Java methods; so there you are.

In the io package there are spreadsheet I/O test scripts that do similar things as xmlread, i.e., writing a file and immediately reading it back. Most spreadsheet file formats are XML-based. Those test scripts have always required some delays, generally with XML and esp. with zipped XML files, otherwise errors (not segfaults) would occur. The more complicated the Java code is, the longer the delays need to be; esp. invoking LibreOffice through the Java-UNO bridge (a beast) would require .5 sec delays.
Now, xmlread.m and xmlwrite.m are contributed functions; until now I simply didn't realize that the xmlread demo might need similar treatment as the spreadsheet test scripts that I did author.
So I am not surprised that a delay mitigates the issue with xmlread as well. Only that this took a while to trickle down :-)

What I'll do:
Add some delay in the xmlread demo and put the XML I/O files back. Since io-2.6.1 a few other bugs have been fixed so I believe a new release might be warranted anyway. If Olaf doesn't get any segfaults I think this bug can be closed then.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 11 Jun 2020 09:10:56 PM UTC, comment #19: 

In my case the crash seems to be happening on the second invocation:


octave:2> pkg load io
octave:3> javaaddpath ("./xml-apis.jar" );
octave:4> javaaddpath ("./xercesImpl.jar" );
octave:5> t1
octave:6> t1
fatal: caught signal Segmentation fault -- stopping myself...
Segmentation fault (core dumped)


Where t1.m is a code from comments 17. On one occasion the octave
just hang w/o crash or returning to command prompt. had to kill -9 it.
Anyway, backtrace it attached.
This is on Centos8.1 with the hg id 2396ee754ce8 (stable).

Dmitri.
--


(file #49244)

Dmitri A. Sergatskov <dasergatskov>
Thu 11 Jun 2020 07:54:14 PM UTC, comment #18: 

Interesting observations Mike. The documentation in the Octave cookbook (http://wiki.octave.org/Cookbook#Load_XML_files) mentions that there is an issue with Java to read from other directories than the working directory. Could that play a role?

I have noticed that xmlread does not work when a variable is passed on to it, I only get it to work via a file.

If timing might be the issue, could it be a solution to build a pause into the xmlread function itself? Obviously, that doesn't improve performance in terms of speed, but it may resolve stability problems. Would you be willing to test this by modifying the xmlread.m and repeat the demo test? If this works, it at least may result in a xmlread version that can be released.

Dennis <dizze>
Thu 11 Jun 2020 07:29:51 PM UTC, comment #17: 

I have tested a few more times with the following observations:

  • Octave file I/O can reliably open and read the SVG file immediately after the print function returns, while the xmlread function often results in a hang or segmentation fault.
  • If the file already exists on the filesystem, the xmlread function works reliably without a hang or segmentation fault.
  • Adding a small time delay between the print function and the xmlread function seems to mitigate the problem.
  • A much smaller / simpler SVG file in the same demo seems to also mitigate the problem.


One suggestion would be to try relying on Octave file I/O functions to read the contents of the file and pass the character buffer to Java XML routines for parsing, instead of only passing a file name to Java.

For example, this set of commands reliably reads the contents of the SVG file into Octave, while the following xmlread function causes the same crash:


sombrero;
print ('sombrero.svg');
str = fileread ('sombrero.svg');
dom = xmlread ('sombrero.svg');


That leads me to believe that this is not related to a problem with the disk or the filesystem or incomplete files not being available, but rather is particular to the Java file I/O system.

Regarding my fourth observation, printing an empty 'axes' instead of 'sombrero' to a SVG file seems to work reliably without a delay.

I am not testing on a SSD or a particularly fast system.

Mike Miller <mtmiller>
Group Member
Thu 11 Jun 2020 03:28:32 PM UTC, comment #16: 

Perhaps I wasn't completely clear. I have jsonlab running in Octave. I only use the function to read json files, but I made that work with just a few small adjustments in the code.I haven't checked the other functions of jsonlab.

My suggestion is to take jsonlab and use that code to parse XML files (jsonlab currently is not able to parse xml, only json files). It is a lot faster than codes that I have used to parse XML to a struct, so besides a possibly more reliable package it may also help to speed up xml parsing. Large xml files take forever or simply cause code such as 'xml2struct' (which uses xmlread) to crash.

Dennis <dizze>
Thu 11 Jun 2020 02:28:08 PM UTC, comment #15: 

If I'm not mistaken there's a current GSOC project for bringing jsonlab to Octave, so your wish seems taken care of :-)

Java itself isn't so bad, it's more the way it has been used that can be troublesome.
FYI, most spreadsheet I/O interfaces in the io package are Java based, all of them have been working reliably for about 9-10 years now.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 11 Jun 2020 12:56:08 PM UTC, comment #14: 

Perhaps the concept of relying on a java library isn't such a good idea after all.

I am going to check if I can find a more future proof way around my problem by obtaining the data in JSON format. There is an excellent package to parse json, called jsonlab (http://iso2mesh.sourceforge.net/cgi-bin/index.cgi?jsonlab). This is written as a matlab/octave native code (as far as I can tell), so no problems with java.

Perhaps someone is interested in translating jsonlab into an XML version. This seems to be quite a major task and personally I do not have time for this. As xmlread is more or less the only way to parse xml in octave, I'd say a more future proof solution is very much welcome.

Dennis <dizze>
Thu 11 Jun 2020 12:12:43 PM UTC, comment #13: 

Dennis, TTBOMK in Windows this has never been a problem. I've only heard of these segfaults / hangs occurring on Linux systems, and then only some (but widely used) distros.

BTW, maybe setting "visible", "off" somewhere in the demo can help avoid the figure popups.

The link that Dmitri posted is quite an illustrative one (thanks Dmitri!).
While a very strictly typed language, the way Java has been applied unfortunately hasn't been so strictly "managed". That in contrast to C/C++ where developers seem to stick much more to conventions.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 11 Jun 2020 10:21:06 AM UTC, comment #12: 

I have repeated the test of Philip up to ii==770 without any error (configuration as per #3) and then aborted because the figure popping up started to become annoying. No errors caused. I am running this on a HP EliteBook x360 1030 G2 with SSD.

In addition to Philip's suggestion, I am wondering whether this may be related to the OS being used. Mike and Dmitri, do you have the possibility to repeat the test with the same configuration on the same machine in e.g. a Windows environment?

Dennis <dizze>
Thu 11 Jun 2020 10:20:58 AM UTC, comment #11: 

I guess my point was to illustrate why people do not like java.
Here is another reading on Xerces:

https://stackoverflow.com/questions/11677572/dealing-with-xerces-hell-in-java-maven

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Thu 11 Jun 2020 08:18:14 AM UTC, comment #10: 

Thanks very much Mike and Dmitri.

Correction:
Please use this link for xmlread/xmlwrite download:
http://hg.code.sf.net/p/octave/io/file/a31d38568851/inst/
(because I now see and only then remembered I had already commented out the demos before finally removing the XML functions).

@Dmitri: both xercesImpl.jar and xml-apis.jar are required for xmlread.m / xmlwrite.m. Maybe xmlread.m can invoke your files too, I'd say it merely needs an (any) XML parser, but that's up to Pantxo (author of the XML functions).

Meanwhile, on my laptop I'm running this:
ii = 0;

while 1
  print ("%d\n", ++ii);
  demo xmlread
endwhile


... and Octave is happily plowing along, count at 600+ now.
Mageia-7, xercesImpl.jar and xml-apis.jar from Nov 26 2010, version 2.11.0, OpenJDK 1.8.0.252

Because AFAICS there's nothing wrong with the xmlread.m code and I can't imagine widely used code like Xerces being at fault, I'm wondering if we're hit by the old elusive issue on fast systems with SSD of trying to read from a file before the low-level system process creating it has completely finalized its operation?
Could easily be tested by inserting a 'pause (0.25)' statement in the demo section immediately before or after the comment line:
'## Read the svg file and check the root node is named "svg"'
(I think (just a WAG) that a .25 sec. delay should do)
Could any of you please try that and report back?

FWIW, my laptop (counter now at 800+) is a 4+ yr old core i5 X230 Thinkpad (w. SSD). No issues there with being too fast :-)

Philip Nienhuis <philipnienhuis>
Group Member
Wed 10 Jun 2020 10:25:54 PM UTC, comment #9: 

On CentOS 8 (Redhat 8) there is no xerces2, but there is xerces-j2. (package name
xerces-j2-2.11.0-34.module_el8.0.0+30+832da3a1.noarch)
Is it the same?

When running demo xmlread I get:

xmlread example 1: failed
xmlread: no xercesImpl.jar and/or xml-apis.jar > v2.11.0 in javaclasspath

I do not have those files on my computer, but I do have
/usr/share/java/xml-commons-apis.jar
and
/usr/share/java/jaxp_parser_impl.jar

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Wed 10 Jun 2020 09:37:09 PM UTC, comment #8: 

I can reproduce it in Octave 5.2.0 or 7.0.0 on Debian, io package version 2.6.1, Xerces2 version 2.12.1, OpenJDK version 11.0.7+9.

I'm only here to confirm the error, and offer myself as a test case for any possible fixes. I don't really use the io package or need this function, just here to help if I can.

Mike Miller <mtmiller>
Group Member
Wed 10 Jun 2020 08:50:14 PM UTC, comment #7: 

@mtmiller,

What is your configuration (OS, Octave version, io package version, etc)?

Would it be possible for you to try the configuration that I listed in comment #3 (possibly on a different OS)?

Dennis <dizze>
Wed 10 Jun 2020 08:00:59 PM UTC, comment #6: 

I can reproduce this on my system, calling 'demo xmlread' either causes Octave to hang using 100% CPU doing something, or a segmentation fault with the following stack trace


#0  0x00007fc4dcf32df0 n/a (n/a + 0x0)
#1  0x00007fc4dc916495 n/a (n/a + 0x0)
#2  0x00007fc4dc916495 n/a (n/a + 0x0)
#3  0x00007fc4dc916450 n/a (n/a + 0x0)
#4  0x00007fc4dc916cf8 n/a (n/a + 0x0)
#5  0x00007fc4dc916cb3 n/a (n/a + 0x0)
#6  0x00007fc4dc90d849 n/a (n/a + 0x0)
#7  0x00007fc4fe104119 n/a (libjvm.so + 0x816119)
#8  0x00007fc4fe4fcb91 n/a (libjvm.so + 0xc0eb91)
#9  0x00007fc4fe4fd4a0 n/a (libjvm.so + 0xc0f4a0)
#10 0x00007fc4fe1b43ac JVM_InvokeMethod (libjvm.so + 0x8c63ac)
#11 0x00007fc4dc91c990 n/a (n/a + 0x0)
#12 0x00007fc4dc916a00 n/a (n/a + 0x0)
#13 0x00007fc4dc916a00 n/a (n/a + 0x0)
#14 0x00007fc4dc916ae6 n/a (n/a + 0x0)
#15 0x00007fc4dc916a00 n/a (n/a + 0x0)
#16 0x00007fc4dc916a00 n/a (n/a + 0x0)
#17 0x00007fc4dc90d849 n/a (n/a + 0x0)
#18 0x00007fc4fe104119 n/a (libjvm.so + 0x816119)
#19 0x00007fc4fe17bd36 n/a (libjvm.so + 0x88dd36)
#20 0x00007fc4fe17dc79 n/a (libjvm.so + 0x88fc79)
#21 0x00007fc537b4dac3 JNIEnv_::CallStaticObjectMethod(_jclass*, _jmethodID*, ...) (liboctinterp.so.7 + 0x988ac3)
#22 0x00007fc537b4634f octave_java::do_javaMethod(void*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, octave_value_list const&) (liboctinterp.so.7 + 0x98134f)
#23 0x00007fc537b4a637 FjavaMethod(octave_value_list const&, int) (liboctinterp.so.7 + 0x985637)


Mike Miller <mtmiller>
Group Member
Wed 10 Jun 2020 07:29:52 PM UTC, comment #5: 

Sorry that link should be;
http://hg.code.sf.net/p/octave/io/file/c1ad88fff372/inst/
(= repo status before removing XML stuff).

Philip Nienhuis <philipnienhuis>
Group Member
Wed 10 Jun 2020 03:59:38 PM UTC, comment #4: 

As I wrote, I gave up trying to convince the octave-forge admin.
Yet I can follow his reasoning to some extent: he has a fairly standard distro where he got that segfault, and reasons that the vast majority of the users has one as well, and releasing an io package with functions that may provoke a segfault even on standard installations isn't a Good Idea.

Maybe you can give it a try by starting a discussion on the help or maintainers ML. You have my sympathy.
Be warned that Java (on which the XML functions are based) is a bit alien to most if not all Octave core devs and I fear that because of that you might get little support. To wit, there are several Java-related patches on the bug and patch tracker that have been sitting there for a loooong time.

BTW you can download the XML I/O functions directly from the repo as well:
http://hg.code.sf.net/p/octave/io/file/tip/inst/

Philip Nienhuis <philipnienhuis>
Group Member
Wed 10 Jun 2020 03:19:19 PM UTC, comment #3: 

Thanks for your explanation. I indeed need xmlread, as it is part of multiple Matlab/Octave scripts. The workaround of copying it from a previous release worked fine, but is cumbersome and may prevent users from updating the io package.

I have installed Octave 5.2.0 with io package version 2.6.1 and copied the xmlread.m from io version 2.4.13. I have java xerces v2_12_1 installed.

I have tested the demo discussed in the cited thread. That is:

while 1
demo xmlread
endwhile

I do not get any error, let alone the segfault. I hope that others can repeat this test and confirm absence of errors on other platforms, as this may demonstrate that the previously reported segfault does not occur. It may have been due to other elements on the reporter's machine. Would this suffice to start re-including xmlread in future releases of the io package?

Dennis <dizze>
Wed 10 Jun 2020 12:11:42 PM UTC, comment #2: 

Obviously the reason is what I wrote in comment #0.
The octave-forge admins refuse to release packages with functions that might cause segfaults, no matter how rarely these occur and no matter if it could be an upstream bug in their system or not.
I've had a long discussion about this but just gave up in the end; see [*] in my submission below.

There is a newer xercesImpl.jar + xmlapis.jar out but (1) I haven't had time to test that and (2) as I couldn't reproduce the segfault in the first place there's little incentive for me to test it anyway.
Neither have I been able to motivate the people experiencing the segfault to try these newer xerces libs; reasons may be that they rather stick to their distro-supplied versions and perhaps they aren't that much interested in Java anyway.

If you really need xmlread/xmlwrite you can simply copy them over from earlier io package releases. Hopefully they work fine for you.

And if you have the energy for it I'd suggest to try to fix the bug/segfault or motivate other people to do it. I'd be happy to re-include the XML functions in te io package but I won't spend more time on it.
:-)

Philip Nienhuis <philipnienhuis>
Group Member
Wed 10 Jun 2020 11:25:58 AM UTC, comment #1: 

For some reason version 2.6.1 of the io package no longer contains xmlread and xmlwrite. This causes problems with codes depending on these routines. What is the reason that these XML routines have been removed from this version?

Dennis <dizze>
Tue 17 Mar 2020 02:13:51 PM UTC, original submission:  

During the release process of io-2.6.0 (still pending) we've hit hangs and crashes related to xmlread.m.
See [*]

Steps to reproduce a crash (taken literally from Olaf's example in the mentioned thread):

octave:2> hf = figure ();
octave:3> sombrero ();
octave:4> fname = [tempname(), ".svg"];
octave:5> print (fname);
octave:6> close (hf)

and loading the needed Java classes:

octave:8> javaaddpath ("/usr/share/java/xercesImpl.jar")
octave:9> javaaddpath ("/usr/share/java/xml-apis.jar")

, I got a segfault with repeatedly executing the further demo code:

octave:10> dom = xmlread (fname);
octave:11> dom.hasChildNodes ()
ans = 1
octave:12> root_node = dom.getChildNodes ().item (0);
octave:13> while (1)
> root_node = dom.getChildNodes ().item (0);
> endwhile
fatal: caught signal Segmentation fault -- stopping myself...
make: *** [Makefile:205: run] Segmentation fault


The xmlread.m code in itself looks correct (conforms to the Javadocs), and trying the essential code parts in Matlab (r2014a) shows no errors and gives correct output. Note that Matlab ships xercesImpl and xml-apis jars into its installation. (I didn't try the while() in the example.)

Furthermore, the involved Java class libs xercesImpl.jar and xml-apis.jar are very widely used (as is Java) so it seems almost inevitable to conclude that the bug is in Octave itself.

FWIW, I've been unable to provoke the hangs or crashes on my Windows systems (Windows 7 -to be retired very soon- and Windows 10) nor on my Mageia 7 Linux systems, all with Octave-7.0.0 and Octave-6.0.0 from Feb. 17.

For the moment I plan to prepare an io package release w/o the xml functions, they can be put back once this issue has been resolved.

Panxto (creator of the xml functions) cc'd

[*] https://sourceforge.net/p/octave/package-releases/405/

Philip Nienhuis <philipnienhuis>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49249:  stack-after-jvm-init.txt added by jwe (29KiB - text/plain)
file #49250:  stack-after-segfault.txt added by jwe (30KiB - text/plain)
file #49248:  bt_10f4ac250b89.txt.gz added by dasergatskov (5KiB - application/gzip)
file #49244:  bt_das.txt.gz added by dasergatskov (4KiB - application/gzip)

 

Digest:
   bug dependencies.

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by dizze (Posted a comment)
  • -email is unavailable- added by philipnienhuis (Submitted the item)
  • -email is unavailable- added by philipnienhuis
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-10-26 siko1056 Release6.0.90 dev
    2020-10-01 siko1056 CategoryOctave Package Interpreter
        Summary[octave forge] (io) xmlread.m makes Octave crash Java xerces library makes Octave crash
    2020-06-13 jwe Attached File- Added stack-after-jvm-init.txt, #49249
        Attached File- Added stack-after-segfault.txt, #49250
    2020-06-13 dasergatskov Attached File- Added bt_10f4ac250b89.txt.gz, #49248
    2020-06-11 dasergatskov Attached File- Added bt_das.txt.gz, #49244
    2020-06-10 mtmiller StatusNone Confirmed
        Summary[octave-forge] (io) xmlread.m makes Octave crash [octave forge] (io) xmlread.m makes Octave crash
    2020-04-28 philipnienhuis Dependencies- Depends on bugs #58266
    2020-03-25 philipnienhuis Summary(octave-forge) [io] xmlread.m makes Octave crash [octave-forge] (io) xmlread.m makes Octave crash
    2020-03-17 philipnienhuis Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code