bugGNU Octave - Bugs: bug #55727, Feature request: add GEJSV as an...

 
 

bug #55727: Feature request: add GEJSV as an additional svd_drivers

Submitter:  Eddy <count>
Submitted:  Sat 16 Feb 2019 10:24:23 PM UTC
   
 
Category:  Octave Function Severity:  1 - Wish
Priority:  3 - Low Item Group:  Feature Request
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 10 Mar 2021 07:11:00 AM UTC, comment #22: 

The w64-64-default buildbot compiled Octave without errors after the last change:
http://buildbot.octave.org:8010/#/builders/18/builds/826

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Tue 09 Mar 2021 07:49:14 PM UTC, comment #21: 

I pushed a patch that will hopefully fix the build error with 64bit Fortran integers here:
https://hg.savannah.gnu.org/hgweb/octave/rev/3f45f5b6fa7f

Markus Mützel <mmuetzel>
Group administrator
Tue 09 Mar 2021 09:54:01 AM UTC, comment #20: 

A fresh mxe-octave build tree fixed it (pfew); but took a long time on my not-too-recent HW.

I don't know why and how my previous build tree got borked; it was exactly coincidental with the svd cset of this bug report.


Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 10:58:44 PM UTC, comment #19: 

With the GEJSV patch out (i.e., updated to hgid f26f9dbfb2c5) the crash still occurs.
Which leads to the conclusion that something in my build tree has gone wrong :-(

So, sorry for the noise.

I had plans to upgrade my Linux box to Mageia 8 anyway, this is a good motive to prioritize it.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 09:15:33 PM UTC, comment #18: 

Indeed, openblas or ref blas makes no difference, both crash.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 09:05:45 PM UTC, comment #17: 

Checking file sizes in the raw mxe-octave install tree I have

  • libblas.dll     593408 b
  • libopenblas.dll 28615 K

so it looks like reference BLAS. Which could be alarming as AFAIU it is supposed to be unconditionally stable.
Switching BLAS on the fly isn't possible, the Linux drive where the mxe-octave tree lives is read-only from Windows. But now on Linux I can do it by hand. But I have no illusions.

Remarkable that on Linux here are no problems.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 07:10:32 PM UTC, comment #16: 

It seems to be crashing in one of the BLAS functions.
Are you using reference BLAS or OpenBLAS (`version -blas` at Octave's command prompt)?
In any case, does it still crash with the other one? You could use John's BLAS switcher to select the other one. It should be at "L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\blas_switch.exe".

I'm not sure if that crash is caused by the change for this report. It looks like it is a simple dot product a*b of two complex double vectors that caused the crash.

Markus Mützel <mmuetzel>
Group administrator
Mon 08 Mar 2021 07:06:51 PM UTC, comment #15: 

forgot attachment

(file #51028)

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 07:06:14 PM UTC, comment #14: 

Here's another backtrace with "thread apply all bt"

ext I'll backout the GEJSV cset and crossbuild

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 06:31:11 PM UTC, comment #13: 

OK, to my surprise it was quite easy, just start Octave in a cli, type __debug-octave__ + wait a little for a gdb window to emerge.
However, saving the backtrace to a file isn't as easy, I had to copy it screen by screen by hand and then stitch wrapped lines together.

It is a backtrace from Octave-7.0.0 on my Linux drive (L:) doing "test CMatrix.cc-tst" in OCTAVE_HOME\share\octave\7.0.0\etc\tests\liboctave\array.
Here it is:

[Thread 3508.0x23ac exited with code 0]

Thread 7 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 3508.0x2664]
0x0000000000f63d17 in zdotu_ () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\libblas.dll
(gdb) bt
#0  0x0000000000f63d17 in zdotu_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\libblas.dll
#1  0x000000006cad3a2f in xzdotu_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctave-8.dll
#2  0x000000006c98d757 in liboctave-8!_Z5xgemmRK13ComplexMatrixS1_15blas_trans_typeS2_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctave-8.dll
#3  0x000000006c98d85c in liboctave-8!_ZmlRK13ComplexMatrixS1_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctave-8.dll
#4  0x000000006f2a3709 in liboctinterp-8!_Z16Fcmdline_optionsRK17octave_value_listi ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#5  0x000000006f762792 in liboctinterp-8!_ZN6octave9binary_opERNS_9type_infoEN12octave_value9binary_opERKS2_S5_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#6  0x000000006f7c2f76 in liboctinterp-8!_ZN6octave8profiler14enter_functionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#7  0x000000006f7d84e4 in liboctinterp-8!_ZN6octave14tree_evaluator23convert_to_const_vectorEPNS_18tree_argument_listE
    () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#8  0x000000006f7ec5e1 in liboctinterp-8!_ZN6octave14tree_evaluator15remove_autoloadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#9  0x000000006fde74fc in liboctinterp-8!_ZN6octave21text_element_fontsize6acceptERNS_14text_processorE ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#10 0x000000006f7e077c in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#11 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE
    () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#12 0x000000006f7d7a18 in liboctinterp-8!_ZN6octave14tree_evaluator21execute_user_functionER20octave_user_functioniRK17octave_value_list () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#13 0x000000006f75a7bc in liboctinterp-8!_Z9FisargoutRN6octave11interpreterERK17octave_value_listi ()
   from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
--Type <RET> for more, q to quit, c to continue without paging--c
#14 0x000000006f7ec495 in liboctinterp-8!_ZN6octave14tree_evaluator15remove_autoloadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#15 0x000000006f7da60f in liboctinterp-8!_ZN6octave14tree_evaluator11eval_stringERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEbRii () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#16 0x000000006f7da9e0 in liboctinterp-8!_ZN6octave14tree_evaluator4evalERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#17 0x000000006fab5cd5 in liboctinterp-8!_ZN6octave11interpreter4evalERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEi () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#18 0x000000006f7a8b41 in liboctinterp-8!_Z5FevalRN6octave11interpreterERK17octave_value_listi () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#19 0x000000006f6c16fb in liboctinterp-8!_ZN14octave_builtin7executeERN6octave14tree_evaluatorEiRK17octave_value_list ()
 from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#20 0x000000006f6fcd4e in liboctinterp-8!_ZN15octave_function4callERN6octave14tree_evaluatorEiRK17octave_value_list () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#21 0x000000006f7ec495 in liboctinterp-8!_ZN6octave14tree_evaluator15remove_autoloadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#22 0x000000006fde74fc in liboctinterp-8!_ZN6octave21text_element_fontsize6acceptERNS_14text_processorE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#23 0x000000006f7e077c in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#24 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#25 0x000000006f7d909f in liboctinterp-8!_ZN6octave14tree_evaluator23visit_try_catch_commandERNS_22tree_try_catch_commandE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#26 0x000000006f7e06bc in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#28 0x000000006f7e06bc in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#29 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#30 0x000000006f7d9c2b in liboctinterp-8!_ZN6octave14tree_evaluator28visit_unwind_protect_commandERNS_27tree_unwind_protect_commandE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#31 0x000000006f7e06bc in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#32 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#33 0x000000006fdda4bc in liboctinterp-8!_ZN6octave11text_parser5parseERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERK12caseless_str () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#34 0x000000006f7df342 in liboctinterp-8!_ZN6octave14tree_evaluator24visit_simple_for_commandERNS_23tree_simple_for_commandE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#35 0x000000006f7e06bc in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#36 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#37 0x000000006f7d7a18 in liboctinterp-8!_ZN6octave14tree_evaluator21execute_user_functionER20octave_user_functioniRK17octave_value_list () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#38 0x000000006f75a7bc in liboctinterp-8!_Z9FisargoutRN6octave11interpreterERK17octave_value_listi () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#39 0x000000006f7ec495 in liboctinterp-8!_ZN6octave14tree_evaluator15remove_autoloadERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#40 0x000000006fde74fc in liboctinterp-8!_ZN6octave21text_element_fontsize6acceptERNS_14text_processorE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#41 0x000000006f7e077c in liboctinterp-8!_ZN6octave14tree_evaluator15visit_statementERNS_14tree_statementE () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#42 0x000000006f7ce3ec in liboctinterp-8!_ZN6octave14tree_evaluator20visit_statement_listERNS_19tree_statement_listE ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#43 0x000000006f7d012a in liboctinterp-8!_ZN6octave14tree_evaluator4evalERSt10shared_ptrINS_19tree_statement_listEEb ()
from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#44 0x000000006f7e0eff in liboctinterp-8!_ZN6octave14tree_evaluator4replEv () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#45 0x000000006fabfb0d in liboctinterp-8!_ZN6octave11interpreter7executeEv () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctinterp-8.dll
#46 0x000000006c10a02b in liboctgui-6!_Z15liboctgui_hg_idB5cxx11v () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctgui-6.dll
#47 0x000000006694b53e in QObject::event(QEvent*) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#48 0x0000000002ca7f73 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Widgets.dll
#49 0x000000006c15b076 in liboctgui-6!_Z15liboctgui_hg_idB5cxx11v () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\liboctgui-6.dll
#50 0x000000006691e77a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#51 0x0000000066924101 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#52 0x00000000669784da in QEventDispatcherWin32::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#53 0x000000006691d363 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#54 0x0000000066758b30 in QThread::run() () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#55 0x0000000066759723 in QThread::setTerminationEnabled(bool) () from L:\philip\devel\octdev\mxe\mxe_64w_20210209D\dist\octave7\mingw64\bin\Qt5Core.dll
#56 0x00007ffb623e7034 in KERNEL32!BaseThreadInitThunk () from C:\Windows\System32\kernel32.dll
#57 0x00007ffb63edd241 in ntdll!RtlUserThreadStart () from C:\Windows\SYSTEM32\ntdll.dll
#58 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)



Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 10:57:25 AM UTC, comment #12: 

I'll try but tonight, gotta do some paid work now :-)

Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 10:45:26 AM UTC, comment #11: 

I backtrace of the crash might be very helpful.

Markus Mützel <mmuetzel>
Group administrator
Mon 08 Mar 2021 10:34:42 AM UTC, comment #10: 

With a crossbuild of the "clean" default branch (w/o local patches but updated to the GEJSV cset) the crash also occurs.
So that patch of bug #48564 is off the hook. If it weren't I'd also expect to have seen problems on Linux (building the dist archive for mxe-octave includes a 'make check' in my setup) and there were none there.

I think the best next step is to backout the GEJSV cset and build that one. Let's see when I'll have time, maybe tonight.



Philip Nienhuis <philipnienhuis>
Group Member
Mon 08 Mar 2021 08:16:01 AM UTC, comment #9: 

@Philip: That sounds like a good approach to me.
For a start, it might be good enough for an initial hint to get the backtrace from the build you currently have.

Checking the changes again, it looks like the default driver might have changed:

--- a/libinterp/corefcn/svd.cc
+++ b/libinterp/corefcn/svd.cc
@@ -62,9 +62,12 @@
 static typename octave::math::svd<T>::Driver
 svd_driver (void)
 {
-  return (Vsvd_driver == "gesvd"
-          ? octave::math::svd<T>::Driver::GESVD
-          : octave::math::svd<T>::Driver::GESDD);
+  if (Vsvd_driver == "gejsv")
+      return octave::math::svd<T>::Driver::GEJSV;
+  else if (Vsvd_driver == "gesdd")
+      return octave::math::svd<T>::Driver::GESDD;
+  else
+      return octave::math::svd<T>::Driver::GESVD;  // default
 }


The new default matches the documentation though.

Also, the w64-64-default builder seems to complain about incompatible integer assignments:
http://buildbot.octave.org:8010/#/builders/18/builds/824/steps/8/logs/stdio

Compilation also stopped. But there's no reference to the error in the log file excerpt.
I'll try to setup a build setup with similar configuration and see if I can reproduce the error.

Markus Mützel <mmuetzel>
Group administrator
Sun 07 Mar 2021 10:01:30 PM UTC, comment #8: 

Configure flags same as you, except (from top of my head) --enable-system-opengl=no instead of your --with-pkg-dir=../mxe-octave-pkg. Oh and I use the native Octave installation, not the one built in mxe-octave.

I've carefully cleaned up the Octave installation in the mxe-octave tree and  built it again, same result.

Maybe it is related to bug #48564 (MLdivide/MRdivide issues), the patch from there is always in my builds as I use that functionality sometimes. My suspicion is raised because CMatrix.cc-tst merely contains simple matrix operations tests. There were no merge conflicts so it's just a guess.
I just found that Octave-7.0.0 is now unusable for me, most operations lead to a crash.

Debugging? - maybe. I run 7.0.0 directly from the mxe-octave build tree (using ext2fsd driver) so I suppose it's a mere case of rebuilding w/o stripping symbols, running Octave and then attaching the debugger to it.
It may be more productive to first build a "pure" default Octave (w/o local patches) and then add the patch from bug #48564. I can do that tomorrow, booted into Linux, in the background on my home desktop while remotely working on my company's network. Otherwise it'll be much later this week that I can spend time on this.

[OT]
BTW Although apparently I'm the only one who ever hit the MLdivide / MRdivide / 1/<matrix> issues in bug #48564, I still think they need to be fixed. The patch I've been using doesn't fully achieve Matlab compatibility yet.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 07 Mar 2021 06:33:50 PM UTC, comment #7: 

It doesn't crash for me on Windows 10 with MXE Octave configured with:

./configure --enable-devel-tools --enable-windows-64 --enable-octave=default --enable-binary-packages --with-ccache --with-pkg-dir=../mxe-octave-pkg


Which configure flags did you use?

Does it also crash if you only run the tests in CMatrix.cc-tst?
Could you try to gather a gdb backtrace of the crash?

Markus Mützel <mmuetzel>
Group administrator
Sun 07 Mar 2021 05:53:00 PM UTC, comment #6: 

With dev Octave updated to this cset, it crashes hard on Windows during __run_test_suite__ , to be precise during processing of CMatrix.cc-tst. It disappears from screen w/o any of the usual Windows error messages "This program has stopped working... etc".

On Linux __run_test_suite__ proceeds uneventful.

The previous crossbuild I made and that ran the test suite w/o issues was at aef9c5caedcd "call handle class set method with nargout = 0 (bug #60143)".
That previous crossbuild runs CMATRICX.cc-tst from the current Octave (= with this cset) w/o issue.

My mxe-octave build tree is fully up-to-date.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 07 Mar 2021 12:40:28 PM UTC, comment #5: 

Thank you very much for your contribution.
I pushed it with very minor changes here:
https://hg.savannah.gnu.org/hgweb/octave/rev/9b6bf68ea663

I had to slightly relax the tolerance in the BISTs for them to pass on my Ubuntu 20.10.

Imho, this is one of those gems you'll find once in a while when roaming across the bug tracker. The author clearly spent a considerable amount of time to create a very high quality contribution.
I'm both delighted and sad each time I come across something like this. Delighted because it is encouraging to see that talented people are willing to contribute to FOSS. But it also saddens me because it sometimes takes quite some time for these contributions to get noticed by us (about two years in this case).
I believe that all maintainers would agree that we try to do our best to keep up with the task. But sometimes some good contributions might fall below our radar. I hope that doesn't discourage talented contributors.

Markus Mützel <mmuetzel>
Group administrator
Tue 19 Mar 2019 05:11:57 PM UTC, comment #4: 

Update to the patch of the GEJSV driver (replace previous one), see attachment bug55727_gejsv_v2.patch.

Fixed few bugs in previous patch: about size of singular vector under some condition, and about the test case of GEJSV. Also it is now follow the C++ style guide tightly.

Now the patch is been tested for all combination of: different matrix sizes (tall, square, fat) with different number of output arguments, different data types (single, double, complex single, complex double) and full SVD v.s. economy SVD. Tested against --enable-address-sanitizer-flags.

Personally, I have used it for a while, so I think it is mature enough now.

To be more clear about what is inside, the changeset is consist of:


Estimated
lines-of-code    What for
202     declare FORTRAN functions
136     C++ interface to FORTRAN utilizer functions
129     compute optimal work space size for GEJSV
132     C++ interface to GEJSV
88      svd C++ class
rest    Fsvd () and documents.


The test script is also uploaded for convenient. Just run test_gejsv_all_cases.m.

(file #46580, file #46581, file #46582)

Eddy <count>
Sun 24 Feb 2019 08:58:14 PM UTC, comment #3: 

As an additional optional driver this should not do any harm.

The patch adds about 800 lines of code.  Some comments are included.  On a first sight it looks good.  I did not test it yet.

Kai Torben Ohlhus <siko1056>
Group Member
Sun 24 Feb 2019 02:57:20 PM UTC, comment #2: 

Add more time test results for svd drivers GEJSV, GESDD and GESVD (see attachment).

Looks good generally, i.e. have almost the same time cost as svd_driver GESVD when asked for left and right singular vectors. Severe slow down happens when matrix size (max of rows and cols) is close to or higher than 10^6.

(file #46349)

Eddy <count>
Mon 18 Feb 2019 02:14:52 AM UTC, comment #1: 

See attachment for the patch (with comments and tests) and test m-file.

The lack of workspace size query in [D,S]GEJSV make this patch significantly larger and involves a bunch of logic to compute it...

However, at the end of the day, GEJSV is shown to be very promising: it passes all the extreme tests appear in bug #55564.

Notably this one:


N = 26;
A = compan (fliplr ([1, 1 ./ cumprod(1:N)]));

a = A(1,:);
c = [1, -(1-sumsq(a)), -sumsq(a(1:end-1))];
s0 = sort(1-[roots(c); zeros(length(a)-2,1)], 'descend') .^0.5;

svd_driver('gesvd');
s1 = svd (A);

svd_driver('gejsv');
s2 = svd (A);

disp('   formula      gesvd        gejsv');
disp([s0 s1 s2]);

   formula      gesvd        gejsv
   6.0890e+26   6.0890e+26   6.0890e+26
   1.0000e+00   3.7617e+09   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   1.0000e+00   1.0000e+00
   1.0000e+00   6.6233e-01   1.0000e+00
   6.6233e-01   2.6583e-10   6.6233e-01


And the time cost is only slightly higher than gesvd.


tocs = @(s) fprintf('t = %7.3f sec: %s\n', toc(), s);

a = rand (2000);
svd_driver ('gesvd');
tic; [u1,s1,v1] = svd (a); tocs(svd_driver ());
svd_driver ('gejsv');
tic; [u2,s2,v2] = svd (a); tocs(svd_driver ());

% output
t =  54.570 sec: gesvd
t =  56.850 sec: gejsv


Memory footprint is larger than GESDD, but totally acceptable.


lwork = extra workspace size. measured in number of double type number.
econ = economy-sized decomposition.
Tested by e.g. svd_driver('gejsv'); [u,s,v]=svd(rand(100,10), 'econ'), with modification to source code to print lwork.

m x n \ algo        SDD        SDDecon        JSV        JSVecon
4x4                268        268        4292        4292
100 x 100        30700        30700        10400        20600
10 x 100        3310        770        7370        4490
100 x 10        3310        770        7370        4490
1000 x 10        32110        770        36170        4490
10000 x 10        320110        770        324170        20010


The large memory cost even for very small problem is due to the routine ormqr, somehow it tells me that it needs space at least 64*(64+1)=4160 for optimal performance.

If we are affordable to accept GESVD as default, why not pay a little more to accept GEJSV.


cheers


(file #46299, file #46300)

Eddy <count>
Sat 16 Feb 2019 10:24:23 PM UTC, original submission:  

Currently Octave supports two svd_drivers: GESVD and GESDD.

However, in some extreme cases, both of then are not satisfactory in terms of relative accuracy.

Such extreme cases can occur in research. e.g. several examples in bug #55564, e.g. total least square method needs the singular vector corresponding to the smallest singular value. See ref1 below for more.

The Jacobi method is known to be more accurate than both QR (GESVD) and Divide&Conquer (GESDD) method, see [ref1]. LAPACK includes such implementations named GEJSV and GESVJ. Accuracy and speed performance is analyzed in [ref2].

The GESVJ, is seems an internal routine for GEJSV, probably not intended for general use.

I'm writing a patch to make GEJSV available in Octave, hope that works.

[ref1]: Jacobi’s Method is More Accurate than QR
[ref2]: The Singular Value Decomposition: Anatomy of Optimizing an Algorithm for Extreme Scale

LAPACK manual GEJSV
LAPACK manual GESVJ

Eddy <count>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #51028:  bug#55727_bt.txt added by philipnienhuis (12KiB - text/plain)
file #46580:  bug55727_gejsv_v2.patch added by count (36KiB - text/x-patch)
file #46581:  test_gejsv_all_cases.m added by count (3KiB - text/x-objcsrc)
file #46582:  svd_std_self_test.m added by count (1KiB - text/x-objcsrc)
file #46349:  blas_speed_survey_svd.pdf added by count (52KiB - application/pdf - time cost of svd drivers.)
file #46299:  svd-add-gejsv-v1.patch added by count (34KiB - text/x-patch - a patch to add gejsv svd driver)
file #46300:  test_gejsv_all_cases.m added by count (2KiB - text/x-objcsrc - a patch to add gejsv svd driver)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by philipnienhuis
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by count (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 14 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-03-10 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2021-03-08 philipnienhuis Attached File- Added bug#55727_bt.txt, #51028
    2021-03-08 philipnienhuis Carbon-Copy- Added philipnienhuis
    2021-03-07 mmuetzel StatusPatch Submitted Ready For Test
    2019-03-19 count Attached File- Added bug55727_gejsv_v2.patch, #46580
        Attached File- Added test_gejsv_all_cases.m, #46581
        Attached File- Added svd_std_self_test.m, #46582
    2019-02-24 siko1056 Severity3 - Normal 1 - Wish
        Priority5 - Normal 3 - Low
        StatusNone Patch Submitted
    2019-02-24 count Attached File- Added blas_speed_survey_svd.pdf, #46349
    2019-02-18 count Attached File- Added svd-add-gejsv-v1.patch, #46299
        Attached File- Added test_gejsv_all_cases.m, #46300

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code