bugGNU Octave - Bugs: bug #58957, [octave forge] (sparsersb) Failure...

 
 

bug #58957: [octave forge] (sparsersb) Failure to install and crash in function

Submitter:  John Donoghue <lostbard>
Submitted:  Fri 14 Aug 2020 06:32:59 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  None
Originator Name:  JohnD Open/Closed:  * Closed
Release:  * 6.0.92 Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 02 Nov 2020 11:49:01 AM UTC, comment #69: 

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Mon 02 Nov 2020 11:48:04 AM UTC, comment #68: 

Don't worry. I won't touch it. ;-)
I just wanted your confirmation.

I pushed the patch here:
https://hg.octave.org/mxe-octave/rev/2734b3818171

The change to the build rule can be reverted, once you released 1.2.0.10 (or whatever version will contain a fix).

I opened bug #59398 for the error message about empty indexing.

Markus Mützel <mmuetzel>
Group administrator
Mon 02 Nov 2020 10:43:48 AM UTC, comment #67: 

Hi Dmitri and Markus,
nice to know that no errors manifest from sparsersb so far.

Thanks for the reverts Markus.

Let the patch be for version 1.2.0.9 only: I patched upstream (upcoming 1.2.0.10) so the patch won't be necessary anymore.

WRT the copious output: yes, that's a feature ;-) please keep it.

WRT sparsersbtg : that is internal stuff not meant for general use (is a test generator).
I cannot reproduce that error -- must have something to do with 'sprand' on Windows.. What's the output of 'sparsersbtg ([1])' there ?
The bug must be trivial to fix: up to you if you want to open a new bugreport for it.

dezperado <dezperado>
Mon 02 Nov 2020 09:54:41 AM UTC, comment #66: 

I locally removed the change from comment #61 and added the sed command from comment #64 (see attachment). I also reverted the local change we made to the build rule for librsb and recompiled Octave 6 for Windows.

With that change, `pkg test sparsersb` no longer crashes Octave.

So I believe, we can use that as a (temporary) fix.

However, there is lots of output in the command window during the test. The output wasn't captured in the diary (and it is a pain to copy many lines of text from the command window on Windows). So, to show just a small portion:

  ..s\sparsersb-1.0.8\x86_64-w64-mingw32-api-v55\sparsersb.cc-tst Will autotune matrix: 100 x 100, type D, 4000 nnz, 40 nnz/r, 21 subms, 16 lsubms, 2.4160 bpnz.
10 iterations (4 th.) took 0.001013s; avg 0.0001013s ( +/-  99.98/900.00 %); best 1.525e-08s; worst 0.001013s; std dev. 0.0003039 (taking best).
Reference operation time is 1.52477e-08 s (5.247e+05 Mflops) with 4 threads.
Starting merge (same threads) based auto-tuning procedure (transA=N, nrhs=1) (max 6 steps, inclusive 3 grace steps) on: 100 x 100, type D, 4000 nnz, 40 nnz/r, 21 subms, 16 lsubms, 2.4160 bpnz (tpop: 1.525e-08  Mflops: 0.000)
Merge (16 -> 10 leaves) took w.c.t. of 0.001001s, ~0s of computing time (of which 0s sorting, 0s analysis)
10 iterations (4 th.) took 0.002015s; avg 0.0002015s ( +/-  99.99/402.72 %); best 1.524e-08s; worst 0.001013s; std dev. 0.000403 (taking best).
Reference operation time is 1.52431e-08 s (5.248e+05 Mflops) with 4 threads.
After merge step 1: tpop: 1.524e-08 s   ~Mflops: 0.000   nsubm:10 otn:4
Applying merge (16 -> 10 leaves, 4 th.) yielded NEGLIGIBLE change (1th in a row) (old/new=1.00030x): 1.525e-08s -> 1.524e-08s, so IGNORING this instance.
Merge (10 -> 7 leaves) took w.c.t. of 0s, ~0s of computing time (of which 0s sorting, 0s analysis)
10 iterations (4 th.) took 0.000999s; avg 9.99e-05s ( +/-  99.98/900.00 %); best 1.526e-08s; worst 0.000999s; std dev. 0.0002997 (taking best).
Reference operation time is 1.52622e-08 s (5.242e+05 Mflops) with 4 threads.
After merge step 2: tpop: 1.526e-08 s   ~Mflops: 0.000   nsubm:7 otn:4
Applying merge (10 -> 7 leaves, 4 th.) yielded NEGLIGIBLE change (2th in a row) (old/new=0.99905x): 1.525e-08s -> 1.526e-08s, so IGNORING this instance.
Merge (7 -> 4 leaves) took w.c.t. of 0s, ~0s of computing time (of which 0s sorting, 0s analysis)
10 iterations (4 th.) took 0.00101s; avg 0.000101s ( +/-  99.98/900.00 %); best 1.904e-08s; worst 0.00101s; std dev. 0.0003031 (taking best).
Reference operation time is 1.9043e-08 s (4.201e+05 Mflops) with 4 threads.
After merge step 3: tpop: 1.904e-08 s   ~Mflops: 0.000   nsubm:4 otn:4
Applying merge (7 -> 4 leaves, 4 th.) yielded SLOWDOWN (1th of 3 tolerable) of  1.249x: 1.525e-08s -> 1.904e-08s.
Merge (4 -> 1 leaves) took w.c.t. of 0s, ~0s of computing time (of which 0s sorting, 0s analysis)
10 iterations (4 th.) took 0s; avg 0s ( +/-   -inf/   nan %); best 1.908e-08s; worst 0s; std dev. 0 (taking best).
Reference operation time is 1.90848e-08 s (4.192e+05 Mflops) with 4 threads.
After merge step 4: tpop: 1.908e-08 s   ~Mflops: 0.000   nsubm:1 otn:4
Applying merge (4 -> 1 leaves, 4 th.) yielded SLOWDOWN (2th of 3 tolerable) of  1.252x: 1.525e-08s -> 1.908e-08s.
Merged all the matrix leaves: no reason to continue merging.
A total of 4 merge steps (of max 6) (16 -> 1 subms) took 0.105s (of which 0.017s partitioning, 0s I/O); computing times: 0s in par. loops, 0s sorting, 0s analyzing)
Total merge + benchmarking process took 0.105s, equivalent to 6886154.6/6886154.6 new/old ops (0.001005s for 1 clones -- as 65907.4/65907.4 ops, or 65907.4/65907.4 ops per clone), SPEEDUP of  1.000x (NO SPEEDUP)
Merging based autotuning FAILED (=NO SPEEDUP); let's try splitting then...


Is this to be expected?
If it isn't we should probably follow up on a new bug report.

Additionally, there is one failing BIST:

>>>>> processing D:\SVN\Octave\test\octave-2020-11-02-10-17-w64_librsb\octave-2020-11-02-10-17-w64\mingw64\share\octave\packages\sparsersb-1.0.8\sparsersbtg.m
***** test
 assert( length( sparsersbtg () ) >= 57846 )
 assert( length( sparsersbtg ([1,1,1,1,1,1]) ) >= 11218 )
 assert( length( sparsersbtg ([1,1,1]) ) >= 11218 )
 assert( length( sparsersbtg ([1]) ) == 0 )
!!!!! test failed
invalid empty index expression


To reproduce:

>> length( sparsersbtg () )
error: invalid empty index expression
error: called from
    sparsersbtg at line 96 column 8


Should I open a separate report for this, too?

(file #50189)

Markus Mützel <mmuetzel>
Group administrator
Sun 01 Nov 2020 11:32:00 PM UTC, comment #65: 

That definitely fixed ASAN error:


LD_PRELOAD=/usr/lib64/libasan.so.5 ./render
%%MatrixMarket matrix coordinate real general
1 1 1
1        1        1
OK



Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 01 Nov 2020 09:44:09 PM UTC, comment #64: 

Thanks Dmitri!

Yes, it was worth.

Indeed, the realloc'ated pointers there were being 'lost', and the original arrays were being freed twice..

The simplest fix seems to be rebuilding librsb after:
 sed -i 's/.A.=.rsb__real.*\([,;]\)/1\1/g' rsb_eps.c
 (which neutralizes the realloc's).

Could you guys give it a try and check if the "render" tests and demo now work ?


Markus, your patch at comment #61 seems adequate, but if the above fixes the problem, maybe encoding that sed line in

 https://hg.octave.org/mxe-octave/file/tip/src/librsb.mk

and adding a note that this is for 1.2.0.9 only would be enough to close the bug.. right ?

dezperado <dezperado>
Sat 31 Oct 2020 07:38:03 PM UTC, comment #63: 

I compiled (on linux) librsdb and the test program with
-fsanitize=address -fno-omit-frame-pointer .


LD_PRELOAD=/usr/lib64/libasan.so.5 ./render
=================================================================
==37196==ERROR: AddressSanitizer: attempting double-free on 0x603000000220 in thread T0:
    #0 0x7f4b2ba5c7b0 in __interceptor_free (/usr/lib64/libasan.so.5+0xef7b0)
    #1 0x7f4b2ab368f5 in rsb__free /home/dima/src/librsb-1.2.0.9/rsb_sys.c:389
    #2 0x7f4b2ab60d27 in rsb__destroy_coo_matrix_t /home/dima/src/librsb-1.2.0.9/rsb_internals.c:247
    #3 0x7f4b2ab33fea in rsb_dump_postscript_from_mtx_t /home/dima/src/librsb-1.2.0.9/rsb_eps.c:959
    #4 0x7f4b2ab3482f in rsb__do_mtx_render /home/dima/src/librsb-1.2.0.9/rsb_eps.c:1101
    #5 0x7f4b2ab438af in rsb_mtx_rndr /home/dima/src/librsb-1.2.0.9/rsb_rsb.c:1317
    #6 0x400f15 in render_test() /home/dima/src/rsbbugs-0.0/render.cc:26
    #7 0x400d28 in main /home/dima/src/rsbbugs-0.0/render.cc:47
    #8 0x7f4b2942e6a2 in __libc_start_main (/lib64/libc.so.6+0x236a2)
    #9 0x400d8d in _start (/home/dima/src/rsbbugs-0.0/render+0x400d8d)

0x603000000220 is located 0 bytes inside of 4-byte region [0x603000000220,0x603000000224)
freed by thread T0 here:
    #0 0x7f4b2ba5cfb8 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xeffb8)
    #1 0x7f4b2ab36717 in rsb__do_realloc /home/dima/src/librsb-1.2.0.9/rsb_sys.c:263
    #2 0x7f4b2ab366c8 in rsb__realloc /home/dima/src/librsb-1.2.0.9/rsb_sys.c:223
    #3 0x7f4b2ab32619 in rsb_dump_postscript_from_coo /home/dima/src/librsb-1.2.0.9/rsb_eps.c:724
    #4 0x7f4b2ab33fcc in rsb_dump_postscript_from_mtx_t /home/dima/src/librsb-1.2.0.9/rsb_eps.c:953
    #5 0x7f4b2ab3482f in rsb__do_mtx_render /home/dima/src/librsb-1.2.0.9/rsb_eps.c:1101
    #6 0x7f4b2ab438af in rsb_mtx_rndr /home/dima/src/librsb-1.2.0.9/rsb_rsb.c:1317
    #7 0x400f15 in render_test() /home/dima/src/rsbbugs-0.0/render.cc:26
    #8 0x400d28 in main /home/dima/src/rsbbugs-0.0/render.cc:47

previously allocated by thread T0 here:
    #0 0x7f4b2ba5d990 in __interceptor_posix_memalign (/usr/lib64/libasan.so.5+0xf0990)
    #1 0x7f4b2ab3683c in rsb__aligned_malloc /home/dima/src/librsb-1.2.0.9/rsb_sys.c:364
    #2 0x7f4b2ab3692d in rsb__malloc /home/dima/src/librsb-1.2.0.9/rsb_sys.c:417
    #3 0x7f4b2ab609e8 in rsb__xallocate_coo_matrix_t /home/dima/src/librsb-1.2.0.9/rsb_internals.c:215
    #4 0x7f4b2ab606da in rsb__allocate_coo_matrix_t /home/dima/src/librsb-1.2.0.9/rsb_internals.c:179
    #5 0x7f4b2ab33dd4 in rsb_dump_postscript_from_mtx_t /home/dima/src/librsb-1.2.0.9/rsb_eps.c:947
    #6 0x7f4b2ab3482f in rsb__do_mtx_render /home/dima/src/librsb-1.2.0.9/rsb_eps.c:1101
    #7 0x7f4b2ab438af in rsb_mtx_rndr /home/dima/src/librsb-1.2.0.9/rsb_rsb.c:1317
    #8 0x400f15 in render_test() /home/dima/src/rsbbugs-0.0/render.cc:26
    #9 0x400d28 in main /home/dima/src/rsbbugs-0.0/render.cc:47

SUMMARY: AddressSanitizer: double-free (/usr/lib64/libasan.so.5+0xef7b0) in __interceptor_free
==37196==ABORTING


FWIW

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 31 Oct 2020 07:25:46 PM UTC, comment #62: 

I tried to run the program that crashes in gdb. But it looks like it succeeds if I do so:

$ gdb render.exe
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from render.exe...
(gdb) r
Starting program: D:\SVN\Octave\bug58957_sparsersb\rsbbugs-0.0\render.exe
[New Thread 11636.0x4738]
[New Thread 11636.0x888]
[New Thread 11636.0x1690]
allocated 8 bytes to 0000000001592750 (in hex:0x8 bytes)
allocated 4 bytes to 00000000015927b0 (in hex:0x4 bytes)
allocated 4 bytes to 0000000001592810 (in hex:0x4 bytes)
allocated 10080 bytes to 0000000001fd5860 (in hex:0x2760 bytes)
allocated 288 bytes to 0000000001592870 (in hex:0x120 bytes)
allocated 8 bytes to 00000000015929e0 (in hex:0x8 bytes)
allocated 16 bytes to 0000000001592a40 (in hex:0x10 bytes)
allocated 8 bytes to 0000000001592aa0 (in hex:0x8 bytes)
[New Thread 11636.0x186c]
[New Thread 11636.0x3e48]
[New Thread 11636.0x2e0]
freeing 0000000001592a40
freeing   16 bytes at 0000000001592a40 (in hex:0x10 bytes)
allocated 8 bytes to 0000000001592a40 (in hex:0x8 bytes)
allocated 32 bytes to 0000000001fd9250 (in hex:0x20 bytes)
freeing 0000000001592aa0
freeing   8 bytes at 0000000001592aa0 (in hex:0x8 bytes)
freeing 00000000015929e0
freeing   8 bytes at 00000000015929e0 (in hex:0x8 bytes)
freeing 0000000001592a40
freeing   8 bytes at 0000000001592a40 (in hex:0x8 bytes)
freeing 0000000001592870
freeing   288 bytes at 0000000001592870 (in hex:0x120 bytes)
allocated 4 bytes to 0000000001592930 (in hex:0x4 bytes)
allocated 4 bytes to 0000000001592990 (in hex:0x4 bytes)
allocated 8 bytes to 00000000015929f0 (in hex:0x8 bytes)
reallocating from 4 to 4 bytes (+0) at 0x0000000001592930 (in hex: to 0x4 bytes)
reallocating from 4 to 4 bytes (+0) at 0x0000000001592990 (in hex: to 0x4 bytes)
reallocating from 8 to 8 bytes (+0) at 0x00000000015929f0 (in hex: to 0x8 bytes)
freeing 0000000001592930
freeing   4 bytes at 0000000001592930 (in hex:0x4 bytes)
freeing 0000000001592990
freeing   4 bytes at 0000000001592990 (in hex:0x4 bytes)
freeing 00000000015929f0
freeing   8 bytes at 00000000015929f0 (in hex:0x8 bytes)
allocated 4 bytes to 0000000001592930 (in hex:0x4 bytes)
allocated 4 bytes to 0000000001592990 (in hex:0x4 bytes)
allocated 8 bytes to 00000000015929f0 (in hex:0x8 bytes)
%%MatrixMarket matrix coordinate real general
1 1 1
1       1       1
freeing 0000000001592930
freeing   4 bytes at 0000000001592930 (in hex:0x4 bytes)
freeing 0000000001592990
freeing   4 bytes at 0000000001592990 (in hex:0x4 bytes)
freeing 00000000015929f0
freeing   8 bytes at 00000000015929f0 (in hex:0x8 bytes)
freeing 0000000001fd9250
freeing   32 bytes at 0000000001fd9250 (in hex:0x20 bytes)
freeing 0000000001fd5860
freeing   10080 bytes at 0000000001fd5860 (in hex:0x2760 bytes)
freeing 00000000015927b0
freeing   4 bytes at 00000000015927b0 (in hex:0x4 bytes)
freeing 0000000001592810
freeing   4 bytes at 0000000001592810 (in hex:0x4 bytes)
freeing 0000000001592750
freeing   8 bytes at 0000000001592750 (in hex:0x8 bytes)
OK
[Thread 11636.0x3e48 exited with code 0]
[Thread 11636.0x186c exited with code 0]
[Thread 11636.0x2e0 exited with code 0]
[Thread 11636.0x1690 exited with code 0]
[Thread 11636.0x4738 exited with code 0]
[Thread 11636.0x888 exited with code 0]
[Inferior 1 (process 11636) exited normally]
(gdb)


If I quit gdb and immediately execute render.exe without gdb, it crashes with an output that is equivalent to that in comment #59 and with the same error code.


Not sure if this is of any relevance: RSB_DISABLE_ALLOCATOR_WRAPPER is 1 in rsb-config.h.

Markus Mützel <mmuetzel>
Group administrator
Sat 31 Oct 2020 06:26:42 PM UTC, comment #61: 

I pushed the temporary patch to MXE Octave here:
https://hg.octave.org/mxe-octave/rev/deca3045c118

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Oct 2020 04:59:49 PM UTC, comment #60: 

It crashes I see.
So it seems we have a well-defined reproducer.
Good -- thanks Markus.
Now we would have to figure out why it crashes!
At least, this seems to exclude faults on Octave's side ;-)_

dezperado <dezperado>
Fri 30 Oct 2020 04:31:54 PM UTC, comment #59: 

I built render.exe in the msys2 environment that comes with Octave for Windows (built with MXE Octave). The librsb library was built with the changes we made a few week ago.
This is the output when executing the file:

$ ./render.exe
allocated 8 bytes to 0000000001642f80 (in hex:0x8 bytes)
allocated 4 bytes to 0000000001642fb0 (in hex:0x4 bytes)
allocated 4 bytes to 0000000001642fe0 (in hex:0x4 bytes)
allocated 10080 bytes to 0000000001647cc0 (in hex:0x2760 bytes)
allocated 288 bytes to 0000000001643010 (in hex:0x120 bytes)
allocated 8 bytes to 0000000001643160 (in hex:0x8 bytes)
allocated 16 bytes to 0000000001643190 (in hex:0x10 bytes)
allocated 8 bytes to 00000000016431d0 (in hex:0x8 bytes)
freeing 0000000001643190
freeing   16 bytes at 0000000001643190 (in hex:0x10 bytes)
allocated 8 bytes to 000000000164ad90 (in hex:0x8 bytes)
allocated 32 bytes to 0000000001e03a40 (in hex:0x20 bytes)
freeing 00000000016431d0
freeing   8 bytes at 00000000016431d0 (in hex:0x8 bytes)
freeing 0000000001643160
freeing   8 bytes at 0000000001643160 (in hex:0x8 bytes)
freeing 000000000164ad90
freeing   8 bytes at 000000000164ad90 (in hex:0x8 bytes)
freeing 0000000001643010
freeing   288 bytes at 0000000001643010 (in hex:0x120 bytes)
allocated 4 bytes to 000000000164abb0 (in hex:0x4 bytes)
allocated 4 bytes to 000000000164abe0 (in hex:0x4 bytes)
allocated 8 bytes to 000000000164ad00 (in hex:0x8 bytes)
reallocating from 4 to 4 bytes (+0) at 0x000000000164abb0 (in hex: to 0x4 bytes)
reallocating from 4 to 4 bytes (+0) at 0x000000000164abe0 (in hex: to 0x4 bytes)
reallocating from 8 to 8 bytes (+0) at 0x000000000164ad00 (in hex: to 0x8 bytes)
freeing 000000000164abb0
freeing   4 bytes at 000000000164abb0 (in hex:0x4 bytes)

$ echo $?
127


I'm not sure which code I should paste as it is when invoking sparsersb("render.. ).

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Oct 2020 02:32:42 PM UTC, comment #58: 

Hi Markus.

WRT comment # 57: if you're comfortable with C, C++ and gdb, that would be ideal: I attach a tarball with a C++ program driving librsb very near to what crashes sparsersb.

You can use it in two ways:

  • run it standalone to see if it crashes in the Windows environment where sparsersb crashes
  • paste the code as it is when invoking sparsersb("render.. ), to see if it crashes there.

The above is only a starting point for now.


WRT comment #56: if that serves the short-term problem of having MXE-Octave consistent, yes, please, thanks.


(file #50170)

dezperado <dezperado>
Fri 30 Oct 2020 01:17:54 PM UTC, comment #57: 

Sorry. I crossed with your comment #55.

There might be a misunderstanding: I didn't want to offer taking over this bug completely. I wanted to offer my help in a temporary solution for Octave's installer for Windows.

Of course, I can continue to run tests for you. But there will probably be little more I can do.

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Oct 2020 01:13:39 PM UTC, comment #56: 

I realize now that what I asked for in comment #54 is what you proposed in comment #52.
In this case: I completely agree that this would be a feasible way.

I went ahead and tested with the attached patch. With it, `pkg test sparsersb` no longer crashes Octave.
If you agree, I'll add that patch to MXE Octave.

(file #50168)

Markus Mützel <mmuetzel>
Group administrator
Fri 30 Oct 2020 01:09:01 PM UTC, comment #55: 

Hi Markus,

> Would it instead (or additionally) be possible for the function to emit an error if the second input argument is "render"?

If you guys give me the #ifdef-based recipe to do that, I'd do it or let's to it straight away.

> I understand if you don't want to make that change in your released version because it seems to be a Windows-only bug.

Seems so, but one shall dig further, and I appreciate your help.

> But we could apply a changeset to your sources in MXE Octave.

It would be best if you could provide a changeset that applies on top of the latest released version.
Sure.

> Please, let me know if you don't agree.

I think a practical short-term solution would be to either remove the test or even block the 'render' demos or the render functionality on windows.

> If you don't want to spend your time on this, please let me now as well, and I can try and have a look myself.

Hmmm yes but we need a small reproducer program. I can write that relatively easily, but you would have to compile and run it under the Windows environment where sparsersb crashes. And you would follow it with gdb. In the best (clearest) case it will crash. But it might not. Is that OK with you ? If yes, I take an hour aside and I prepare that reproducer. 


In all this, in let me remark again:


That "render" functionality is only to get the "recursive sparse blocks" rendering of matrices represented in sparsersb. Its only use is to produce EPS plots to display how sparsersb arranges submatrices. I use it when I do presentation of sparsersb/librsb. Or sure, to debug librsb or sparsersb.

The 99.9% of octave-sparsersb users would only use it in order to squeeze multicore performance when multiplying sparse matrices by vectors or multi-vectors.
And if somebody is not getting speedup by using sparsersb (because e.g. using too small a matrix), than I see no reason of using sparsersb.

dezperado <dezperado>
Fri 30 Oct 2020 12:10:22 PM UTC, comment #54: 

Removing the tests doesn't prevent the function from crashing Octave.
Would it instead (or additionally) be possible for the function to emit an error if the second input argument is "render"?

I understand if you don't want to make that change in your released version because it seems to be a Windows-only bug.
But we could apply a changeset to your sources in MXE Octave.

It would be best if you could provide a changeset that applies on top of the latest released version.
Please, let me know if you don't agree. If you don't want to spend your time on this, please let me now as well, and I can try and have a look myself.

Markus Mützel <mmuetzel>
Group administrator
Thu 29 Oct 2020 09:31:26 AM UTC, comment #53: 

Agree to this compromise.  At least running "test sparsersb.cc" (file #50149) from MS Windows 10 Home (2004) with Octave 6.0.92 works

PASSES 992 out of 992 tests


--- sparsersb-1.0.8/src/sparsersb.cc    2020-08-11 23:54:00.000000000 +0900
+++ sparsersb.cc        2020-10-29 18:15:13.142776011 +0900
@@ -3403,13 +3403,6 @@
 %! [S, NROWS] = sparsersb("sparsersb_temporary_matrix_file.mtx", "D");
 %! assert(NROWS==2);
 %!test
-%! rrm=sparsersb(sprand(1000,1000,0.001));
-%! sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024); # will use defaults for rWidth
-%!test
-%! rrm=sparsersb(sprand(1000,1000,0.001));
-%! sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024,1024);
-%! # sparsersb(rrm,"renderb", "sparsersb_temporary_renderb.eps"); sparsersb(rrm,"renders", "sparsersb_temporary_renders.eps"); # FIXME
-%!test
 %! a=sparsersb(sprand(100,100,0.4));
 %! sparsersb(a,"autotune");
 %!test
@@ -5705,11 +5698,4 @@
 %! endif
 %! disp("")
 %! endfor
-
-%!demo
-%! disp("'sparsersb' can render sparse matrices into Encapsulated Postscript files showing the RSB blocks layout.")
-%! rm = sparsersb(sprand(100000,100000,.0001));
-%! sparsersb(rm,'render','sptest.eps')
-%! disp("You can open sptest.eps now.")
-%%!demo
 */


After changing "sparsersb.cc-tst" in my installation manually, I can run the test suite without crash


pkg test sparsersb
...
PASS   994
FAIL     7


The failures should be fixed in the next release.

@John and Markus: Are you okay with this fix?

Kai Torben Ohlhus <siko1056>
Group Member
Wed 28 Oct 2020 07:48:04 AM UTC, comment #52: 

Hi Kai & co.

I've no time at the moment to investigate the issue (windows octave+sparsersb crashing when rendering limit case matrices).

However, the scope of sparsersb is not rendering tiny sparse matrices ( sparsersb(matrix,"render.. ), but computing with sparse matrices.

As long as the test suite passes the non-'render' tests, the scope of the package is fulfilled.

May I ask John D. and Markus:
 If in sparsersb.cc you comment out the demos and tests using "render", does the test suite passes all the other tests on Windows ?
(you can use the sparsersb.cc I am attaching)

If this is the case, we turn off functionality on Windows and declare it broken until we fix it.

Makes sense ?

(file #50149)

dezperado <dezperado>
Wed 28 Oct 2020 03:25:56 AM UTC, comment #51: 

How is the state of this issue?

Yesterday in the developer meeting (https://wiki.octave.org/Online_Developer_Meeting_(2020-10-27)) it was suggested to remove erroneous packages from the Octave 6 release to avoid further release blockers.

Kai Torben Ohlhus <siko1056>
Group Member
Wed 07 Oct 2020 07:39:10 PM UTC, comment #50: 

Thank you for the renewed input, Markus.
I observe the last free() happens on a pointer to memory which has been reallocated -- that was related to bugs in the past. This is a hint further for me to investigate! Thx, M.

dezperado <dezperado>
Wed 07 Oct 2020 01:41:05 PM UTC, comment #49: 

I re-compiled with the local changes we made last week.

This is the output before the crash:

octave:1> pkg load sparsersb
octave:2> for i=1:1000; sparsersb(sparsersb([1]),"render", "crash.eps" ,1024); endfor
allocated 16 bytes to 0000000020db6e80 (in hex:0x10 bytes)
allocated 8 bytes to 0000000020e5e080 (in hex:0x8 bytes)
allocated 8 bytes to 0000000020e5de10 (in hex:0x8 bytes)
allocated 4 bytes to 0000000020e5e110 (in hex:0x4 bytes)
freeing 0000000020e5e110
freeing   4 bytes at 0000000020e5e110 (in hex:0x4 bytes)
allocated 10080 bytes to 0000000020ec9290 (in hex:0x2760 bytes)
allocated 288 bytes to 000000002008e460 (in hex:0x120 bytes)
allocated 8 bytes to 0000000020e5e110 (in hex:0x8 bytes)
allocated 16 bytes to 0000000020db6780 (in hex:0x10 bytes)
allocated 8 bytes to 0000000020e5d5d0 (in hex:0x8 bytes)
freeing 0000000020db6780
freeing   16 bytes at 0000000020db6780 (in hex:0x10 bytes)
allocated 8 bytes to 0000000020e5d7b0 (in hex:0x8 bytes)
allocated 32 bytes to 0000000020e84770 (in hex:0x20 bytes)
freeing 0000000020e5d5d0
freeing   8 bytes at 0000000020e5d5d0 (in hex:0x8 bytes)
freeing 0000000020e5e110
freeing   8 bytes at 0000000020e5e110 (in hex:0x8 bytes)
freeing 0000000020e5d7b0
freeing   8 bytes at 0000000020e5d7b0 (in hex:0x8 bytes)
freeing 000000002008e460
freeing   288 bytes at 000000002008e460 (in hex:0x120 bytes)
allocated 4 bytes to 0000000020e5e110 (in hex:0x4 bytes)
allocated 4 bytes to 0000000020e5da20 (in hex:0x4 bytes)
allocated 8 bytes to 0000000020e5d5d0 (in hex:0x8 bytes)
reallocating from 4 to 4 bytes (+0) at 0x0000000020e5e110 (in hex: to 0x4 bytes)
reallocating from 4 to 4 bytes (+0) at 0x0000000020e5da20 (in hex: to 0x4 bytes)
reallocating from 8 to 8 bytes (+0) at 0x0000000020e5d5d0 (in hex: to 0x8 bytes)
freeing 0000000020e5e110
freeing   4 bytes at 0000000020e5e110 (in hex:0x4 bytes)


Markus Mützel <mmuetzel>
Group administrator
Wed 23 Sep 2020 06:10:55 PM UTC, comment #48: 

yes - just calling it once

John Donoghue <lostbard>
Group Member
Wed 23 Sep 2020 03:58:22 PM UTC, comment #47: 

Please don't use that line, but repeat the test with:

> for i=1:1000; sparsersb(sparsersb([1]),"render", "crash.eps" ,1024); endfor


that is more deterministic than using sprand (let's put that aside). Does it crash now ?

dezperado <dezperado>
Wed 23 Sep 2020 03:34:09 PM UTC, comment #46: 

After adding in the pkg load:

> pkg load sparsersb

SPARSERSB.INIT

>> rrm=sparsersb(sprand(1000,1000,0.001));


>> sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024);


* crash **


(gdb) where
#0  0x00007ff92fa591b3 in ntdll!RtlIsNonEmptyDirectoryReparsePointAllowed ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ff92fa615e2 in ntdll!RtlpNtMakeTemporaryKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x00007ff92fa618ea in ntdll!RtlpNtMakeTemporaryKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#3  0x00007ff92fa6a8a9 in ntdll!RtlpNtMakeTemporaryKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#4  0x00007ff92f9a080d in ntdll!RtlGetCurrentServiceSessionId ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#5  0x00007ff92f99fb91 in ntdll!RtlFreeHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#6  0x00007ff92efd9cfc in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
#7  0x00007ff92f01a0f8 in ungetwc () from C:\WINDOWS\System32\msvcrt.dll
#8  0x00007ff92f00cb19 in msvcrt!clearerr_s ()
   from C:\WINDOWS\System32\msvcrt.dll
#9  0x00007ff92f00cbc0 in msvcrt!fclose () from C:\WINDOWS\System32\msvcrt.dll
#10 0x0000000032e07b58 in rsb__do_mtx_render (
    filename=0x2e3c3550 "sparsersb_temporary_render.eps", mtxAp=0x2f521070,
    pmWidth=pmWidth@entry=1024, pmHeight=pmHeight@entry=512, rflags=48)
    at rsb_eps.c:1111
#11 0x0000000032b1b4c5 in rsb_mtx_rndr (filename=<optimized out>,
    mtxAp=<optimized out>, pmWidth=pmWidth@entry=1024,
    pmHeight=pmHeight@entry=512, rflags=rflags@entry=48) at rsb_rsb.c:1317


John Donoghue <lostbard>
Group Member
Wed 23 Sep 2020 12:41:47 PM UTC, comment #45: 

Thanks John. Can you please try it out under Windows using file #49851 as a basis, to check if this solves the crash ?

dezperado <dezperado>
Wed 23 Sep 2020 12:23:07 PM UTC, comment #44: 

You need something thatwill call the init function, for init fltk its handled by the graphics toolkit manager.

You can do a similar load on pkg load by placing PKG_ADD statements in the sources.

In addition since there are now multiple octave callable functions in the .oct file, you will need to add autlod statements for each in the source.

// PKG_ADD: autoload ("__init_sparsersb__", "sparsersb.oct");
// PKG_ADD: autoload ("sparsersb", "sparsersb.oct");
// PKG_ADD: _init_sparsersb_();

That will create a PKG_ADD file when the package gets installed that will be run when the pkg is loaded.




John Donoghue <lostbard>
Group Member
Wed 23 Sep 2020 09:13:23 AM UTC, comment #43: 

I attach a sparsersb.cc with added mlock init as:


2701 static octave_value_list
2702 F__librsb_check__ (octave::interpreter& interp, const octave_value_list&, int)
2703 {
2704   // empty for now.
2705   return octave_value_list ();
2706 }
2707
2708 DEFMETHOD_DLD (__init_sparsersb__ , interp, , ,
2709                " -*- texinfo -*-\n\
2710 @deftypefn {} {} __init_sparsersb__ ()\n\
2711 Undocumented internal function.\n\
2712 @end deftypefn ")
2713 {
2714   if (! rsboi_sparse_rsb_loaded )
2715     {
2716       interp.mlock ();
2717
2718       install_sparse_rsb();
2719       std::cout << "SPARSERSB.INIT\n";
2720     }
2721
2722   return octave_value_list ();
2723 }


On my Linux box, I must say _init_sparsersb_ seems not to trigger.
Please let me know if it makes sense on the Window build at least.

(file #49851)

dezperado <dezperado>
Wed 23 Sep 2020 07:13:41 AM UTC, comment #42: 

sparsersb by itself will initialize librsb once, and the install_sparse_rsb invocation at each sparsersb() invocation is no performance cost.

But if on Windows Octave might sweep sparsersb status out of memory in between sparsersb calls, this is surely a problem.

You are suggesting mimicking _init_fltk_ . So I looked at it, and I'm trying creating a _init_sparsersb_, but I find a difficulty with 'doc:' and I'm not enough into Octave internals to figure it out quickly -- do you have a suggestion ? Please see

+verbatim
2718 DEFMETHOD_DLD (_init_sparsersb_ , interp, , ,                                                                                                                                                                                         
2719                doc: /* -*- texinfo -*-
2720 @deftypefn {} {} _init_sparsersb_ ()
2721 Undocumented internal function.
2722 @end deftypefn */)
-verbatim
in the attached sources.



(file #49848)

dezperado <dezperado>
Wed 23 Sep 2020 06:32:14 AM UTC, comment #41: 

Not sure if I got this right. But instead of repeatedly initializing and freeing rsb, it might be easier (and more performant) to lock the necessary function(s) in memory.
See e.g. here for how it is done in `__init_fltk__` for the `fltk_graphics_toolkit`:
https://hg.savannah.gnu.org/hgweb/octave/file/39997caa42f6/libinterp/dldfcn/__init_fltk__.cc#l2511

Markus Mützel <mmuetzel>
Group administrator
Wed 23 Sep 2020 06:20:44 AM UTC, comment #40: 

Yes: calling `rsb_mtx_rndr` without prior `rsb_lib_init` is a consistent recipe for crash!

sparsersb.oct expects one initialization only via rsb_lib_init.
Or at least, no re-calling of rsb_lib_init unless memory is being freed (e.g. matrices destroyed) and rsb_lib_exit called before.

Depending on the librsb configure options (memory debugging ones), it might be easier to spot that (at a cost, though).

dezperado <dezperado>
Wed 23 Sep 2020 06:08:44 AM UTC, comment #39: 

Since you mentioned "rsb_lib_init": IIRC, there was an issue with one of the .oct files in Octave core that lead to similar crashes on Windows because the module stack was lost between calls (or something like that...).
That was solved by `mlock`ing that file.

Could that be causing the issues here, too? I.e., could (effectively) calling `rsb_mtx_rndr` without prior `rsb_lib_init` cause such a crash?

Markus Mützel <mmuetzel>
Group administrator
Tue 22 Sep 2020 10:55:02 PM UTC, comment #38: 

Hi Markus.

Well, you did the test I asked you: compiling and running that program; thanks.

For coherence with what you observed, most of that program's main() block (all except for the rsb_lib_init() line) shall be substitutible to the rsb_mtx_rndr() call in sparsersb.cc, and you should expect the same crash. And if not, that may hint to a possible innocence of rsb_mtx_rndr ..

dezperado <dezperado>
Tue 22 Sep 2020 04:59:38 PM UTC, comment #37: 

Sorry, I'm not sure I understand what I am supposed to do.
I cross-built Octave with all its dependencies (including librsb) for Windows on Linux with MXE Octave.
MXE Octave cleans up the build directories of each package it has built. I no longer have a librsb build directory.
Instead, I have an msys2 environment with installed gcc and librsb. That is the same librsb that Octave is using.
So, I ran the following at the msys2 shell:

$ gcc rsb_mtx_rndr.c -otest.exe -lrsb -fopenmp
$ gdb test.exe
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from test.exe...
(gdb) r
Starting program: D:\SVN\Octave\bug58957\test.exe
[New Thread 23092.0x2fe4]
[New Thread 23092.0x590]
[New Thread 23092.0x4d50]
[New Thread 23092.0x3f74]
[New Thread 23092.0x15dc]
[New Thread 23092.0xde4]
OK
[Thread 23092.0xde4 exited with code 0]
[Thread 23092.0x15dc exited with code 0]


Is this a meaningful test? In case it is: The test program didn't crash.
I ran the program about 20-30 times but it never crashed.

I am attaching the "crash.eps" file that was created by that program in case that helps.

One difference is that this test program was built natively on Windows. "sparsersb" was cross-built on Linux for Windows.
To test if that makes a difference, I re-built and re-installed sparsersb on Windows. In Octave:

>> pkg install -forge sparsersb
warning: LFLAGS is deprecated and will be removed in a future version of Octave, use LDFLAGS instead
For information about changes from previous versions of the sparsersb package, run 'news sparsersb'.
>> pkg load sparsersb
>> pkg list
[...]
          sparsersb *|   1.0.8 | C:\Users\Markus\octave\sparsersb-1.0.8
[...]
>> sparsersb(sparsersb([1]),"render", "crash.eps" ,1024);


But Octave crashed on the last command.

(file #49838)

Markus Mützel <mmuetzel>
Group administrator
Mon 21 Sep 2020 07:14:56 PM UTC, comment #36: 

Hi Markus,

Ok, let's continue investigating with the 64-bit build.

I am attaching a program mimicking the
 sparsersb(sparsersb([1]),"render", "crash.eps" ,1024);
crashing sequence.

Could you please substitute it to that of examples/transpose.c in a configured librsb build directory, then  re-run 'make', and then run ./transpose to check if it crashes ?




(file #49834)

dezperado <dezperado>
Mon 21 Sep 2020 03:46:09 PM UTC, comment #35: 

With a 32bit build, the loop completed once without issues. Running it multiple times crashed Octave (alongside the attached gdb). No backtrace for that unfortunately.
But I'm starting to get the impression that something unrelated is off with the 32bit Windows build anyway.

So maybe we should focus on the 64bit build first.
With it, Octave crashes on the first run of the loop.
Backtrace from gdb:

warning: Critical error detected c0000374

Thread 26 received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 6520.0x4d34]
0x00007ffb33b2ed03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
(gdb) bt
#0  0x00007ffb33b2ed03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ffb33b37ae2 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x00007ffb33b37dca in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#3  0x00007ffb33b3d7f1 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#4  0x00007ffb33a49640 in ntdll!RtlAllocateHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#5  0x00007ffb33a45d21 in ntdll!RtlFreeHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#6  0x00007ffb31bc9c9c in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
#7  0x0000000086d56fae in rsb__free (p=0x5ff76560) at rsb_sys.c:389
#8  0x0000000086a7398b in rsb__destroy_coo_matrix_t (cmp=0x5cd8b1d0)
    at rsb_internals.c:247
#9  0x0000000086d56998 in rsb_dump_postscript_from_mtx_t (br=1, bc=1,
    all_nnz=1, height=512, width=1024, mtxAp=0x5f8569c0,
    fd=0x7ffb31c3fb20 <msvcrt!_iob+288>) at rsb_eps.c:959
#10 rsb__do_mtx_render (filename=0x5cd8b3f0 "crash.eps", mtxAp=0x5f8569c0,
    pmWidth=1024, pmHeight=512, rflags=48) at rsb_eps.c:1101
#11 0x0000000086a67578 in Fsparsersb (args=..., nargout=0)
    at sparsersb.cc:2926
#12 0x000000000127aa77 in octave_builtin::execute (this=0x5e4a4e70, tw=...,
--Type <RET> for more, q to quit, c to continue without paging--c
    nargout=0, args=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-builtin.cc:59
#13 0x00000000012bb9c4 in octave_function::call (this=0x5e4a4e70, tw=..., nargout=0, args=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-fcn.cc:57
#14 0x00000000013b7af1 in octave::tree_index_expression::evaluate_n (this=0x840626f0, tw=..., nargout=0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-idx.cc:526
#15 0x0000000001d5ffdf in octave::tree_index_expression::evaluate (this=<optimized out>, tw=..., nargout=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-idx.h:109
#16 0x00000000013ac760 in octave::tree_evaluator::visit_statement (this=0x5f4850c0, stmt=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3239
#17 0x000000000139af2c in octave::tree_statement::accept (tw=..., this=0x5c129fd0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:124
#18 octave::tree_evaluator::visit_statement_list (this=0x5f4850c0, lst=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3298
#19 0x00000000013abcca in octave::tree_statement_list::accept (tw=..., this=0x82011ad0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:201
#20 octave::tree_evaluator::execute_range_loop<double> (loop_body=0x82011ad0, ult=..., line=1, rng=..., this=0x5f4850c0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:2516
#21 octave::tree_evaluator::visit_simple_for_command (this=0x5f4850c0, cmd=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:2566
#22 0x00000000013ac6b0 in octave::tree_evaluator::visit_statement (this=0x5f4850c0, stmt=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3216
#23 0x000000000139af2c in octave::tree_statement::accept (tw=..., this=0x5c12a090) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:124
#24 octave::tree_evaluator::visit_statement_list (this=0x5f4850c0, lst=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3298
#25 0x000000000139b49a in octave::tree_statement_list::accept (tw=..., this=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:201
#26 octave::tree_evaluator::eval (this=0x5f4850c0, stmt_list=..., interactive=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:403
#27 0x00000000016d24f0 in octave::interpreter::main_loop (this=this@entry=0x5f484360) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/corefcn/interpreter.cc:1278
#28 0x00000000016d6baa in octave::interpreter::execute (this=this@entry=0x5f484360) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/corefcn/interpreter.cc:806
#29 0x000000003a402c78 in octave::interpreter_qobject::execute (this=0x59064af0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libgui/src/interpreter-qobject.cc:87
#30 0x000000003a4aaf5d in octave::interpreter_qobject::qt_static_metacall (_o=0x137156804d730000, _id=<optimized out>, _a=0x5f8569c0, _c=<optimized out>) at libgui/src/moc-interpreter-qobject.cc:96
#31 0x0000000066945c16 in QObject::event(QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#32 0x00000000578b7953 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Widgets.dll
#33 0x00000000578c058e in QApplication::notify(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Widgets.dll
#34 0x000000003a450d66 in octave::octave_qapplication::notify (this=0x59079ca0, receiver=<optimized out>, ev=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libgui/src/octave-qobject.cc:132
#35 0x000000006691ac8a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#36 0x0000000066921561 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#37 0x0000000066974d5f in QEventDispatcherWin32::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#38 0x0000000066919963 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#39 0x0000000066758ef0 in QThread::run() () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#40 0x0000000066759ae3 in QThread::setTerminationEnabled(bool) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#41 0x00007ffb33116fd4 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
#42 0x00007ffb33a7cec1 in ntdll!RtlUserThreadStart () from C:\WINDOWS\SYSTEM32\ntdll.dll
#43 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)


librsb-1.2.0.9 was configured with:

autoreconf -fi && ./configure --host='x86_64-w64-mingw32' --build='x86_64-pc-linux-gnu' --prefix='/home/osboxes/Documents/Repositories/Octave/mxe-octave/usr/x86_64-w64-mingw32' --enable-shared --disable-static CPPFLAGS='-I/home/osboxes/Documents/Repositories/Octave/mxe-octave/usr/x86_64-w64-mingw32/include' LDFLAGS='-L/home/osboxes/Documents/Repositories/Octave/mxe-octave/usr/x86_64-w64-mingw32/lib' --disable-c-examples --disable-fortran-examples --disable-sparse-blas-interface --disable-octave-testing


The full configure and build log of librsb is attached in case that helps.

(file #49830)

Markus Mützel <mmuetzel>
Group administrator
Mon 21 Sep 2020 02:10:52 PM UTC, comment #34: 

Thanks Markus.

What about


for i=1:1000; sparsersb(sparsersb([1]),"render", "crash.eps" ,1024); endfor


 -- does it crash on any of the 32 or 64 bit builds you used in comment #33 ?

BTW, if bug #58790 is triggered by memory allocation failures, then I find it difficult to occur in the cases here, which use minimal amounts of memory...

p.s. I'm back at a keyboard after a few weeks off.

dezperado <dezperado>
Sun 13 Sep 2020 12:33:51 PM UTC, comment #33: 

Thanks.

Tried with a Windows 32bit build with the new versions changes (Octave hg id 08049eff8733 and MXE Octave hg id b982f2df58cf).
That means the tests are done with of-sparsersb-1.0.8 and librsb-1.2.0.9:

The code from comment #0 doesn't crash Octave. Not even if I call it 1000 times in a loop.

"pkg test sparsersb" crashes Octave without gdb being able to catch anything. That might be bug #58790 striking again.

So I tried again with the same sources but a Windows 64bit build.
Running "pkg test sparsersb" still crashes Octave. But ´gdb´ is able to catch the error.
Part of the gdb session with top of the backtrace:

warning: Critical error detected c0000374

Thread 25 received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 17608.0xcb4]
0x00007ff806cced03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
(gdb) bt
#0  0x00007ff806cced03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ff806cd7ae2 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x00007ff806cd7dca in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#3  0x00007ff806cdd7f1 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#4  0x00007ff806be9640 in ntdll!RtlAllocateHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#5  0x00007ff806be5d21 in ntdll!RtlFreeHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#6  0x00007ff806b09c9c in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
#7  0x0000000086516fae in rsb__free (p=0x593a2de0) at rsb_sys.c:389
#8  0x000000008623398b in rsb__destroy_coo_matrix_t (cmp=0x5ac56540)
    at rsb_internals.c:247
#9  0x0000000086516998 in rsb_dump_postscript_from_mtx_t (br=1, bc=1,
    all_nnz=1, height=512, width=1024, mtxAp=0x604027d0,
    fd=0x7ff806b7fb50 <msvcrt!_iob+336>) at rsb_eps.c:959
#10 rsb__do_mtx_render (filename=0x84a1e0a0 "sparsersb_temporary_render.eps",
    mtxAp=0x604027d0, pmWidth=1024, pmHeight=512, rflags=48) at rsb_eps.c:1101
#11 0x0000000086227578 in Fsparsersb (args=..., nargout=0)
    at sparsersb.cc:2926
#12 0x000000000126aa77 in octave_builtin::execute (this=0x5770e010, tw=...,
--Type <RET> for more, q to quit, c to continue without paging--c
    nargout=0, args=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-builtin.cc:59
#13 0x00000000012ab9c4 in octave_function::call (this=0x5770e010, tw=..., nargout=0, args=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-fcn.cc:57


Same again for the example in comment #0:

warning: Critical error detected c0000374

Thread 25 received signal SIGTRAP, Trace/breakpoint trap.
[Switching to Thread 16084.0x3ecc]
0x00007ff806cced03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
(gdb) bt
#0  0x00007ff806cced03 in ntdll!RtlIsZeroMemory ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#1  0x00007ff806cd7ae2 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#2  0x00007ff806cd7dca in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#3  0x00007ff806cdd7f1 in ntdll!RtlpNtSetValueKey ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#4  0x00007ff806be9640 in ntdll!RtlAllocateHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#5  0x00007ff806be5d21 in ntdll!RtlFreeHeap ()
   from C:\WINDOWS\SYSTEM32\ntdll.dll
#6  0x00007ff806b09c9c in msvcrt!free () from C:\WINDOWS\System32\msvcrt.dll
#7  0x00007ff806b49ef0 in ungetwc () from C:\WINDOWS\System32\msvcrt.dll
#8  0x00007ff806b3c9c9 in msvcrt!clearerr_s ()
   from C:\WINDOWS\System32\msvcrt.dll
#9  0x00007ff806b3ca70 in msvcrt!fclose () from C:\WINDOWS\System32\msvcrt.dll
#10 0x0000000086516898 in rsb__do_mtx_render (
    filename=0x82240f90 "sparsersb_temporary_render.eps", mtxAp=0x60a7b010,
    pmWidth=1024, pmHeight=512, rflags=48) at rsb_eps.c:1111
#11 0x0000000086227578 in Fsparsersb (args=..., nargout=0)
    at sparsersb.cc:2926
#12 0x000000000119aa77 in octave_builtin::execute (this=0x57772a00, tw=...,
    nargout=0, args=...)
--Type <RET> for more, q to quit, c to continue without paging--c
    at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-builtin.cc:59
#13 0x00000000011db9c4 in octave_function::call (this=0x57772a00, tw=..., nargout=0, args=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/octave-value/ov-fcn.cc:57
#14 0x00000000012d7af1 in octave::tree_index_expression::evaluate_n (this=0x5fb30550, tw=..., nargout=0) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-idx.cc:526
#15 0x0000000001c7ffdf in octave::tree_index_expression::evaluate (this=<optimized out>, tw=..., nargout=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-idx.h:109
#16 0x00000000012cc760 in octave::tree_evaluator::visit_statement (this=0x5a7dc570, stmt=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3239
#17 0x00000000012baf2c in octave::tree_statement::accept (tw=..., this=0x822ef630) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:124
#18 octave::tree_evaluator::visit_statement_list (this=0x5a7dc570, lst=...) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:3298
#19 0x00000000012bb49a in octave::tree_statement_list::accept (tw=..., this=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-stmt.h:201
#20 octave::tree_evaluator::eval (this=0x5a7dc570, stmt_list=..., interactive=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/parse-tree/pt-eval.cc:403
#21 0x00000000015f24f0 in octave::interpreter::main_loop (this=this@entry=0x5a7db810) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/corefcn/interpreter.cc:1278
#22 0x00000000015f6baa in octave::interpreter::execute (this=this@entry=0x5a7db810) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libinterp/corefcn/interpreter.cc:806
#23 0x000000003bc82c78 in octave::interpreter_qobject::execute (this=0x575f9b00) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libgui/src/interpreter-qobject.cc:87
#24 0x000000003bd2af5d in octave::interpreter_qobject::qt_static_metacall (_o=0xb5869fedff1e0000, _id=<optimized out>, _a=0x8652860a <__mingw_vfprintf+58>, _c=<optimized out>) at libgui/src/moc-interpreter-qobject.cc:96
#25 0x0000000066945c16 in QObject::event(QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#26 0x000000005d4e7953 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Widgets.dll
#27 0x000000005d4f058e in QApplication::notify(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Widgets.dll
#28 0x000000003bcd0d66 in octave::octave_qapplication::notify (this=0x5760d370, receiver=<optimized out>, ev=<optimized out>) at /home/osboxes/Documents/Repositories/Octave/mxe-octave/tmp-default-octave/octave-7.0.0/libgui/src/octave-qobject.cc:132
#29 0x000000006691ac8a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#30 0x0000000066921561 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#31 0x0000000066974d5f in QEventDispatcherWin32::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#32 0x0000000066919963 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#33 0x0000000066758ef0 in QThread::run() () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#34 0x0000000066759ae3 in QThread::setTerminationEnabled(bool) () from D:\SVN\Octave\test\OC67B0~1\OCTAVE~1\mingw64\bin\Qt5Core.dll
#35 0x00007ff804d36fd4 in KERNEL32!BaseThreadInitThunk () from C:\WINDOWS\System32\kernel32.dll
#36 0x00007ff806c1cec1 in ntdll!RtlUserThreadStart () from C:\WINDOWS\SYSTEM32\ntdll.dll
#37 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)


So, it looks like the 32bit version might be ok. But the 64bit version crashes.

Markus Mützel <mmuetzel>
Group administrator
Sun 13 Sep 2020 12:14:03 AM UTC, comment #32: 

Pushed the new versions

John Donoghue <lostbard>
Group Member
Fri 11 Sep 2020 09:18:06 PM UTC, comment #31: 

I can push the update for sparser/libsrb
Although it appeared to be crashing in the way

John Donoghue <lostbard>
Group Member
Fri 11 Sep 2020 06:24:06 PM UTC, comment #30: 

I rebuilt Octave and dependencies without stripping symbols and got the backtraces for the crashes with "pkg test" and the "render" command from comment #0. They now include line numbers to the sources.
I hope these are more helpful.

This is still with librsb 1.2.0.8 and sparsersb 1.0.7.
Does it even make sense to test with these versions? Or should I better wait for a patch that updates MXE Octave to build newer versions?

(file #49787, file #49788)

Markus Mützel <mmuetzel>
Group administrator
Fri 11 Sep 2020 06:20:51 AM UTC, comment #29: 

The configure script of of-sparsersb seems to execute ´sizemax() > intmax ("int32")´ in Octave to check if octave_idx_type is 64bit:
https://sourceforge.net/p/octave/sparsersb/ci/default/tree/src/configure.ac (currently in line 129)

This is unreliable when cross-building for an incompatible platform (e.g. mingw binaries on a linux build system) because the native Octave might have been configured differently from the target.

Would it be possible to remove that test? Or at least to not set RSBOI_DETECTED_LONG_IDX when cross-compiling?

Markus Mützel <mmuetzel>
Group administrator
Thu 03 Sep 2020 06:28:56 AM UTC, comment #28: 

The backtrace was captured with an installation from the current release candidate of Octave 6.0.90 as available on alpha.gnu.org.
That is not a debug build. So the symbols have been stripped and compiler optimizations are active. (Please, correct me if that is wrong.)
The former means that there are no line numbers to the source files and that function names are mangled.
The latter means that the compiler might have optimized out some variables and might have inlined some functions.

Maybe the compiler inlined a function call in "rsb.do_mtx_render" which might explain a "missing" layer in the stack.

Markus Mützel <mmuetzel>
Group administrator
Wed 02 Sep 2020 10:38:45 PM UTC, comment #27: 

The without symbolds is just that they get stripped in the mxe octave version so are not available.

John Donoghue <lostbard>
Group Member
Wed 02 Sep 2020 10:38:02 PM UTC, comment #26: 

To be clear, the bug report was opened originally for the new version of sparser and librsb which I was in the process of verifying before setting in mxe.

In comment #22, i tried the old mxe-octave 5.2 that had the older version and it was crashing the same as it does in the 6.0.90 older version.

John Donoghue <lostbard>
Group Member
Wed 02 Sep 2020 10:35:16 PM UTC, comment #25: 


comment #23:

>
> comment #21:
> > This also seems to happen with the sparsersb included in Octave 6.0.90a.
> ...
>    why 1.0.7 ? Are you using 1.0.8 version of sparsersb?
> Version 1.0.7 is not meant to work with librsb-1.2.0.9.
> ...

Uh, I now understand you're using "the version that is there".

And you write "without the symbols" .. in the sense that the ones you provided in the stack trace are not to be trusted ?

dezperado <dezperado>
Wed 02 Sep 2020 10:34:00 PM UTC, comment #24: 

The sparser version in mxe is still the previous version (1.0.7), as is the  librsb (1.2.0.8)

But has the same error with the render

John Donoghue <lostbard>
Group Member
Wed 02 Sep 2020 10:24:40 PM UTC, comment #23: 


comment #21:

> This also seems to happen with the sparsersb included in Octave 6.0.90a.
>
> I'm attaching a backtrace from a crash that happened while running "pkg test sparsersb" and another one from the command in point 2) in comment #10.
> They are without symbols. But I hope, they are helpful anyway.
>
> (file #49730, file #49731)


Thanks Markus.
your stack trace is useful! But a bit strange:

1) In both logs the sparsersb's path seems to be
   "C:\Program Files\GNU Octave\Octave-6.0.90\mingw64\lib\octave\packages\sparsersb-1.0.7\x86_64-w64-mingw32-api-v55\sparsersb.oct"
   why 1.0.7 ? Are you using 1.0.8 version of sparsersb?
Version 1.0.7 is not meant to work with librsb-1.2.0.9.

2) In sparsersb_bt_pkg_test.log
 I see stack in the trace:
 sparsersb!rsb.do_util_merge_sorted_subarrays_in_place ()
 sparsersb!rsb__do_get_pixmap_RGB_from_matrix ()
 sparsersb!rsb__do_get_pixmap_RGB_from_matrix ()
 sparsersb!rsb.do_mtx_render ()

and that is very strange:rsb__do_get_pixmap_RGB_from_matrix shall not be there.
Indeed, sparsersb.cc calls rsb_mtx_rndr, which in turn calls  rsb__do_mtx_render.
But from there I've no idea why rsb__do_get_pixmap_RGB_from_matrix is there.. that function is being invoked by rsb_file_mtx_rndr, which sparsersb.cc does not invoke in any way.
Corrupt stack trace ?

3) In sparsersb_bt_render.log
 rsb.do_mtx_render seems to invoke rsb.destroy_coo_matrix_t
 This seems fine, in that rsb__do_mtx_render calls a static function which calls rsb__destroy_coo_matrix_t.

dezperado <dezperado>
Tue 01 Sep 2020 03:35:14 PM UTC, comment #22: 

I believe previous version of the packages didn't include the tests from the .cc files, which we now do which is where I saw it.

It happens in octave 5.2 as well - I hadnt checked versions before that.


John Donoghue <lostbard>
Group Member
Tue 01 Sep 2020 01:20:25 PM UTC, comment #21: 

This also seems to happen with the sparsersb included in Octave 6.0.90a.

I'm attaching a backtrace from a crash that happened while running "pkg test sparsersb" and another one from the command in point 2) in comment #10.
They are without symbols. But I hope, they are helpful anyway.

(file #49730, file #49731)

Markus Mützel <mmuetzel>
Group administrator
Thu 27 Aug 2020 05:45:22 PM UTC, comment #20: 

To test I compiled with #warnings added to the if and else to see which was displayed.

So I guess its it using 64bit indexes because of the  OCTAVE_ENABLE_64 event if it didnt set the RSBOI_DETECTED_LONG_IDX


John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 04:18:41 PM UTC, comment #19: 

It appears to be using #define RSBOI_O64_R32 1

John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 03:45:29 PM UTC, comment #18: 

Ok so you definitely need 64 bit indices in octave.

One more check you can do now is to locate in sparsersb.cc the lines:

#if defined(USE_64_BIT_IDX_T) || defined(OCTAVE_ENABLE_64) || defined(RSBOI_DETECTED_LONG_IDX) /* 4.1.0+ / 4.0.3 / any */
#define RSBOI_O64_R32 1                                                                                                      
#else /* USE_64_BIT_IDX_T */
#define RSBOI_O64_R32 0
#endif /* USE_64_BIT_IDX_T */

and discover which branch of the two is being compiled (first one is right, second wrong).

dezperado <dezperado>
Thu 27 Aug 2020 03:30:29 PM UTC, comment #17: 

#  define OCTAVE_ENABLE_64 1
#  define OCTAVE_SIZEOF_IDX_TYPE 8

John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 03:13:42 PM UTC, comment #16: 

Yes; and what are their values in your case ?

dezperado <dezperado>
Thu 27 Aug 2020 03:10:56 PM UTC, comment #15: 

looking at octave includes, octave has the following defines

OCTAVE_SIZEOF_IDX_TYPE

and

OCTAVE_ENABLE_64




John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 02:29:41 PM UTC, comment #14: 

Ok it would normally be calling the native linux octave at that point that may or may not be compiled with the same settings as the cross compiled octave.

I'll try a couple of things



John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 12:53:49 PM UTC, comment #13: 

Thanks John.

  • It seems you are compiling without -DRSBOI_DETECTED_LONG_IDX=1. So sparsersb believes Octave is using 32 bit indices. Is this really the case ? If not, can you somehow force it (even changing sparsersb.cc ) ?

 

  • Please forget 'test sparsersbtg' --- that's not really meant for users; I was asking for 'test sparsersb' -- that what matters.


dezperado <dezperado>
Thu 27 Aug 2020 12:08:54 PM UTC, comment #12: 

In win64, after commenting out the 2 render tests:

  PASS                              998
  FAIL                                1

The fail was


>>>>> processing AppData\Local\Programs\GNU Octave\Octave-6.0.1_1\mingw64\share\octave\packages\sparsersb-1.0.8\sparsersbtg.m
***** test
 assert( length( sparsersbtg () ) >= 57846 )
 assert( length( sparsersbtg ([1,1,1,1,1,1]) ) >= 11218 )
 assert( length( sparsersbtg ([1,1,1]) ) >= 11218 )
 assert( length( sparsersbtg ([1]) ) == 0 )
!!!!! test failed
invalid empty index expression



The cross compiled package flags:

octave sparsersb package is configured with:
  SPARSERSB_LDFLAGS:     -L/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/lib -lrsb
  SPARSERSB_CXXFLAGS:    -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/
  SPARSERSB_CXX11:       -std=gnu++11
  OCTAVE:                echo
  MKOCTFILE:             /home/jade/mxe-octave/build-64/usr/bin/x86_64-w64-mingw32-mkoctfile --verbose -g



The compile command:


x86_64-w64-mingw32-g++ -c -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include  -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/octave-6.0.1/octave/.. -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/octave-6.0.1/octave -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include   -fopenmp -I/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/include/ -DHAVE_OCTAVE_VALUE_ISCOMPLEX -std=gnu++11 -g    -DRSB_SPARSERSB_LABEL=sparsersb sparsersb.cc -o /tmp/oct-hVqkQj.o



librsb cross compiled flags:


configure: =============== Build Programs and Flags ===============================
(you can override these at build time; e.g.: 'make CC=cc')
        CC                     : x86_64-w64-mingw32-gcc
        CFLAGS                 : -g -O2 -D__USE_MINGW_ANSI_STDIO=1 -fopenmp -std=c99
        NOUNROLLCFLAGS         : -fno-unroll-loops
        FC (to disable, FC='') : x86_64-w64-mingw32-gfortran
        FCFLAGS                : -g -O2 -fopenmp
        LD                     : /home/jade/mxe-octave/build-64/usr/bin/x86_64-w64-mingw32-ld
        LDFLAGS                : -L/home/jade/mxe-octave/build-64/usr/x86_64-w64-mingw32/lib
        LIBS                   :  -lm -lgfortran
        AR                     : x86_64-w64-mingw32-ar
        ARFLAGS                : cru
        M4                     : m4
        OCTAVE                 : octave
        DOXYGEN                : false
        HELP2MAN               : false
=== Additional flags affecting only the benchmark program (rsbench): ===
        RSB_RSBENCH_LIBS             :
        RSB_RSBENCH_CFLAGS           :
========= Main code generator values, this build vs defaults ===========
(if these differ from the defaults, you need to have M4 and run 'make cleanall' and 'make')
        All Numerical types          : "int,double,float,float complex,double complex"
        Numerical types              : "double,float,float complex,double complex" vs "double,float,float complex,double complex"
        Build Sparse BLAS Interface  : "no" vs "yes"
        Util. Kernels Unroll         : "16" vs "16"
        Triangular solve zero check  : "yes" vs "yes"
============== Build Configuration, this build vs defaults =============
(if you reconfigure and change these, you need to run 'make clean' and 'make')
        Supported I/O functionality level   : "7" vs "0"
        Interface Error Verbosity           : "0" vs "0"
        Internals Error Verbosity           : "0" vs "0"
        Memory hierarchy info, detected     : "L3:16/64/8192K,L2:8/64/256K,L1:8/64/32K"
        Memory hierarchy info, selected     : ""
        Maximum of Supported threads        : "64"
        Build Fortran examples              : "no"
        Build C examples                    : "no"
        Install Sparse BLAS Fortran modules : "no"
        Install pkg-config "librsb.pc" file : "no"
        Build Octave generated tester       : "no"
        Build Octave generated tester (int) : "no"
        Build HTML and man documentation    : "no"
        gzipped matrices support via zlib   : "no"
        gather elapsed time in librsb       : "no"


John Donoghue <lostbard>
Group Member
Thu 27 Aug 2020 07:52:37 AM UTC, comment #11: 

I've a correction wrt my previous post at point 0; there I meant:

"If you comment out all the 'render' tests from sparsersb.cc, do all the other 900 tests pass ?"

dezperado <dezperado>
Tue 25 Aug 2020 04:31:02 PM UTC, comment #10: 

I have the fix for the crash on [0] or empty matrices.
But not for [1]..

0) I am assuming "test sparsersb" succeeds giving

 PASSES 994 out of 994 tests

, right ?

1) Can you check whether you compiled sparsersb.cc with

 -DRSBOI_DETECTED_LONG_IDX=1

?

2) Can you obtain a stack trace of

  sparsersb(sparsersb([1]),"render", "crash.eps" ,1024)

crashing ?

dezperado <dezperado>
Tue 25 Aug 2020 03:38:13 PM UTC, comment #9: 

yes

John Donoghue <lostbard>
Group Member
Tue 25 Aug 2020 12:45:19 PM UTC, comment #8: 

I can replicate this crashing:

 sparsersb(sparsersb([0]),"render", "crash.eps" ,1024)

but I cannot replicate this crashing:

 sparsersb(sparsersb([1]),"render", "crash.eps" ,1024)

I guess both of the above lines crash your instance, right ?

dezperado <dezperado>
Tue 25 Aug 2020 12:08:25 PM UTC, comment #7: 

Yes that crashes.

Content of crash.mtx file:


%%MatrixMarket matrix coordinate real general
23 23 1
16        10        0.2840275319617540


John Donoghue <lostbard>
Group Member
Mon 24 Aug 2020 04:13:12 PM UTC, comment #6: 

Strange.. empty matrices are not to be expected here!

Do you experience any crash if you run the following ?

for i=10:1000;
 rrm=sparsersb(sprand(i,i,0.001)); # <- increasing matrix size
 if nnz(rrm) > 0
   sparsersb(rrm,"save","crash.mtx");
   sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024);
 end
end


dezperado <dezperado>
Mon 17 Aug 2020 05:04:22 PM UTC, comment #5: 

Installing 1.0.7 and 1.0.6 in octave 6 and octave 5.2 shows the same issue for me, so i guess it may not be something new in windows.

John Donoghue <lostbard>
Group Member
Mon 17 Aug 2020 12:04:47 PM UTC, comment #4: 

Anything with an I value < 23 produces an invalid crash file, that when loading:

>> rrm=sparsersb("crash.mtx");

Input has missing elements on the diagonal.

file content:


%%MatrixMarket matrix coordinate real symmetric
10 10 0


Starting at I=30 seems to consistently crash.


Using rsbench doesn't seem to ever crash, however the .sps file generated doesn't plot any data, whereas the octave call (when it doesn't crash does)

I attached the eps file generated using a 1000x1000 mtx file in rsbench as well as the crash.mtx file that was used.





(file #49683, file #49684)

John Donoghue <lostbard>
Group Member
Mon 17 Aug 2020 09:08:30 AM UTC, comment #3: 

1)
Sorry I made a typo and pasted wrong code;
what I intended originally in the loop was:

for i=10:1000;
 rrm=sparsersb(sprand(i,i,0.001)); # <- increasing matrix size
 sparsersb(rrm,"save","crash.mtx");
 # if next step crashes, we at least get crash.mtx
 sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024);
end

Running the above shall allow you to obtain the more or less 'smallest' crashing matrix in "crash.mtx".
With that it would be interesting to see if there is a matrix that leads to a 100% of crashes.

2) There is a chance the bug is totally related to librsb.
To debunk that, you may try once or repeated:
 rsbench  --plot -f crash.mtx > crash.eps # does it render or crash ?
You can first use librsb's A.mtx, then the crash.mtx from point 1).

3) Repeat either of 1) and 2) with threading off, by running:
 export OMP_NUM_THREADS=1
before the test.

4) Rebuild sparsersb.oct from sparsersb.cc by commenting the call to rsb_mtx_rndr(); then invoke it with "render" as in 1). If this crashes sparsersb, tne rendering function can be declared innocent.

dezperado <dezperado>
Sun 16 Aug 2020 11:38:49 PM UTC, comment #2: 

It may be a windows only issue?

I ran the loop at it crashed.

On loading the matrix and doing the render worked ok, but repeating it a couple of times with the same file caused it to intermittently crash.

So its possibly something in the render not the matrix itself?

Even doing something to generate the matrix once only but render multiple times with cause the crash:



rrm=sparsersb(sprand(1000,1000,0.001));
for i=10:1000;
 sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024); end



When it crashes the rendered .eps file as a size of 0.

John Donoghue <lostbard>
Group Member
Sat 15 Aug 2020 11:18:42 PM UTC, comment #1: 

Thanks for the double problem report!

With respect to the crash, can you please

1) run a loop like:
 
for i=10:1000;
 rrm=sparsersb(sprand(1000,1000,0.001));
 sparsersb(rrm,"save","crash.mtx");
 sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024); end

so to save the matrix before triggering a crash and

2) after the crash, restart octave and

 rrm=sparsersb("crash.mtx") # load the matrix
 sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024);

to check if with that same matrix it crashes again ?

3) If you are able to find such a "crash-inducing" matrix, can you attach it in this forum ?

dezperado <dezperado>
Fri 14 Aug 2020 06:32:59 PM UTC, original submission:  

Using the latest sparsersb 1.0.8

Attempting to install in Windows 64 when octave was installed to a path that has a space in it fails.

Looking at the error, the Makefile is setting environment varaiable CXXFLAGS to specify the include paths rather than passing directly to mkoctfile and so msys is expanding them to full windows names.

Changing the makefile (diff) make it work for me:


-       LFLAGS="$(LDFLAGS)" LDFLAGS="$(LDFLAGS)" CXXFLAGS="$(CXXFLAGS) $(CXXFLAGS_CXX11)" $(MKOCTFILE) -D'RSB_SPARSERSB_LABEL=sparsersb' -o $@ $<
+       $(MKOCTFILE) $(LDFLAGS) $(CXXFLAGS) $(CXXFLAGS_CXX11) -D'RSB_SPARSERSB_LABEL=sparsersb' -o $@ $<


For the crash, running some of the tested that are embedded in sparsersb.cc will do it:


rrm=sparsersb(sprand(1000,1000,0.001));
sparsersb(rrm,"render", "sparsersb_temporary_render.eps" ,1024);


It doesn't happen 100% of the time - more like 80% ?

Crash seen in both octave 5.2.0 and 6.0.1




John Donoghue <lostbard>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #50170:  rsbbugs-0.0.tar.gz added by dezperado (82KiB - application/gzip)
file #50168:  mingw-of-sparsersb-1-no_render.patch added by mmuetzel (817B - application/octet-stream)
file #50149:  sparsersb.cc added by dezperado (213KiB - text/x-c++src)
file #49851:  sparsersb.cc added by dezperado (213KiB - text/x-c++src)
file #49848:  sparsersb.cc added by dezperado (214KiB - text/x-c++src)
file #49838:  crash.eps added by mmuetzel (2KiB - application/postscript)
file #49834:  rsb_mtx_rndr.c added by dezperado (2KiB - text/x-csrc)
file #49830:  librsb added by mmuetzel (216KiB - application/octet-stream)
file #49787:  sparsersb_bt_pkg_test_with_symbols.log added by mmuetzel (36KiB - application/octet-stream)
file #49788:  sparsersb_bt_render_with_symbols.log added by mmuetzel (6KiB - application/octet-stream)
file #49730:  sparsersb_bt_pkg_test.log added by mmuetzel (22KiB - application/octet-stream)
file #49731:  sparsersb_bt_render.log added by mmuetzel (5KiB - application/octet-stream)
file #49683:  t.eps added by lostbard (686B - application/postscript)
file #49684:  crash.mtx added by lostbard (28KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by siko1056 (Posted a comment)
  • -email is unavailable- added by mmuetzel (Updated the item)
  • -email is unavailable- added by dezperado (Posted a comment)
  • -email is unavailable- added by lostbard (Submitted the item)
  • -email is unavailable- added by lostbard (Adding package maintainer)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 20 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-11-02 mmuetzel StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
    2020-11-02 mmuetzel Attached File- Added bug58957_librsb_double_free.patch, #50189
    2020-10-30 dezperado Attached File- Added rsbbugs-0.0.tar.gz, #50170
    2020-10-30 mmuetzel Attached File- Added mingw-of-sparsersb-1-no_render.patch, #50168
    2020-10-29 siko1056 StatusNone Patch Submitted
    2020-10-28 dezperado Attached File- Added sparsersb.cc, #50149
    2020-10-28 siko1056 Releasedev 6.0.92
    2020-09-23 dezperado Attached File- Added sparsersb.cc, #49851
    2020-09-23 dezperado Attached File- Added sparsersb.cc, #49848
    2020-09-22 mmuetzel Attached File- Added crash.eps, #49838
    2020-09-21 dezperado Attached File- Added rsb_mtx_rndr.c, #49834
    2020-09-21 mmuetzel Attached File- Added librsb, #49830
    2020-09-11 mmuetzel Attached File- Added sparsersb_bt_pkg_test_with_symbols.log, #49787
        Attached File- Added sparsersb_bt_render_with_symbols.log, #49788
    2020-09-01 mmuetzel Attached File- Added sparsersb_bt_pkg_test.log, #49730
        Attached File- Added sparsersb_bt_render.log, #49731
    2020-08-17 lostbard Attached File- Added t.eps, #49683
        Attached File- Added crash.mtx, #49684
    2020-08-14 lostbard Carbon-Copy- Added -email is unavailable-

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code