bugGNU Octave - Bugs: bug #60081, loading bad hdf file corrupts...

 
 

bug #60081: loading bad hdf file corrupts memory; segfault at exit

Submitter:  Dmitri A. Sergatskov <dasergatskov>
Submitted:  Fri 19 Feb 2021 11:41:23 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * 6.1.0 Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 21 Feb 2021 05:22:49 PM UTC, comment #9: 

I pushed the patches to stable here:
https://hg.savannah.gnu.org/hgweb/octave/rev/26039aa0d2c4
https://hg.savannah.gnu.org/hgweb/octave/rev/cee1cf711a4e

Thanks again for testing.

Closing as fixed.

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Feb 2021 04:50:43 PM UTC, comment #8: 

Yes, the second patch fixed the "heap-buffer-overflow".

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 21 Feb 2021 04:46:08 PM UTC, comment #7: 

Thanks for testing.

The attached additional patch will hopefully fix that issue as well.

With that change the original file is actually correctly read in Octave for me.
But I think the first patch should still be applied to avoid a similar issue if `load` is used with a HDF5 file that actually stores types that Octave cannot recognize.

(file #50902)

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Feb 2021 04:17:47 PM UTC, comment #6: 

I confirm that I do not get the crash with this patch on "normal" binary. With ASAn I still get "heap-buffer-overflow" at what looks to me

#2 0x7fa6bfaddf25 in hdf5_read_next_data_internal ../libinterp/corefcn/ls-hdf5.cc:746


octave:1> load("test_matlab_h5write_bad.hdf5")
=================================================================
==2362753==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000213817 at pc 0x7fa6c14fb19d bp 0x7fa6878ec9f0 sp 0x7fa6878ec198
READ of size 8 at 0x602000213817 thread T6 (QThread)
    #0 0x7fa6c14fb19c  (/lib64/libasan.so.5+0xad19c)
    #1 0x7fa6b3eab0da in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (/lib64/libstdc++.so.6+0x1300da)
    #2 0x7fa6bfaddf25 in hdf5_read_next_data_internal ../libinterp/corefcn/ls-hdf5.cc:746
    #3 0x7fa6b9e29375  (/lib64/libhdf5.so.103+0x134375)
    #4 0x7fa6b9e30924 in H5G__node_iterate (/lib64/libhdf5.so.103+0x13b924)
    #5 0x7fa6b9d62998  (/lib64/libhdf5.so.103+0x6d998)
    #6 0x7fa6b9d63e5a in H5B_iterate (/lib64/libhdf5.so.103+0x6ee5a)
    #7 0x7fa6b9e36a4b in H5G__stab_iterate (/lib64/libhdf5.so.103+0x141a4b)
    #8 0x7fa6b9e33501 in H5G__obj_iterate (/lib64/libhdf5.so.103+0x13e501)
    #9 0x7fa6b9e2a641 in H5G_iterate (/lib64/libhdf5.so.103+0x135641)
    #10 0x7fa6b9e26d97 in H5Giterate (/lib64/libhdf5.so.103+0x131d97)
    #11 0x7fa6bfae0777 in read_hdf5_data(std::istream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, octave_value&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, string_vector const&, int, int) ../libinterp/corefcn/ls-hdf5.cc:1133
    #12 0x7fa6bfab7cc3 in octave::load_save_system::load_vars(std::istream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, octave::load_save_format const&, octave::mach_info::float_format, bool, bool, bool, string_vector const&, int, int, int) ../libinterp/corefcn/load-save.cc:425
    #13 0x7fa6bfac4042 in octave::load_save_system::load(octave_value_list const&, int) ../libinterp/corefcn/load-save.cc:1301
    #14 0x7fa6bfac71b9 in Fload(octave::interpreter&, octave_value_list const&, int) ../libinterp/corefcn/load-save.cc:1658
<...deleted...>


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sun 21 Feb 2021 04:07:45 PM UTC, comment #5: 

It looks like the HDF5 library has issues if the execution is interrupted inside a function the library is calling.

The attached patch catches the exception and lets the HDF5 library recover from the error instead. This avoids the crash on exit.

The library seems to be more verbose in this case (at least the HDF5 library that comes with Ubuntu 20.10). But that is probably ok because an invalid HDF5 file is not the "normal" use case:

octave:1> load("test_matlab_h5write_bad.hdf5")
warning: load: error while reading hdf5 item 'data'
HDF5-DIAG: Error detected in HDF5 (1.10.6) thread 0:
  #000: ../../../src/H5Gdeprec.c line 771 in H5Giterate(): group iteration failed
    major: Symbol table
    minor: Iteration failed
  #001: ../../../src/H5Gint.c line 854 in H5G_iterate(): error iterating over links
    major: Symbol table
    minor: Iteration failed
  #002: ../../../src/H5Gobj.c line 698 in H5G__obj_iterate(): can't iterate over symbol table
    major: Symbol table
    minor: Iteration failed
  #003: ../../../src/H5Gstab.c line 557 in H5G__stab_iterate(): iteration operator failed
    major: Symbol table
    minor: Can't move to next iterator location
  #004: ../../../src/H5B.c line 1211 in H5B_iterate(): B-tree iteration failed
    major: B-Tree node
    minor: Iteration failed
  #005: ../../../src/H5B.c line 1167 in H5B__iterate_helper(): B-tree iteration failed
    major: B-Tree node
    minor: Iteration failed
  #006: ../../../src/H5Gnode.c line 1015 in H5G__node_iterate(): iteration operator failed
    major: Symbol table
    minor: Can't move to next iterator location
octave:2> exit



(file #50899)

Markus Mützel <mmuetzel>
Group administrator
Sun 21 Feb 2021 12:10:18 PM UTC, comment #4: 

I tried with octave 6.2.0 and 7.0.0.

Without gui it just "wait" (see att1.png)

With gui, after "exit", the gui closes but the process I had to kill manually


Leonardo <pocker>
Sat 20 Feb 2021 11:36:03 PM UTC, comment #3: 

This is what I see with 6.1.91:



octave:1> load("test_matlab_h5write_bad.hdf5")
error: octave_base_value::load_binary(): wrong type argument '<unknown type>'
octave:2> exit
fatal: caught signal Segmentation fault -- stopping myself...
Segmentation fault (core dumped)


Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Sat 20 Feb 2021 11:32:51 PM UTC, comment #2: 

Yes. Because the hdf is bad. Did you get a segfault when you exit octave?

Dmitri.

Dmitri A. Sergatskov <dasergatskov>
Sat 20 Feb 2021 11:05:55 PM UTC, comment #1: 

I tried to load the hdf5 file and I got this:

octave:1> load("test_matlab_h5write_bad.hdf5");
error: octave_base_value::load_binary(): wrong type argument '<unknown type>'

Leonardo <pocker>
Fri 19 Feb 2021 11:41:23 PM UTC, original submission:  

I tried to load “bad” hdf file (see https://octave.discourse.group/t/low-level-write-hdf5-file-in-matlab-for-use-in-octave/792/2 for details) on octave 6.1.1 and it failed and then octave segfault at exit. The hdf file is attached. When I tried to do the same with ASAN I got:


octave:1> load("test_matlab_h5write_bad.hdf5");
=================================================================
==2098438==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000215617 at pc 0x7ffff6eb319d bp 0x7fffbd2c9a10 sp 0x7fffbd2c91b8
READ of size 8 at 0x602000215617 thread T6 (QThread)
    #0 0x7ffff6eb319c  (/lib64/libasan.so.5+0xad19c)
    #1 0x7fffe98630da in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) (/lib64/libstdc++.so.6+0x1300da)
    #2 0x7ffff5495f25 in hdf5_read_next_data_internal ../libinterp/corefcn/ls-hdf5.cc:744
    #3 0x7fffef7e1375  (/lib64/libhdf5.so.103+0x134375)
    #4 0x7fffef7e8924 in H5G__node_iterate (/lib64/libhdf5.so.103+0x13b924)
    #5 0x7fffef71a998  (/lib64/libhdf5.so.103+0x6d998)
    #6 0x7fffef71be5a in H5B_iterate (/lib64/libhdf5.so.103+0x6ee5a)
    #7 0x7fffef7eea4b in H5G__stab_iterate (/lib64/libhdf5.so.103+0x141a4b)
    #8 0x7fffef7eb501 in H5G__obj_iterate (/lib64/libhdf5.so.103+0x13e501)
    #9 0x7fffef7e2641 in H5G_iterate (/lib64/libhdf5.so.103+0x135641)
    #10 0x7fffef7ded97 in H5Giterate (/lib64/libhdf5.so.103+0x131d97)
    #11 0x7ffff54986ea in read_hdf5_data(std::istream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool&, octave_value&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, string_vector const&, int, int) ../libinterp/corefcn/ls-hdf5.cc:1110
    #12 0x7ffff546fcc3 in octave::load_save_system::load_vars(std::istream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, octave::load_save_format const&, octave::mach_info::float_format, bool, bool, bool, string_vector const&, int, int, int) ../libinterp/corefcn/load-save.cc:425
    #13 0x7ffff547c042 in octave::load_save_system::load(octave_value_list const&, int) ../libinterp/corefcn/load-save.cc:1301
    #14 0x7ffff547f1b9 in Fload(octave::interpreter&, octave_value_list const&, int) ../libinterp/corefcn/load-save.cc:1658
    #15 0x7ffff46b9c1e in octave_builtin::execute(octave::tree_evaluator&, int, octave_value_list const&) ../libinterp/octave-value/ov-builtin.cc:65
    #16 0x7ffff47b1bc1 in octave_function::call(octave::tree_evaluator&, int, octave_value_list const&) ../libinterp/octave-value/ov-fcn.cc:57
    #17 0x7ffff4b5aeee in octave::tree_index_expression::evaluate_n(octave::tree_evaluator&, int) ../libinterp/parse-tree/pt-idx.cc:402
    #18 0x7ffff4b60cc2 in octave::tree_index_expression::evaluate(octave::tree_evaluator&, int) (/home/dima/src/octave/gcc_debug/libinterp/.libs/liboctinterp.so.8+0x18cacc2)
    #19 0x7ffff4b115e0 in octave::tree_evaluator::visit_statement(octave::tree_statement&) ../libinterp/parse-tree/pt-eval.cc:3032
    #20 0x7ffff4b7c72a in octave::tree_statement::accept(octave::tree_walker&) ../libinterp/parse-tree/pt-stmt.h:124
    #21 0x7ffff4b12270 in octave::tree_evaluator::visit_statement_list(octave::tree_statement_list&) ../libinterp/parse-tree/pt-eval.cc:3117
    #22 0x7ffff45e9d62 in octave::tree_statement_list::accept(octave::tree_walker&) ../libinterp/parse-tree/pt-stmt.h:201
    #23 0x7ffff4af84c9 in octave::tree_evaluator::eval(std::shared_ptr<octave::tree_statement_list>&, bool) ../libinterp/parse-tree/pt-eval.cc:404
    #24 0x7ffff5405200 in octave::interpreter::main_loop() ../libinterp/corefcn/interpreter.cc:1269
    #25 0x7ffff53face4 in octave::interpreter::execute() ../libinterp/corefcn/interpreter.cc:797
    #26 0x7ffff65da5e2 in octave::interpreter_qobject::execute() ../libgui/src/interpreter-qobject.cc:87
    #27 0x7ffff6894c0f in octave::interpreter_qobject::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) libgui/src/moc-interpreter-qobject.cc:95
    #28 0x7ffff1c8a275 in QObject::event(QEvent*) (/lib64/libQt5Core.so.5+0x288275)
    #29 0x7ffff25aa5f4 in QApplicationPrivate::notify_helper(QObject*, QEvent*) (/lib64/libQt5Widgets.so.5+0x1675f4)
    #30 0x7ffff25b1b0f in QApplication::notify(QObject*, QEvent*) (/lib64/libQt5Widgets.so.5+0x16eb0f)
    #31 0x7ffff66f6113 in octave::octave_qapplication::notify(QObject*, QEvent*) ../libgui/src/octave-qobject.cc:136
    #32 0x7ffff1c60325 in QCoreApplication::notifyInternal2(QObject*, QEvent*) (/lib64/libQt5Core.so.5+0x25e325)
    #33 0x7ffff1c63596 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) (/lib64/libQt5Core.so.5+0x261596)
    #34 0x7ffff1cb3406  (/lib64/libQt5Core.so.5+0x2b1406)
    #35 0x7fffe4c6667c in g_main_context_dispatch (/lib64/libglib-2.0.so.0+0x4d67c)
    #36 0x7fffe4c66a47  (/lib64/libglib-2.0.so.0+0x4da47)
    #37 0x7fffe4c66adf in g_main_context_iteration (/lib64/libglib-2.0.so.0+0x4dadf)
    #38 0x7ffff1cb318a in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (/lib64/libQt5Core.so.5+0x2b118a)
    #39 0x7ffff1c5f18a in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (/lib64/libQt5Core.so.5+0x25d18a)
    #40 0x7ffff1abdd41 in QThread::exec() (/lib64/libQt5Core.so.5+0xbbd41)
    #41 0x7ffff1abf075  (/lib64/libQt5Core.so.5+0xbd075)
    #42 0x7fffe8d49149 in start_thread (/lib64/libpthread.so.0+0x8149)
    #43 0x7fffe8a7af22 in clone (/lib64/libc.so.6+0xfcf22)

0x602000215617 is located 0 bytes to the right of 7-byte region [0x602000215610,0x602000215617)
allocated by thread T6 (QThread) here:
    #0 0x7ffff6ef7990 in operator new[](unsigned long) (/lib64/libasan.so.5+0xf1990)
    #1 0x7ffff46a25f7 in std::_MakeUniq<char []>::__array std::make_unique<char []>(unsigned long) /usr/include/c++/8/bits/unique_ptr.h:833
    #2 0x7ffff5495d8d in hdf5_read_next_data_internal ../libinterp/corefcn/ls-hdf5.cc:731
    #3 0x7fffef7e1375  (/lib64/libhdf5.so.103+0x134375)

Thread T6 (QThread) created by T0 here:
    #0 0x7ffff6e58ea3 in __interceptor_pthread_create (/lib64/libasan.so.5+0x52ea3)
    #1 0x7ffff1abeaf6 in QThread::start(QThread::Priority) (/lib64/libQt5Core.so.5+0xbcaf6)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/lib64/libasan.so.5+0xad19c)
Shadow bytes around the buggy address:
  0x0c048003aa70: fa fa fd fa fa fa 00 00 fa fa 00 fa fa fa 00 fa
  0x0c048003aa80: fa fa 00 fa fa fa 00 fa fa fa 00 fa fa fa 00 fa
  0x0c048003aa90: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa 00 00
  0x0c048003aaa0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 fa
  0x0c048003aab0: fa fa 00 fa fa fa 00 00 fa fa 00 00 fa fa 00 00
=>0x0c048003aac0: fa fa[07]fa fa fa 00 00 fa fa 00 00 fa fa 00 00
  0x0c048003aad0: fa fa 00 00 fa fa 00 07 fa fa 00 fa fa fa 00 fa
  0x0c048003aae0: fa fa 00 fa fa fa 00 fa fa fa 00 fa fa fa fa fa
  0x0c048003aaf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c048003ab00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c048003ab10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2098438==ABORTING


Dmitri.



Dmitri A. Sergatskov <dasergatskov>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #50894:  att1.png added by pocker (13KiB - image/png)
file #50891:  test_matlab_h5write_bad.hdf5 added by dasergatskov (5KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Updated the item)
  • -email is unavailable- added by pocker (Posted a comment)
  • -email is unavailable- added by dasergatskov (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-02-21 mmuetzel StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
    2021-02-21 mmuetzel Attached File- Added bug60081_hdf5_non_zero_terminated.patch, #50902
    2021-02-21 mmuetzel Attached File- Added bug60081_hdf5_bad_file.patch, #50899
        StatusNone Patch Submitted
    2021-02-21 pocker Attached File- Added att1.png, #50894
    2021-02-19 dasergatskov Attached File- Added test_matlab_h5write_bad.hdf5, #50891

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code