bugGNU Octave - Bugs: bug #50674, audiorecorder() crashes in...

 
 

bug #50674: audiorecorder() crashes in getaudiodata()

Submitter:  Lars Kindermann <larskindermann>
Submitted:  Wed 29 Mar 2017 10:37:45 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Segfault, Bus Error, etc.
Status:  Fixed Assigned to:  mtmiller
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 13 Dec 2018 05:37:21 AM UTC, comment #13: 

@Lars: I checked in your cset here https://hg.savannah.gnu.org/hgweb/octave/rev/471513a016ec.

I added a comment to the code that referencse this bug report so no one will accidentally undo the change in the future.

Marking as fixed and closing report.

Rik <rik5>
Group administrator
Thu 27 Sep 2018 11:55:02 PM UTC, comment #12: 

@Mike: You did a lot of the earlier work on PortAudio.  Can you review this patch?

Rik <rik5>
Group administrator
Wed 26 Sep 2018 04:47:16 PM UTC, comment #11: 

I'm running this patch for few month now with no more crashes.
I have created and attached a changeset, could you please push it?

Here is a little example for a live audio display.
It works exactly as in Matlab now:


#example for live recording and processing
recorder = audiorecorder ();
record (recorder)
pause (1)
for i=1:60000
  get (recorder);
  data = getaudiodata (recorder);
  #size (data)
  plot(data(end-40000:end,:))
  xlim([0 40000])
  ylim([-1 1])
  drawnow
end
stop (recorder)

 


(file #45103)

Lars Kindermann <larskindermann>
Sat 16 Jun 2018 12:42:58 AM UTC, comment #10: 

No reply on this, yet? It really would be nice to have a non-crashing getaudiodata() in the repository. Again: in Matlab it is possible to call getaudiodata() while recording. Both proposed patches seem to solve the problem, but my one does the job without introducing any additional mutexes...

Lars Kindermann <larskindermann>
Mon 04 Jun 2018 02:31:12 PM UTC, comment #9: 

Here is a much smaller patch. It adresses just a possible buffer  increase during getaudiodata() but it seems to fix the problem completely for me. Can somebody else please test? 


diff -r 13b1b9a0d9c5 libinterp/dldfcn/audiodevinfo.cc
--- a/libinterp/dldfcn/audiodevinfo.cc        Sun Jun 03 19:51:11 2018 +0200
+++ b/libinterp/dldfcn/audiodevinfo.cc        Mon Jun 04 15:32:40 2018 +0200
@@ -1637,9 +1637,12 @@
 octave_value
 audiorecorder::getaudiodata (void)
 {
-  Matrix audio (2, left.size ());
-
-  for (unsigned int i = 0; i < left.size (); i++)
+
+  unsigned int ls = left.size ();
+
+  Matrix audio (2, ls);
+
+  for (unsigned int i = 0; i < ls; i++)
     {
       audio(0,i) = left[i];
       audio(1,i) = right[i];



(file #44291)

Lars Kindermann <larskindermann>
Mon 14 May 2018 09:07:32 AM UTC, comment #8: 

This is still open and continues to crash 4.4.0 and dev. Shouldn't jwe's patch be checked in now?

Lars Kindermann <larskindermann>
Thu 31 Aug 2017 04:51:23 AM UTC, comment #7: 

Mike, to keep compability, getaudiodata() should work while recording.

I don't have Matlab access at the moment to verify for the current version, but I did a lot of audio processing with Matlab which I try to port to Octave now. The Matlab options for real-time recording and processing are:

Audiorecorder object: easy to use, it is possible to access data during async record, either in a loop or timer-fcn, but you can only retrieve the complete data from the start of the recording, like Octave does now (apart from the bug). This means after some time getaudiodata returns very big arrays where only the end is of interest, and finally the memory will be full. But for some time you can do real-time processing thiy way.
 
Data aquisition toolbox: works like it should be: you can record forever because buffers can be released after retrieval. The only problem is the restriction to standard audio settings: 2 channels and limited samplingrates.

Playrec: best solution so far, simultanous recording and playback of lots of channels, resolutions and samplerates, all that portaudio supports. I managed to process 16 channels at 192kHz and 32Bit resolution. Problem: difficult to compile and setup. But code is open source.

Lars Kindermann <larskindermann>
Wed 30 Aug 2017 09:16:06 PM UTC, comment #6: 

Lars - if you have access to Matlab, can you confirm that Matlab lets you read the partial audio data from the audiorecorder object while it is still in the recording state?

I could see another valid solution, and more efficient, would be to simply return an empty array while the recorder is still recording.

I don't have access to Matlab so I don't know what the correct behavior is.

Mike Miller <mtmiller>
Group Member
Tue 29 Aug 2017 11:33:34 PM UTC, comment #5: 

Just so everyone looking at this is aware, the audioplayer already supports a user callback function handle to supply the audio samples sent to the sound device. This is a way to have a user function generate the audio sequence in "real time" instead of from a complete data array stored in memory.

The plan was to provide the same functionality for the audiorecorder object, so a user function could be passed to the constructor, and it would call the user function with blocks of data for processing. I think this was causing problems and wasn't finished by the end of the GSoC project. But there are some hooks in there to add this capability in the future.

Does Matlab actually allow reading of the audio array while the audiorecorder object is still in the (asynchronous) recording state?

Mike Miller <mtmiller>
Group Member
Tue 29 Aug 2017 11:15:38 PM UTC, comment #4: 

The patch is still pending, so at the moment there is no way to process audio data while the recorder is still running.


Regarding buffer handling with portaudio, a look at the playrec code (http://www.playrec.co.uk) might give some insights. I think there it is done very similar to jwe's ideas with a chain of buffers.

To do realtime audio processing, at the moment this playrec toolbox is the only option availabe for Octave. (except some weird solution like recording externally using sox into a named pipe which can be read as a file by octave)

In Matlab, the Data Aquisition toolbox provides the functionality to record from the soundcard continuously foreverm, so they don't have to put this ability into the audiorecorder object. It cannot  be used for unlimited time because its buffer cannot be flushed and will grow forever until the memory is exhausted.

If somebody is going to dive into the audiorecorder code, what about introducing a getaudiodata(recorder,'flush') option, which will clear the audio buffer after reading, thus providing the simpelest possible method for continuous audio processing as an alternative to the Data Aquisition toolbox - which is not availabe for Octave anyhow?

Lars Kindermann <larskindermann>
Thu 30 Mar 2017 09:24:12 PM UTC, comment #3: 

I verified the patch fixes things for me.

@jwe: You raised the suggestion that this might require more in-depth work to fix then just the mutex lock.  Should you check in your patch as an immediate interim fix?  If the concern is more about performance, should that be the subject of another bug report?

Rik <rik5>
Group administrator
Thu 30 Mar 2017 01:42:03 PM UTC, comment #2: 

Using the attached patch, I was able to avoid the crash.  So it seems that portaudio_record_callback could be called while getaudiodata is active.  If so, and if the callback function causes the left and right data vectors to be resized/reallocated, then we are in trouble, and that would explain the double free crash.

However, the implementation looks like trouble to me since it can resize and possibly copy the left and right data buffers many times.  Maybe we should use a list of buffers so that once once is created, its address can't change?  And each buffer in the list should be allocated to have some predetermined size so that it is not resized.  When a buffer fills, we just add another to the list.  Then getting data out and into a contiguous array means concatenating all these buffers.

Even with a scheme like that, locking might be necessary, but it would be good to limit the time spent locked.  I admit I don't understand exactly how all this works, but I imagine that locking could disrupt recording if copying a large buffer (or list of buffers) requires too much time.

(file #40211)

John W. Eaton <jwe>
Group administrator
Thu 30 Mar 2017 05:07:02 AM UTC, comment #1: 

I used a slightly modified program which I have attached.


more off
recorder = audiorecorder ()
record (recorder)
for i=1:20
  printf ("############################################################\n");
  printf ("Executing Loop #%d\n", i);
  printf ("############################################################\n");

  pause (1);

  get (recorder)
  printf ("executed get (recorder)\n");

  data = getaudiodata (recorder);
  printf ("got data\n", i);

  size (data)
end
stop (recorder)


Here is a backtrace from gdb, also attached.


#0  0x00007f3c29e32428 in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007f3c29e3402a in __GI_abort () at abort.c:89
#2  0x00007f3c29e747ea in __libc_message (do_abort=do_abort@entry=2,
    fmt=fmt@entry=0x7f3c29f8d2e0 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007f3c29e7ce0a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>,
    str=0x7f3c29f8d3f0 "double free or corruption (out)", action=3) at malloc.c:5004
#4  _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3865
#5  0x00007f3c29e8098c in __GI___libc_free (mem=<optimized out>) at malloc.c:2966
#6  0x00007f3c2beb1f53 in Array<double>::ArrayRep::~ArrayRep (this=0x1952790,
    __in_chrg=<optimized out>) at ./liboctave/array/Array.h:170
#7  0x00007f3c2beb1d79 in Array<double>::~Array (this=0x1a17790,
    __in_chrg=<optimized out>) at ./liboctave/array/Array.h:306
#8  0x00007f3c2beb1ae0 in MArray<double>::~MArray (this=0x1a17790,
    __in_chrg=<optimized out>) at ./liboctave/array/MArray.h:83
#9  0x00007f3c2beb1b48 in NDArray::~NDArray (this=0x1a17790, __in_chrg=<optimized out>)
    at ./liboctave/array/dNDArray.h:38
#10 0x00007f3c2bef25ac in octave_base_matrix<NDArray>::~octave_base_matrix (
    this=0x1a17780, __in_chrg=<optimized out>)
    at ./libinterp/octave-value/ov-base-mat.h:68
#11 0x00007f3c2bf7567c in octave_matrix::~octave_matrix (this=0x1a17780,
    __in_chrg=<optimized out>) at ./libinterp/octave-value/ov-re-mat.h:97
#12 0x00007f3c2bf756ac in octave_matrix::~octave_matrix (this=0x1a17780,
    __in_chrg=<optimized out>) at ./libinterp/octave-value/ov-re-mat.h:97
#13 0x00007f3c2bea61bf in octave_value::operator= (this=0x198e120, a=...)
    at ./libinterp/octave-value/ov.h:362
#14 0x00007f3c2c379fb4 in octave_value::assign (this=0x198e120,
    op=octave_value::op_asn_eq, rhs=...) at libinterp/octave-value/ov.cc:1586
#15 0x00007f3c2c7fd052 in symbol_table::symbol_record::symbol_record_rep::assign (
    this=0x198e020, op=octave_value::op_asn_eq, value=..., context=18446744073709551615)
    at libinterp/corefcn/symtab.h:241
#16 0x00007f3c2c7fd151 in symbol_table::symbol_record::assign (this=0x7fff3a85e768,
    op=octave_value::op_asn_eq, value=..., context=18446744073709551615)
---Type <return> to continue, or q <return> to quit---
   cn/symtab.h:547
#17 0x00007f3c2c7fcb0d in octave_lvalue::assign (this=0x7fff3a85e760, op=octave_value::op_asn_eq, rhs=...)
    at libinterp/corefcn/oct-lvalue.cc:38
#18 0x00007f3c2c4395ba in octave::tree_simple_assignment::rvalue1 (this=0x198ef60) at libinterp/parse-tree/pt-assign.cc:103
#19 0x00007f3c2c446261 in octave::tree_evaluator::visit_statement (this=0x1404dd0, stmt=...) at libinterp/parse-tree/pt-eval.cc:685
#20 0x00007f3c2c4687fe in octave::tree_statement::accept (this=0x198efa0, tw=...) at libinterp/parse-tree/pt-stmt.cc:188
#21 0x00007f3c2c446405 in octave::tree_evaluator::visit_statement_list (this=0x1404dd0, lst=...)
    at libinterp/parse-tree/pt-eval.cc:727
#22 0x00007f3c2c469460 in octave::tree_statement_list::accept (this=0x185fdb0, tw=...) at libinterp/parse-tree/pt-stmt.cc:323
#23 0x00007f3c2c444cfa in octave::tree_evaluator::visit_simple_for_command (this=0x1404dd0, cmd=...)
    at libinterp/parse-tree/pt-eval.cc:306
#24 0x00007f3c2c4528aa in octave::tree_simple_for_command::accept (this=0x198fe80, tw=...) at libinterp/parse-tree/pt-loop.cc:127
#25 0x00007f3c2c446179 in octave::tree_evaluator::visit_statement (this=0x1404dd0, stmt=...) at libinterp/parse-tree/pt-eval.cc:659
#26 0x00007f3c2c4687fe in octave::tree_statement::accept (this=0x198fdf0, tw=...) at libinterp/parse-tree/pt-stmt.cc:188
#27 0x00007f3c2c446405 in octave::tree_evaluator::visit_statement_list (this=0x1404dd0, lst=...)
    at libinterp/parse-tree/pt-eval.cc:727
#28 0x00007f3c2c469460 in octave::tree_statement_list::accept (this=0x172b8b0, tw=...) at libinterp/parse-tree/pt-stmt.cc:323
#29 0x00007f3c2c36a833 in octave_user_script::do_multi_index_op (this=0x19907d0, nargout=0, args=...)
    at libinterp/octave-value/ov-usr-fcn.cc:153
#30 0x00007f3c2c379c4d in octave_value::do_multi_index_op (this=0x7fff3a85ed70, nargout=0, idx=...)
    at libinterp/octave-value/ov.cc:1529
#31 0x00007f3c2c44ae83 in octave::tree_identifier::rvalue (this=0x191a450, nargout=0, lvalue_list=0x0)
    at libinterp/parse-tree/pt-id.cc:93
#32 0x00007f3c2c42a4a6 in octave::tree_identifier::rvalue (this=0x191a450, nargout=0) at libinterp/parse-tree/pt-id.h:118
#33 0x00007f3c2c44b264 in octave::tree_identifier::rvalue1 (this=0x191a450, nargout=0) at libinterp/parse-tree/pt-id.cc:126
#34 0x00007f3c2c446261 in octave::tree_evaluator::visit_statement (this=0x1404dd0, stmt=...) at libinterp/parse-tree/pt-eval.cc:685
#35 0x00007f3c2c4687fe in octave::tree_statement::accept (this=0x19798c0, tw=...) at libinterp/parse-tree/pt-stmt.cc:188
#36 0x00007f3c2c446405 in octave::tree_evaluator::visit_statement_list (this=0x1404dd0, lst=...)
    at libinterp/parse-tree/pt-eval.cc:727
#37 0x00007f3c2c469460 in octave::tree_statement_list::accept (this=0x1575f20, tw=...) at libinterp/parse-tree/pt-stmt.cc:323
#38 0x00007f3c2c77759b in octave::interpreter::main_loop (this=0x13f54a0) at libinterp/corefcn/interpreter.cc:939
#39 0x00007f3c2c776382 in octave::interpreter::execute (this=0x13f54a0) at libinterp/corefcn/interpreter.cc:663
#40 0x00007f3c2bea51f5 in octave::application::execute_interpreter (this=0x7fff3a85f3b0) at libinterp/octave.cc:394
#41 0x00007f3c2bea54ba in octave::cli_application::execute (this=0x7fff3a85f3b0) at libinterp/octave.cc:439
#42 0x00000000004017f4 in main (argc=8, argv=0x7fff3a85f608) at src/main-cli.cc:90


There is a double free or other corruption in malloc from the getaudiodata call.


(file #40205, file #40206)

Rik <rik5>
Group administrator
Wed 29 Mar 2017 10:37:45 PM UTC, original submission:  

Executing this script hangs Octave after few iteratons and leaves it stuck at 100% CPU, both in GUI and on commandline.


recorder = audiorecorder ()
record (recorder)
for i=1:60
  pause (1)
  get (recorder)
  data = getaudiodata (recorder);
  size (data)
end
stop (recorder)


CTRL-C then terminates the GUI. On commandline this info is shown:


^C^C^Cfatal: caught signal Interrupt -- stopping myself...
attempting to save variables to 'octave-workspace'...
error: octave_base_value::save_binary(): wrong type argument 'audiorecorder'
terminate called after throwing an instance of 'octave::execution_exception'
octave exited with signal 6


I wonder if it is hardware dependent, a portaudio thing or Octave?

Lars Kindermann <larskindermann>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #45103:  bug50674.changeset added by larskindermann (976B - application/octet-stream)
file #44291:  diffs_v2.txt added by larskindermann (533B - text/plain - small patch )
file #40211:  diffs.txt added by jwe (2KiB - text/plain)
file #40205:  tst_rec.m added by rik5 (428B - text/x-objcsrc)
file #40206:  btrace.log added by rik5 (6KiB - text/x-log)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by jwe (Updated the item)
  • -email is unavailable- added by larskindermann (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-12-13 rik5 StatusPatch Submitted Fixed
        Open/ClosedOpen Closed
    2018-09-28 mtmiller Assigned toNone mtmiller
    2018-09-26 mtmiller Carbon-CopyRemoved 80942 -
    2018-09-26 larskindermann Attached File- Added bug50674.changeset, #45103
    2018-06-04 larskindermann Attached File- Added diffs_v2.txt, #44291
    2017-08-31 rik5 Carbon-CopyRemoved 72865 -
    2017-03-30 rik5 StatusConfirmed Patch Submitted
    2017-03-30 jwe Attached File- Added diffs.txt, #40211
    2017-03-30 rik5 StatusNone Confirmed
    2017-03-30 rik5 Attached File- Added tst_rec.m, #40205
        Attached File- Added btrace.log, #40206

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code