bugGNU Octave - Bugs: bug #50674, audiorecorder() crashes in...

 
 

bug #50674: audiorecorder() crashes in getaudiodata()

Submitted by:  Lars Kindermann <larskindermann>
Submitted on:  Wed 29 Mar 2017 10:37:45 PM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Segfault, Bus Error, etc.
Status: Patch SubmittedAssigned to: None
Originator Name: Open/Closed: Open
Release: devOperating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 31 Aug 2017 04:51:23 AM UTC, comment #7:

Mike, to keep compability, getaudiodata() should work while recording.

I don't have Matlab access at the moment to verify for the current version, but I did a lot of audio processing with Matlab which I try to port to Octave now. The Matlab options for real-time recording and processing are:

Audiorecorder object: easy to use, it is possible to access data during async record, either in a loop or timer-fcn, but you can only retrieve the complete data from the start of the recording, like Octave does now (apart from the bug). This means after some time getaudiodata returns very big arrays where only the end is of interest, and finally the memory will be full. But for some time you can do real-time processing thiy way.

Data aquisition toolbox: works like it should be: you can record forever because buffers can be released after retrieval. The only problem is the restriction to standard audio settings: 2 channels and limited samplingrates.

Playrec: best solution so far, simultanous recording and playback of lots of channels, resolutions and samplerates, all that portaudio supports. I managed to process 16 channels at 192kHz and 32Bit resolution. Problem: difficult to compile and setup. But code is open source.

Lars Kindermann <larskindermann>
Wed 30 Aug 2017 09:16:06 PM UTC, comment #6:

Lars - if you have access to Matlab, can you confirm that Matlab lets you read the partial audio data from the audiorecorder object while it is still in the recording state?

I could see another valid solution, and more efficient, would be to simply return an empty array while the recorder is still recording.

I don't have access to Matlab so I don't know what the correct behavior is.

Mike Miller <mtmiller>
Project Administrator
Tue 29 Aug 2017 11:33:34 PM UTC, comment #5:

Just so everyone looking at this is aware, the audioplayer already supports a user callback function handle to supply the audio samples sent to the sound device. This is a way to have a user function generate the audio sequence in "real time" instead of from a complete data array stored in memory.

The plan was to provide the same functionality for the audiorecorder object, so a user function could be passed to the constructor, and it would call the user function with blocks of data for processing. I think this was causing problems and wasn't finished by the end of the GSoC project. But there are some hooks in there to add this capability in the future.

Does Matlab actually allow reading of the audio array while the audiorecorder object is still in the (asynchronous) recording state?

Mike Miller <mtmiller>
Project Administrator
Tue 29 Aug 2017 11:15:38 PM UTC, comment #4:

The patch is still pending, so at the moment there is no way to process audio data while the recorder is still running.

Regarding buffer handling with portaudio, a look at the playrec code (http://www.playrec.co.uk) might give some insights. I think there it is done very similar to jwe's ideas with a chain of buffers.

To do realtime audio processing, at the moment this playrec toolbox is the only option availabe for Octave. (except some weird solution like recording externally using sox into a named pipe which can be read as a file by octave)

In Matlab, the Data Aquisition toolbox provides the functionality to record from the soundcard continuously foreverm, so they don't have to put this ability into the audiorecorder object. It cannot be used for unlimited time because its buffer cannot be flushed and will grow forever until the memory is exhausted.

If somebody is going to dive into the audiorecorder code, what about introducing a getaudiodata(recorder,'flush') option, which will clear the audio buffer after reading, thus providing the simpelest possible method for continuous audio processing as an alternative to the Data Aquisition toolbox - which is not availabe for Octave anyhow?

Lars Kindermann <larskindermann>
Thu 30 Mar 2017 09:24:12 PM UTC, comment #3:

I verified the patch fixes things for me.

@jwe: You raised the suggestion that this might require more in-depth work to fix then just the mutex lock. Should you check in your patch as an immediate interim fix? If the concern is more about performance, should that be the subject of another bug report?

Rik <rik5>
Project Administrator
Thu 30 Mar 2017 01:42:03 PM UTC, comment #2:

Using the attached patch, I was able to avoid the crash. So it seems that portaudio_record_callback could be called while getaudiodata is active. If so, and if the callback function causes the left and right data vectors to be resized/reallocated, then we are in trouble, and that would explain the double free crash.

However, the implementation looks like trouble to me since it can resize and possibly copy the left and right data buffers many times. Maybe we should use a list of buffers so that once once is created, its address can't change? And each buffer in the list should be allocated to have some predetermined size so that it is not resized. When a buffer fills, we just add another to the list. Then getting data out and into a contiguous array means concatenating all these buffers.

Even with a scheme like that, locking might be necessary, but it would be good to limit the time spent locked. I admit I don't understand exactly how all this works, but I imagine that locking could disrupt recording if copying a large buffer (or list of buffers) requires too much time.

(file #40211)

John W. Eaton <jwe>
Project Administrator
Thu 30 Mar 2017 05:07:02 AM UTC, comment #1:

I used a slightly modified program which I have attached.

Here is a backtrace from gdb, also attached.

There is a double free or other corruption in malloc from the getaudiodata call.

(file #40205, file #40206)

Rik <rik5>
Project Administrator
Wed 29 Mar 2017 10:37:45 PM UTC, original submission:

Executing this script hangs Octave after few iteratons and leaves it stuck at 100% CPU, both in GUI and on commandline.

CTRL-C then terminates the GUI. On commandline this info is shown:

I wonder if it is hardware dependent, a portaudio thing or Octave?

Lars Kindermann <larskindermann>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #40211:  diffs.txt added by jwe (2KiB - text/plain)
file #40205:  tst_rec.m added by rik5 (428B - text/x-objcsrc)
file #40206:  btrace.log added by rik5 (6KiB - text/x-log)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by jwe (Updated the item)
  • -unavailable- added by larskindermann (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 31 Aug 2017 03:14:54 PM UTCrik5Carbon-CopyRemoved 72865=>-
    Thu 30 Mar 2017 09:24:12 PM UTCrik5StatusConfirmed=>Patch Submitted
    Thu 30 Mar 2017 01:42:03 PM UTCjweAttached File-=>Added diffs.txt, #40211
    Thu 30 Mar 2017 05:07:28 AM UTCrik5StatusNone=>Confirmed
    Thu 30 Mar 2017 05:07:02 AM UTCrik5Attached File-=>Added tst_rec.m, #40205
      Attached File-=>Added btrace.log, #40206

    Back to the top


    Powered by Savane 3.1-cleanup1