bugGNU Octave - Bugs: bug #56600, system blocks when user calls a...

 
 

bug #56600: system blocks when user calls a background command ending in ampersand

Submitter:  Richard <crobar>
Submitted:  Tue 09 Jul 2019 10:44:20 AM UTC
   
 
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Matlab Compatibility
Status:  Confirmed Assigned to:  None
Originator Name:  Richard Crozier Open/Closed:  * Open
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Wed 10 Jul 2019 07:02:06 PM UTC, comment #7: 

I think we should support commands ending with ampersand, and be compatible in the return values and not block when called with two return values. We are basically almost compatible, we just need to return an empty string immediately when the command is backgrounded and the user requests command output. Maybe there is something simple we can do in octave_procbuf to handle this correctly?

I see that octave_procbuf already uses


execl (SHELL_PATH, "sh", "-c", command, static_cast<void *> (nullptr));


to run the system command argument when output is requested.

When I run "sh -c 'sleep 10 &'" at a command prompt, it returns immediately. However, when I try to capture the output with "foo=$(sh -c 'sleep 10 &')", it blocks. This is basically what's going on in this bug report. We need a way for that to return immediately if the user is trying to capture command output but we know the command is backgrounded.

Mike Miller <mtmiller>
Group Member
Wed 10 Jul 2019 09:23:43 AM UTC, comment #6: 

To answer Mike, yes, the real bug/incompatibility is that "Octave's system function does not return immediately when a command with an ampersand is called and two return values are requested"

John, can you suggest how I might test for the questions you asked about Matlab?

At the very least I know that Matlab launches the commands in a shell on Linux, a shell with their screwed up environment variables in fact, which sometimes screws up loading glibc. I actually wrote a wrapper for the matlab system command which clears LD_LIBRARY_PATH and sources .bashrc again before running the actual requested command to get around this problem. Perhaps this gives some clues to what Matlab is doing?


Richard <crobar>
Tue 09 Jul 2019 11:39:25 PM UTC, comment #5: 

There are a couple of different paths through the system command.  For the one when output is requested and you are on a Unixy system, you end up forking and executing something like


/bin/sh -c "sleep 5 &"


in the child process.  The parent is connected to the stdout from this command using a pipe and tries to read.  It seems like that is blocking, even when the command should be executed in the background.

Maybe there is something simple that we can do, like waiting to see whether the child exits "immediately" (seems like asking for race condition problems to me)?  Or maybe a whole different implementation of system is needed to correctly handle this kind of command consistently for Windows and Unix systems.

What does Matlab do for system commands?  Are they always executed with the native system command interpreter?  On Windows, is command.exe used to execute them?

Is Matlab expecting the command interpreter (either /bin/sh or command.exe) to execute the full "foo &" command, or is Matlab parsing the command to see whether it ends with "&" and then doing something special to execute as a background process?

How compatible do we want to be here?

I'm not sure how best to fix this, but these are some issues to think about if anyone is interested.

John W. Eaton <jwe>
Group administrator
Tue 09 Jul 2019 10:36:52 PM UTC, comment #4: 

Thanks. I take from this that the bug here is that Octave's system function does not return immediately when a command with an ampersand is called and two return values are requested. Do you agree that should be what this bug is about?

IMHO the "async" argument to system is essentially a different function, it is not Matlab compatible by definition.

Mike Miller <mtmiller>
Group Member
Tue 09 Jul 2019 09:51:26 PM UTC, comment #3: 

ok, it's not quite what I thought.

In Matlab the equivalent to 'async' is just to end the command with the ampersand character. In Octave if you do the following:


>> [s, res] = system ('sleep 5 &')
s = 0
res =


The process is not asynchronous, it waits for the result. On matlab it does not, and res is empty, i.e. the following returns immediately:


>> [s, res] = system ('sleep 5 &')

s =

     0.0000e+000


res =

  0×0 empty char array


On Octave however, if you don't ask for the return argument, res, i.e.


>> s = system ('sleep 5 &')
s = 0


It does actually run asynchronously, or at least Octave doesn't seem to wait for the result.

Doing


>> system ('sleep 5 &')


appears to return immediately on both Matlab and Octave.

I think this has confused me into thinking that on Octave, the "async" keyword was required to run a process asynchronously. So there is a subtle incompatibility (if you ask for the second return argument Octave always waits, Matlab doesn't), just not what I thought.

The documentation states that the status command holds the pid of the shell if it is run asynchronously, but this is not the case if you just used the ampersand character to achieve this.


Richard <crobar>
Tue 09 Jul 2019 05:36:46 PM UTC, comment #2: 

The "async" keyword is an Octave extension, as far as I can tell, it is not documented here: https://www.mathworks.com/help/matlab/ref/system.html

So a different return value when the "async" argument is used is not a Matlab compatibility issue. Given that, can you reword this so that you can describe where the Matlab compatibility problem with the system function exists?

Mike Miller <mtmiller>
Group Member
Tue 09 Jul 2019 10:49:57 AM UTC, comment #1: 

I meant third return argument, which could contain pid.

Richard <crobar>
Tue 09 Jul 2019 10:44:20 AM UTC, original submission:  

The Octave 'system' command operates differently from Matlab (on Linux at least, not tried windows) in that when an async process is started it returns the pid of the process in the status output. In contrast Matlab just returns zero if the command was successful.

In principle I like that Octave returns the PID, it's really useful, but it does break code that tests for zero return value, and is not compatible with Matlab. Perhaps a third argument could be added to Octave's 'system' command?

Richard <crobar>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by crobar (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-01-31 mtmiller Release5.1.0 dev
    2019-07-10 mtmiller Severity3 - Normal 2 - Minor
        Priority5 - Normal 3 - Low
        StatusNeed Info Confirmed
        Summary'system' commnad returns pid for async process, not on Matlab system blocks when user calls a background command ending in ampersand
    2019-07-09 mtmiller StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code