bugGNU Octave - Bugs: bug #49211, "test syscall" often...

 
 

bug #49211: "test syscall" often hangs on Windows, needs reliable waitpid implementation

Submitted by:  Hartmut <hardy>
Submitted on:  Wed 28 Sep 2016 03:58:03 PM UTC  
 
Category:  Octave Function Severity:  3 - Normal
Priority:  3 - Low Item Group:  Regression
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  Closed
Release:  dev Operating System:  Microsoft Windows

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Sun 14 Jun 2020 08:12:09 PM UTC, comment #13: 

No reported issues after a few weeks.
Closing as fixed.

Markus Mützel <mmuetzel>
Project Member
Sun 24 May 2020 08:02:19 PM UTC, comment #12: 

We are early in the Octave 7 cycle, so I pushed the change here:
https://hg.savannah.gnu.org/hgweb/octave/rev/abbab3ed2a5f

In case this should cause trouble, we could still revert.

Marking as ready for test.

Markus Mützel <mmuetzel>
Project Member
Fri 22 May 2020 11:09:59 AM UTC, comment #11: 

Looks like I didn't attach the patch last time.

Re-reading the comments again, I'm no longer sure if I correctly got the point of this bug.
Should "waitpid" implicitly close the input and output stream of the process and thereby indicate that the child process can terminate? Otherwise, I would expect it to block indefinitely if the child process never terminates.
It looks like the general expectation is that it should not block in this case...
What is the expected behavior?

(file #49145)

Markus Mützel <mmuetzel>
Project Member
Thu 21 May 2020 03:45:40 PM UTC, comment #10: 

The attached patch implements a wrapper for waitpid on Windows using win32 API functions.

It works for me using the following test case:

clear
more off
for n = 1:10
  disp ("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
  disp (num2str (n));
  disp ("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
  [in, out, pid] = popen2 ('C:\Windows\system32\sort.exe', "/R")
  fclose (in);  # important or the process never stops
  disp('before waitpid')  # ==> no longer hangs here
  waitpid (pid)
  disp('after waitpid')
  fclose (out);
endfor

Also calling the originally motivating test in a loop no longer hangs for me:

clear
more off
cd (fullfile (OCTAVE_HOME, 'share/octave/7.0.0/etc/tests/libinterp/corefcn'))
for n = 1:10
  disp ("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
  disp (num2str (n));
  disp ("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
  test("syscalls.cc-tst", "verbose")
endfor

I'm not sure if this still qualifies as a regression because the original issue no longer occurs since changeset 610f88ed2b78 (comment #7). So this should probably go to default imho.

Markus Mützel <mmuetzel>
Project Member
Fri 07 Oct 2016 01:54:44 AM UTC, comment #9: 
Mike Miller <mtmiller>
Project Administrator
Thu 06 Oct 2016 09:48:56 PM UTC, comment #8: 

I'm not sure that WaitForInputIdle is what we are looking for as a replacement for waitpid, which is supposed to check the status of a process, typically to wait until it exits.  It looks like WaitForInputIdle will tell you if a running process is idle and ready for input.

John W. Eaton <jwe>
Project Administrator
Thu 06 Oct 2016 07:43:43 PM UTC, comment #7: 

For now, I checked in the following change:

http://hg.savannah.gnu.org/hgweb/octave/rev/610f88ed2b78

John W. Eaton <jwe>
Project Administrator
Thu 29 Sep 2016 07:20:39 PM UTC, comment #6: 

Bleah!  I think that Octave needs it's own function wrapper around specific MS API functions.  This is what we do for popen2 where we call things like CreateProcess

  status = CreateProcess (0, command, 0, 0, TRUE, 0, 0, 0, &si, &pi);

Documentation for CreateProcess is here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx.

It seems that maybe WaitForInputIdle might be a companion function: https://msdn.microsoft.com/en-us/library/windows/desktop/ms687022%28v=vs.85%29.aspx

This report clearly needs someone who both understands and is interested in the Windows API.  I can't help much here because I code on Linux.

Rik <rik5>
Project Administrator
Thu 29 Sep 2016 06:58:57 PM UTC, comment #5: 

Right, which is exactly what we used to do (waitpid callable but non-functional).

I did some very light searching on cwait / _cwait, and MSDN has a comment saying that it is only supported in CRT (console) applications, not in desktop applications (if anyone understands this jargon). This may mean that the gnulib waitpid wrapper is no good for our purposes and we may have to write our own wrapper around a function like WaitForSingleObject.

Mike Miller <mtmiller>
Project Administrator
Thu 29 Sep 2016 06:10:16 PM UTC, comment #4: 

@Mike: you're right.  I just tried with version 4.0.2 that I had lying around.

[a,b,c] = waitpid (-1)
a = -1
b = 0
c = waitpid: not supported on this system

One choice, if we can't get a gnulib wrapper to work, would be to simply let the child zombie army grow until it is finally reaped when Octave is closed.

Rik <rik5>
Project Administrator
Thu 29 Sep 2016 05:33:45 PM UTC, comment #3: 

Well yes, in Octave 4.0.3 waitpid is not implemented (if I'm reading the source right).

Try

>> [pid, status, msg] = waitpid (-1)
pid = -1
status = 0
msg = No child processes

in Octave 4.0.3. The error message should say "waitpid: not supported on this system" and other than that the function will do nothing.

In Octave 4.2 it uses gnulib's waitpid wrapper around the _cwait function.

Mike Miller <mtmiller>
Project Administrator
Thu 29 Sep 2016 11:42:31 AM UTC, comment #2: 

Has the waitpip command from gnulib changed between Octave 4.0.3 and 4.2.0-rc2 ? (Octave 4.0.3 does not show those hang-ups, yet.) This might give an indication what to change (back)?

Hartmut <hardy>
Thu 29 Sep 2016 03:59:39 AM UTC, comment #1: 

Confirmed.  I think the problem is that Windows just doesn't really have the same notion of children, process IDs, etc.

I played around with your script and waitpid, even when it returns, doesn't seem to do anything.  I used the Process Monitor in Windows and I could see that there were still 'sort' processes left over.

What I did find that worked was using

fclose (in);
fclose (out);

At that point, I got no extra sort processes.

Maybe we need a waitpid process that is specifically written for Windows.  The popen2 call is written for Windows, but waitpid is coming from gnulib.

Rik <rik5>
Project Administrator
Wed 28 Sep 2016 03:58:03 PM UTC, original submission:  

I have observed that with the current Octave-4.2.0-rc2 the "test syscall" sometimes hangs under Windows OS. (With Octave 4.0.3 this still runs fine, its a regression)

I was able to reduce the underlying test code to a very simple example, that nearly always creates a "freeze in" of the Octave process on my Windows system:

clear
more off
#cd "C:/Octave/Octave-4.2.0-rc2/share/octave/4.2.0-rc2/etc/tests/libinterp/corefcn"
for n = 1:10
    disp("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
    disp(num2str(n));
    disp("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
    # test("syscalls.cc-tst", "verbose")
    # use test code (Win version) directly:
    # (removed many code lines, but "preserve" the hang-up)
   [in, out, pid] = popen2 ('C:\Windows\system32\sort.exe', "/R")
   disp('before waitpid')  # ==> hangs here = during waitpid()
   waitpid (pid)
   disp('after waitpid')
endfor

I have the gut feeling, that this bug could have the same origin as several other bugs reported recently for 4.2.0-rc2 under Windows:

  • a "hang-up" of "test copyobj", using gnuplot and the print command (bug #49179)
  • occasional "hang-ups" of "test legend"  (mentioned somewhere in the bug tracker)
  • "hang ups" during compare_plot_demos with gnuplot (bug #49180)

The common point is that all those freezing tests call another Windows executable. Since all the other "freezes" have much more difficult underlying code, my hope is that this very short test script can help to remove all of those regression bugs at once.

Hartmut <hardy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #49145:  bug49211_waitpid_win32.patch added by mmuetzel (2KiB - application/octet-stream)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by hardy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-06-14 mmuetzel StatusReady For Test => Fixed
        Open/ClosedOpen => Closed
    2020-05-24 mmuetzel StatusPatch Submitted => Ready For Test
    2020-05-22 mmuetzel Attached File- => Added bug49211_waitpid_win32.patch, #49145
    2020-05-21 mmuetzel StatusConfirmed => Patch Submitted
    2016-11-17 mtmiller Dependencies- => bugs #49179 is dependent
    2016-11-17 mtmiller Priority5 - Normal => 3 - Low
        Release4.2.0-rc2 => dev
        Summary"test syscall" often hangs on Windows => "test syscall" often hangs on Windows, needs reliable waitpid implementation
    2016-09-29 rik5 StatusNone => Confirmed

    Back to the top


    Powered by Savane 3.5