bugGNU Octave - Bugs: bug #59952, [octave forge] (parallel) command...

 
 

bug #59952: [octave forge] (parallel) command line function call issues

Submitter:  None
Submitted:  Wed 27 Jan 2021 04:48:36 AM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Need Info Assigned to:  None
Originator Name:  Stavros Kousidis Originator Email:  -email is unavailable-
Open/Closed:  * Closed Release:  * 6.1.0
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sat 13 Feb 2021 10:42:11 PM UTC, comment #6: 

Hi Olaf.

The weird hanging behaviour was occurring when I was running the dev version via `make run`. The correct package appeared loaded in pkg list, so I didn't think the 'other' parallel package would interfere.

However, I removed my standard installation of parallel v4.0.0 as you asked, and I created a tar.gz of the dev version via `make dist` and installed that normally in octave. Now I get the same behaviour as you.

Happy for the bug to be closed, thank you for your patience and diligence. Looking forward to the package release with the fix :)

Tasos Papastylianou <tpapastylianou>
Sat 13 Feb 2021 08:01:41 PM UTC, comment #5: 


> Would you prefer I open a [new] bug report for the 'hanging'
> behaviour?


This could be treated in this report I think. But please double-check that your installation from the development version is not corrupt, i.e not mixed with another installation (e.g. from the released version). E.g. it could be that you call the 'new' parcellfun, but parcellfun calls the 'old' _rfeval_slave_. To be safe, everything from the old installation should be moved out of the way.

Olaf Till <i7tiol>
Group Member
Fri 12 Feb 2021 03:41:18 PM UTC, comment #4: 

Hi Olaf, thanks for checking!

On my machine (linux mint 19.3 / ubuntu 18.04.1), v4.0.0 from octave forge produces the array of -1s as stated earlier, and the development version (tried just now) hangs indefinitely; pressing Ctrl-C dumps the following message:


octave:4> vector_z = pararrayfun( nproc, @(x,y) mul(x,y), vector_x, vector_y)
^Ccould not load variable
could not load variable
error: __rfeval_slave__: could not read argument index
could not load variable
could not receive result
error: __parcellfun_get_next_result__: could not receive result
error: called from
    parcellfun at line 201 column 16
    pararrayfun at line 85 column 28
error: __rfeval_slave__: could not read argument index


So I do not get the same error / behaviour as you, but I do agree that the behaviour on your end would close this particular bug report nicely (though maybe with a slightly more informative error could be useful). Would you prefer I open a [new] bug report for the 'hanging' behaviour?

Tasos Papastylianou <tpapastylianou>
Fri 12 Feb 2021 02:31:57 PM UTC, comment #3: 

With the latest development version of 'parallel', I get:
+vebatim+
octave:1> pkg load parallel
octave:2>
octave:2> function retval = mul(x,y)

>   retval = x*y;
> endfunction

octave:3>
octave:3> vector_x = 1:2^3;
octave:4> vector_y = 1:2^3;
octave:5>
octave:5> vector_z = pararrayfun(nproc, @(x,y) mul(x,y), vector_x, vector_y)
execution error
error: _parcellfun_get_next_result_: could not receive result
error: called from
    parcellfun at line 201 column 16
    pararrayfun at line 85 column 28
octave:6>
-verbatim-

Olaf Till <i7tiol>
Group Member
Wed 10 Feb 2021 10:05:35 PM UTC, comment #2: 

Hi Olaf. I'm the person who answered the stackoverflow question and encouraged Stavros to file this bug.

As I mentioned in that stackoverflow answer, I suspected that the recent move from fork to bespoke workers might be the reason for this behaviour, as you also mentioned, and I agree that it is a reasonable limitation, and that it should be more clearly documented.

However, this was not the only reason I encouraged Stavros to file this as a bug. The other, presumably more important behaviour here which you did not comment on is the fact that pararrayfun does not raise an error gracefully when the command-line function is not suitably defined locally on the worker, but instead this silently returns a -1 as if it were a legitimate return from the function. Presumably this -1 is a return value from a c/c++ function denoting an error, which may be expected in the context of the underlying c calls, but in the context of a legitimate-looking return value to octave instead of an error message, this is a very serious bug. It is also unclear whether such silent error values only happen in this particular scenario, or if this also happens when other errors take place inside a worker.

In my opinion it is this that is the more serious issue in this bug report, and not simply whether commandline-functions are compatible in the first place. (Apologies, I don't know if this is a known issue already).

Tasos Papastylianou <tpapastylianou>
Wed 10 Feb 2021 06:44:02 PM UTC, comment #1: 

This is expected with parallel-4.0.0. I thought this was documented, but reading the docs again, I see that it actually isn't (which should be changed, but this is not the current bug).

'parcellfun()' (and 'pararrayfun()') had to be reworked (with much effort) not to use the 'fork()' system call -- because this causes problems -- but to use separately started Octave processes. Functions defined on the fly, e.g. with the command line interface, don't exist in these separate processes. This was different when 'parcellfun()' used 'fork()'. There is currently no way to get this to work with the current 'parcellfun()' implementation, sorry.

Olaf Till <i7tiol>
Group Member
Wed 27 Jan 2021 04:48:36 AM UTC, original submission:  

I have trouble calling a command line function with pararrayfun (likewise for parcellfun). When I execute the following code:


pkg load parallel

function retval = mul(x,y)
  retval = x*y;
endfunction

vector_x = 1:2^3;
vector_y = 1:2^3;

vector_z = pararrayfun(nproc, @(x,y) mul(x,y), vector_x, vector_y)
vector_z = pararrayfun(nproc, @(x,y) x*y, vector_x, vector_y)


I get the following output:


vector_z =

  -1  -1  -1  -1  -1  -1  -1  -1

vector_z =

    1    4    9   16   25   36   49   64


That is, the call to the command line function does not seem to work, whereas the same as an anonymous function calling the built-in multiplication is working.

The maching is x86_64 with Debian bullseye and 5.10.0-1-amd64 kernel. Octave's version is 6.1.1~hg.2020.12.27-1. The `pkg list` command gives me:


Package Name  | Version | Installation directory
--------------+---------+-----------------------
   dataframe  |   1.2.0 | /usr/share/octave/packages/dataframe-1.2.0
    parallel *|   4.0.0 | /usr/share/octave/packages/parallel-4.0.0
      struct *|  1.0.16 | /usr/share/octave/packages/struct-1.0.16


The same code works flawless on armv7l with Debian buster and 4.14.150-odroidxu4 kernel. That is the call to the command line function and the built-in multiplication function produce the output:


parcellfun: 8/8 jobs done
vector_z =

    1    4    9   16   25   36   49   64

parcellfun: 8/8 jobs done
vector_z =

    1    4    9   16   25   36   49   64


On that machine Octave's version is 4.4.1 and `pkg list` gives:



Package Name  | Version | Installation directory
--------------+---------+-----------------------
   dataframe  |   1.2.0 | /usr/share/octave/packages/dataframe-1.2.0
    parallel *|   3.1.3 | /usr/share/octave/packages/parallel-3.1.3
      struct *|  1.0.15 | /usr/share/octave/packages/struct-1.0.15


This behavior with a workaround has been posted on Stack Overflow here:

https://stackoverflow.com/q/65898175/15081838

It has also been suggested to submit it as a potential bug.

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by tpapastylianou (Posted a comment)
  • -email is unavailable- added by i7tiol (Posted a comment)
  • -email is unavailable- added by siko1056 (Updated the item)
  • -email is unavailable- added by tpapastylianou
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-02-13 i7tiol StatusWont Fix Need Info
    2021-02-10 i7tiol StatusNone Wont Fix
        Open/ClosedOpen Closed
    2021-01-27 siko1056 SummaryOctave parallel package: Command line function call issues [octave forge] (parallel) command line function call issues
    2021-01-27 tpapastylianou Carbon-Copy- Added tpapastylianou

    Back to the top

    Powered by Savane 3.13-4b48.
    Corresponding source code