bugGNU Octave - Bugs: bug #60360, [octave forge] (parallel) fills up...

 
 

bug #60360: [octave forge] (parallel) fills up RAM, then crashes

Submitter:  None
Submitted:  Thu 08 Apr 2021 12:49:51 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  None Assigned to:  None
Originator Name:  Matthias Brennwald Originator Email:  -email is unavailable-
Open/Closed:  * Open Release:  * 6.2.0
Operating System:  * GNU/Linux Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 08 Jun 2021 05:17:33 AM UTC, comment #1: 

I can't reproduce this with parallel-4.0.1 on Octave-6.2.0. And I can't see that the given error message has something to do with starting too many processes filling up the RAM. It rather looks like an error message due to a bug of a previous package version to me.

Olaf Till <i7tiol>
Group Member
Thu 08 Apr 2021 12:49:51 PM UTC, original submission:  

The pararrayfun function seems to start too many parallel processes, such that RAM fills up, and processes will fail.

Here's a simple example to illustrate the problem:


function y = paral_func(N,x);
        y = x + sum(sum(abs(fft(rand(N)))))/N^2;
endfunction

Nproc = nproc;
y = pararrayfun(Nproc, @(x) paral_func(10000,x), 1:100)


On my machine (ncore = 8 and 16 GB of RAM) I get this:

could not save variable.
could not save variable.
could not save variable.
could not save variable.
...
could not load variable
could not receive result
error: __parcellfun_get_next_result__: could not receive result
error: called from
    parcellfun at line 199 column 16
    pararrayfun at line 85 column 28
    paral0 at line 2 column 3


I guess parcellfun could just stop hammering out new processes once it runs into this problem, and try again when one of the previously started processes returns and has released its memory.

Or is it just me doing something wrong?

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by i7tiol (Posted a comment)
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-04-08 mmuetzel Carbon-CopyRemoved 102357 -
    2021-04-08 mmuetzel Summary[octave forge] (package_name): fills up RAM, then crashes [octave forge] (parallel) fills up RAM, then crashes

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code