bugGNU Octave - Bugs: bug #60360, [octave forge] (parallel) fills up...

 
 

bug #60360: [octave forge] (parallel) fills up RAM, then crashes

Submitted by:  None
Submitted on:  Thu 08 Apr 2021 12:49:51 PM UTC  
 
Category:  Octave Forge Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  None
Status:  None Assigned to:  None
Originator Name:  Matthias Brennwald Originator Email:  -email is unavailable-
Open/Closed:  Open Release:  6.2.0
Operating System:  GNU/Linux

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 08 Apr 2021 12:49:51 PM UTC, original submission:  

The pararrayfun function seems to start too many parallel processes, such that RAM fills up, and processes will fail.

Here's a simple example to illustrate the problem:

function y = paral_func(N,x);
        y = x + sum(sum(abs(fft(rand(N)))))/N^2;
endfunction

Nproc = nproc;
y = pararrayfun(Nproc, @(x) paral_func(10000,x), 1:100)

On my machine (ncore = 8 and 16 GB of RAM) I get this:

could not save variable.
could not save variable.
could not save variable.
could not save variable.
...
could not load variable
could not receive result
error: __parcellfun_get_next_result__: could not receive result
error: called from
    parcellfun at line 199 column 16
    pararrayfun at line 85 column 28
    paral0 at line 2 column 3

I guess parcellfun could just stop hammering out new processes once it runs into this problem, and try again when one of the previously started processes returns and has released its memory.

Or is it just me doing something wrong?

Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-04-08 mmuetzel Carbon-CopyRemoved 102357 => -
    2021-04-08 mmuetzel Summary[octave forge] (package_name): fills up RAM, then crashes => [octave forge] (parallel) fills up RAM, then crashes

    Back to the top


    Powered by Savane 3.6