bugGNU Parallel - build and execute command lines from standard input in parallel - Bugs: bug #42336, --noswap is not usable on OS X due...

 
 

bug #42336: --noswap is not usable on OS X due to vmstat/vm_stat issue

Submitted by:  Zhiming Wang <zmwang>
Submitted on:  Sun 11 May 2014 09:36:10 PM UTC  
 
Category: NoneSeverity: 3 - Normal
Item Group: NoneStatus: Fixed
Privacy: PublicAssigned to: Ole Tange <tange>
Open/Closed: Closed

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Wed 04 Jun 2014 04:09:20 PM UTC, comment #7:

Yeah, I was able to force the system to swap. --noswap worked in the expected way, at least for some simple usage cases.

Zhiming Wang <zmwang>
Wed 04 Jun 2014 09:47:07 AM UTC, comment #6:

Force swapping:

perl -ne 'BEGIN {$a=`cat semibig.iso`} push(@b,$a)'

Each press of <enter> will take up memory of the size semibig.iso.

This should work on every platform that can run GNU Parallel.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Wed 04 Jun 2014 01:47:58 AM UTC, comment #5:

seems to work for me. Bottom line: --noswap won't get stuck anymore when the system is not swapping.

I had a hard time forcing the system to swap, though. Apple has introduced some new memory management features like memory compression since 10.9, and now it's pretty hard to make it swap. Simply mallocing massive blocks won't work. Running several virtual machines at the same time did not make it swap either. Not sure how to proceed with the test, but I suppose it's okay now.

Zhiming Wang <zmwang>
Wed 04 Jun 2014 12:03:18 AM UTC, comment #4:

Fixed in [47919ddb97b].

Please test.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Mon 12 May 2014 07:43:03 AM UTC, comment #3:

An example output of

is

As you already know from the man page, vm_stat 1 2 is wrong.

Zhiming Wang <zmwang>
Mon 12 May 2014 07:40:39 AM UTC, comment #2:

According to https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/vm_stat.1.html

the correct command might be:

vm_stat -c 2 1

So provide the output of that, too.

Ole Tange <tange>
Project AdministratorIn charge of this item.
Mon 12 May 2014 07:38:17 AM UTC, comment #1:

I have never used vm_stat on a mac.

What I need is not the 3rd line from vmstat, but the 4th:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
4 0 100156 13679368 205072 3573280 0 0 51 176 32 24 66 21 13 0
1 0 100156 13679280 205072 3573308 0 0 0 60 5825 13679 13 2 85 0

What is the output of 'vm_stat 1 2' on a mac?

Ole Tange <tange>
Project AdministratorIn charge of this item.
Sun 11 May 2014 09:36:10 PM UTC, original submission:

This issue was originally discussed in #35523, but it was closed without being fixed.

On OS X there is no vmstat, but rather vm_stat, and its output is very different. For instance:

vm_stat on OS X:

vmstat on Ubuntu:

So --noswap in its current shape is not usable in OS X. Hope this get fixed, which I suppose shouldn't be much harder than checking uname plus some custom line editing for OS X. Thanks.

Zhiming Wang <zmwang>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by tange (Posted a comment)
  • -unavailable- added by zmwang (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 05 Jun 2014 07:54:30 AM UTCtangeStatusNeed Info=>Fixed
      Open/ClosedOpen=>Closed
    Mon 12 May 2014 07:38:17 AM UTCtangeStatusNone=>Need Info
      Assigned toNone=>tange

    Back to the top


    Powered by Savane 3.1-cleanup1