bugGNU Octave - Bugs: bug #59869, [octave forge] (parallel) tests...

 
 

bug #59869: [octave forge] (parallel) tests trigger "int32 scalar cannot be indexed with" on systems with 1 CPU

Submitter:  Christian Ehrhardt <paelzer>
Submitted:  Thu 14 Jan 2021 07:46:35 AM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Regression
Status:  None Assigned to:  None
Originator Name:  Christian Ehrhardt Open/Closed:  * Open
Release:  * other Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 15 Jan 2021 09:09:11 AM UTC, comment #9: 

I think it is essential to know the Octave release in which the bug was triggered, even if the bug is in a package. ('parallel' e.g. tries to consider many configuration aspects related to the Octave version.)

Olaf Till <i7tiol>
Group Member
Fri 15 Jan 2021 07:41:25 AM UTC, comment #8: 

I changed the release from 6.1.0 to other because we usually tag the release only for bugs in Octave core. The "release" field is usually set to "other" for bugs in Octave Forge packages.
At the moment, I'd guess that this is an issue in the package (not in Octave core). In case it should turn out differently, we can still re-tag the release.

Markus Mützel <mmuetzel>
Group administrator
Thu 14 Jan 2021 07:46:36 PM UTC, comment #7: 


> I can't reproduce it, so I depend on you doing debugging.


I can try to help as long as there are clear instructions what I can do. But really - to reproduce just use a single-cpu system (or if you have no KVM or cloud to do so, just reboot with maxcpus=1 - that does the same.
 

> Since their are containers involved, some notes:


No containers involved, those were just listed as not reproducing environments initially. KVM guests are the easiest, just give them only one vCPU and you have a reproducing environment.

> - configure, make, and running must happen in the same environment with respect to system- and octave- libraries.


All packages built with the same build dependencies in Debian/Ubuntu. And since they built close to each other those where mostly identical.
 

> - I don't know if container systems can trick exec() ...


As I said, if you read in detail no container(magic) needed. Just 1 CPU and this should reproduce

> Further notes:
>
> You can look if the spawned octave processes result from the same Octave binary as the spawning process.


I can look what binary spawns (when I have more time = tomorrow). But what exactly do you want me to check or report there? Do you want me to verify if spawning&spawned processes are all from the same binary?

> The Octave release was changed from 6.1 to 'other' -- at what release did the issue happen?


As I outlined in the report 6.1.1 is right and octave-parallel 4.0 - not sure what/who/what changed the release - but it was not intentional.

Christian Ehrhardt <paelzer>
Thu 14 Jan 2021 05:09:25 PM UTC, comment #6: 

Well, no need to add me...

I can't reproduce it, so I depend on you doing debugging. Setting debug_on_error(true) and looking what is wrong at this position would be a start.

Since their are containers involved, some notes:

- configure, make, and running must happen in the same environment with respect to system- and octave- libraries.

- I don't know if container systems can trick exec() to run a program outside the current container, but if yes, this should not take place in spawning octave processes by parallel.

Further notes:

You can look if the spawned octave processes result from the same Octave binary as the spawning process.

The Octave release was changed from 6.1 to 'other' -- at what relese did the issue happen?

Olaf Till <i7tiol>
Group Member
Thu 14 Jan 2021 12:23:53 PM UTC, comment #5: 

Wrong user name. Trying to add Olaf Till again.

Markus Mützel <mmuetzel>
Group administrator
Thu 14 Jan 2021 12:19:00 PM UTC, comment #4: 

Sorry. That was a long shot admittedly.

IIUC, the error occurs because for some reason `__parcellfun_get_next_result__` returns an int32 instead of a cell array as the second output argument.

CC'ing Olaf Till as the package maintainer. He'll probably understand this much better than me.

Markus Mützel <mmuetzel>
Group administrator
Thu 14 Jan 2021 12:09:11 PM UTC, comment #3: 
Christian Ehrhardt <paelzer>
Thu 14 Jan 2021 11:39:56 AM UTC, comment #2: 


comment #1:

> Could you try building the parallel package from its repository (or cherry-pick that change on top of parallel 4.0.0)?


Yeah sure, I'm on it already!
It applies and builds cleanly - just a bit more time for the tests to complete execution.

Christian Ehrhardt <paelzer>
Thu 14 Jan 2021 09:36:44 AM UTC, comment #1: 

This might be fixed in the parallel repository already here:
https://sourceforge.net/p/octave/parallel/ci/ae48f019ea37ee68abe07ce067be5795707895bd/

Could you try building the parallel package from its repository (or cherry-pick that change on top of parallel 4.0.0)?

Markus Mützel <mmuetzel>
Group administrator
Thu 14 Jan 2021 07:46:35 AM UTC, original submission:  

Hi,
I was tracking an issue of octave (6.1.1~hg.2020.12.27-3) and octave-parallel (4.0.0-2) which continuously started to always fail [1] in the Ubuntu autopkgtest infrastructure [2].

The error looked like:


[inst/pararrayfun.m]
>>>>> /tmp/autopkgtest.hf2ucm/build.AhK/src/inst/pararrayfun.m
...
 assert (res = pararrayfun (2, @ (x, y) x * y, [1, 2, 3, 4], [2, 3, 4, 5], "ChunksPerProc", 2, "CumFunc", @ (a, b) a + b), 40)
!!!!! test failed
int32 scalar cannot be indexed with {
5 tests, 4 passed, 0 known failure, 0 skipped


The biggest problem with that initially was that it only triggered in our autopkgtest environment.
It happened not on one, but on all VM based architectures x86 + arm64 + ppc64el + s390x.
If I run the same code in Ubuntu or Debian in:
- local containers
- or all DebCI runs which are in containers
- or Ubuntu's armhf runs which are in containers
- on bare metal
It all works just fine and as expected.

Way more debug data and steps that were taken in between can be found on the Ubuntu bug tracker for this at [3]. Looking back some of those steps are hilarious, but afterwards you are always smarter :-)

I was eventually able to recreate the issue on a KVM guest in the same Openstack cloud that does our test runs. And I realized that if I run it in a local KVM guest it is failing as well IFF I give it only one CPU.

So it seems the test needs at least 2 CPUs to not fail.

I was trying to further corner the root cause and found that - on the affected systems - the following is enough to reproduce the issue:


# on Ubuntu hirsute (21.04) or Debian-sid do
$ apt install octave-parallel
$ octave-cli
octave:1> pkg ('load', 'parallel');
octave:2> pararrayfun (2, @ (x, y) x * y, [1, 2, 3, 4], [2, 3, 4, 5], "ChunksPerProc", 2, "CumFunc", @ (a, b) a + b), 40
 error: int32 scalar cannot be indexed with {
 error: called from
     parcellfun at line 206 column 25
     chunk_parcellfun at line 47 column 25
     parcellfun at line 142 column 28
     pararrayfun at line 85 column 28


Dropping the 2 after ChunksPerProc to 1 makes it not fail that bad.

 octave:6> pararrayfun (1, @ (x, y) x * y, [1, 2], "ChunksPerProc", 1, "CumFunc", @ (a, b) a + b), 40
 ans = 40


It could be related to the min of "2" CPUs that I've found.
But TBH - I don't really know what I'm changing here.


I've found that when downgrading to the former Debian/Ubuntu versions being:
- octave=5.2.0-3build1
- octave-parallel=3.1.3-4build1
- octave-common=5.2.0-3build1
- octave-struct=1.0.16-6
Then the test as run from octave parallel works even on just a one CPU system:


octave:2> assert (res = parcellfun (2, @ (x, y) x * y, {1, 2, 3, 4}, {2, 3, 4, 5}, "ChunksPerProc", 2, "CumFunc", @ (a, b) a + b), 40)
parcellfun: 4/4 jobs done


Therefore I'd classify it as a regression unless you think otherwise.

Would it be possible to change the test in a way to not fail on those small systems? Or if this is considered a bug in octave/octave-parallel fix it there=

After an discussion on IRC I was asked to file a bug which I hereby do. The test is easy for me and I'm to be able to create affected (in our cloud) and unaffected (local VMs) systems as needed. So if there is a good suggestion where to look at - let me know.

[1]: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-hirsute/hirsute/amd64/o/octave-parallel/20210112_151852_47229@/log.gz
[2]: https://wiki.ubuntu.com/ProposedMigration/AutopkgtestInfrastructure
[3]: https://bugs.launchpad.net/ubuntu/+source/octave-parallel/+bug/1911400

Christian Ehrhardt <paelzer>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by i7tiol (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by paelzer (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-01-14 mmuetzel Carbon-Copy- Added i7tiol
    2021-01-14 mmuetzel Carbon-CopyRemoved otiol -
    2021-01-14 mmuetzel Release6.1.0 other
        Carbon-Copy- Added otiol
    2021-01-14 mmuetzel Summaryoctave-parallel tests trigger &quot;int32 scalar cannot be indexed with&quot; on systems with 1 CPU [octave forge] (parallel) tests trigger "int32 scalar cannot be indexed with" on systems with 1 CPU

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code