bugGNU Octave - Bugs: bug #53557, pkg.tst should not write to the...

 
 

bug #53557: pkg.tst should not write to the test directory or the user's home directory

Submitter:  Rik <rik5>
Submitted:  Wed 04 Apr 2018 06:21:19 PM UTC
   
 
Category:  Test Suite Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * GNU/Linux
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Fri 06 Apr 2018 08:41:51 PM UTC, comment #10: 

I made an additional change on the default branch to fix this for me. I can now run the tests with either a nonexistent HOME directory or with my own HOME directory and my existing pkg configuration is not affected.

There are still some other issues with pkg writing to HOME when it shouldn't, but that can be handled later.

https://hg.savannah.gnu.org/hgweb/octave/rev/ca20eb3d0535

I also had to make a further change related to bug #53567 on the stable branch for the tests to pass for me.

Mike Miller <mtmiller>
Group Member
Fri 06 Apr 2018 12:18:59 AM UTC, comment #9: 

@Mike: It's worth a shot.  It would be nice if saving/restoring local_list was the only thing necessary.

Rik <rik5>
Group administrator
Fri 06 Apr 2018 12:13:36 AM UTC, comment #8: 

I don't think the tests are the issue.  I think they probably just expose the fact that the underlying pkg routines depend on a HOME variable that points to a directory with read access.

FWIW, when I use run-octave with a non-existent or non-writeable directory I get a Qt message box that pops up with


Configuration file "/tmp/tmphome/.config/lt-octave-guirc" not writable.
Please contact your system administrator.


The command then hangs.

Rik <rik5>
Group administrator
Fri 06 Apr 2018 12:07:39 AM UTC, comment #7: 

If I don't set HOME to a nonexistent directory, and if a test fails, then I am left with 'mfile_basic_test' looking like an installed package in my normal package cache file. When I start Octave again I have to run 'pkg rebuild' to get the bogus package out of my cache file.

Setting and restoring pkg("local_list") just like pkg("prefix") might be the only missing piece here, I'll test that if I get a chance.

Mike Miller <mtmiller>
Group Member
Thu 05 Apr 2018 11:53:25 PM UTC, comment #6: 

I just built 2fcd829a6878, and this doesn't look fixed to me. I get four test failures and plenty of messages about failure to write to HOME:


$ HOME=/nonexistent ./run-octave --no-history --norc --silent --eval "__octave_config_info__ hg_id"
ans = 2fcd829a6878
$ HOME=/nonexistent ./run-octave --no-history --norc --silent --eval "chdir ('../test/pkg'); [a,b,c,d,e,f] = test ('pkg.tst', 'quiet', stdout)"
>>>>> processing pkg.tst
***** test
 for i = 1:numel (mfile_pkg_name)
   silent_pkg_install (mfile_pkg_zip{i});
   pkg ("uninstall", mfile_pkg_name{i});
 endfor

!!!!! test failed
save: unable to open output file '/nonexistent/.octave_packages'
shared variables   scalar structure containing the fields:

    old_prefix = /nonexistent/octave
    old_archprefix = /nonexistent/octave
    prefix = /run/user/1000/tmpdir/oct-bnIcCf
    restorecfg =

onCleanup (@() pkg ("prefix", old_prefix, old_archprefix))

    rmtmpdir =

onCleanup (@() confirm_recursive_rmdir (0, "local") && rmdir (prefix, "s"))

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_basic_test.zip
      [1,2] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_minimal_test.zip
    }

***** test
 for i = 1:numel (mfile_pkg_name)
   silent_pkg_install ("-local", mfile_pkg_zip{i});
   pkg ("uninstall", mfile_pkg_name{i});
 endfor
!!!!! test failed
save: unable to open output file '/nonexistent/.octave_packages'
shared variables   scalar structure containing the fields:

    old_prefix = /nonexistent/octave
    old_archprefix = /nonexistent/octave
    prefix = /run/user/1000/tmpdir/oct-bnIcCf
    restorecfg =

onCleanup (@() pkg ("prefix", old_prefix, old_archprefix))

    rmtmpdir =

onCleanup (@() confirm_recursive_rmdir (0, "local") && rmdir (prefix, "s"))

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_basic_test.zip
      [1,2] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_minimal_test.zip
    }

***** test
 for i = 1:numel (mfile_pkg_name)
  name = mfile_pkg_name{i};
  silent_pkg_install ("-local", mfile_pkg_zip{i});
  unwind_protect
    pkg ("load", name);
    pkg ("unload", name);
  unwind_protect_cleanup
    pkg ("uninstall", name);
  end_unwind_protect
 endfor

!!!!! test failed
save: unable to open output file '/nonexistent/.octave_packages'
shared variables   scalar structure containing the fields:

    old_prefix = /nonexistent/octave
    old_archprefix = /nonexistent/octave
    prefix = /run/user/1000/tmpdir/oct-bnIcCf
    restorecfg =

onCleanup (@() pkg ("prefix", old_prefix, old_archprefix))

    rmtmpdir =

onCleanup (@() confirm_recursive_rmdir (0, "local") && rmdir (prefix, "s"))

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_basic_test.zip
      [1,2] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_minimal_test.zip
    }

***** test
 silent_pkg_install ("-local", mfile_pkg_zip{1});
 [desc, flag] = pkg ("describe", mfile_pkg_name{1});
 ## FIXME: this only tests that the describe command runs,
 ##        not that the output is in anyway correct.
 pkg ("uninstall", mfile_pkg_name{1});
!!!!! test failed
save: unable to open output file '/nonexistent/.octave_packages'
shared variables   scalar structure containing the fields:

    old_prefix = /nonexistent/octave
    old_archprefix = /nonexistent/octave
    prefix = /run/user/1000/tmpdir/oct-bnIcCf
    restorecfg =

onCleanup (@() pkg ("prefix", old_prefix, old_archprefix))

    rmtmpdir =

onCleanup (@() confirm_recursive_rmdir (0, "local") && rmdir (prefix, "s"))

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_basic_test.zip
      [1,2] = /run/user/1000/tmpdir/oct-bnIcCf/mfile_minimal_test.zip
    }

warning: onCleanup: error caught while executing cleanup function:
pkg: cannot create prefix /nonexistent/octave: Permission denied


Mike Miller <mtmiller>
Group Member
Thu 05 Apr 2018 05:56:08 PM UTC, comment #5: 

I changed the pkg testing script to use a temporary directory writeable by the user which is cleaned up at the end of testing.  See http://hg.savannah.gnu.org/hgweb/octave/rev/2fcd829a6878.

Marking as fixed and closing report.

Rik <rik5>
Group administrator
Wed 04 Apr 2018 07:40:17 PM UTC, comment #4: 

In fact the test directory itself should not be written to either. The '__run_test_suite__' function chdirs to the test directory when running the fixed test scripts, and that directory may not be writable to the user.

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 07:37:20 PM UTC, comment #3: 

Tests should not depend on a writable HOME directory at all. I think pkg.tst should be rewritten to do all of its work in a temporary directory with pkg prefix set accordingly.

I occasionally build and run the Octave test suite with HOME set to a nonexistent directory, all tests should pass under this condition.

Mike Miller <mtmiller>
Group Member
Wed 04 Apr 2018 07:01:48 PM UTC, comment #2: 

/home/buildbotu/octave

is a bad directory. There are multiple buildbot slaves
 on this computer and they all share the same home directory.
So if you write something to $HOME/octave/ you going to have a problem.  The directory should be specific to the build.

Dmitri.
--


Dmitri A. Sergatskov <dasergatskov>
Wed 04 Apr 2018 06:31:26 PM UTC, comment #1: 

Duh, I found pkg.tst in the directory test/pkg/pkg.tst.  I've been working so much on the stable branch preparing for the release and it isn't a part of 4.4.

Adding Juan Pablo to the CC list.


Rik <rik5>
Group administrator
Wed 04 Apr 2018 06:21:19 PM UTC, original submission:  

The file pkg.tst is failing tests.  For example, see this buildbot: http://buildbot.octave.org:8010/builders/clang-fedora/builds/1444/steps/test/logs/stdio.


>>>>> processing pkg.tst
***** test
 for i = 1:numel(mfile_pkg_name)
  silent_pkg_install ({'-local', mfile_pkg_zip{i}});
  pkg ('uninstall', mfile_pkg_name{i});
 endfor
!!!!! test failed
couldn't create installation directory /home/buildbotu/octave/mfile_basic_test-0.1.0 : File exists
shared variables   scalar structure containing the fields:

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = mfile_basic_test.zip
      [1,2] = mfile_minimal_test.zip
    }

    silent_pkg_install =

@(args) evalc (sprintf (['pkg install', repmat(' %s', 1, numel (args))], args {:}))

***** test
 for i = 1:numel(mfile_pkg_name)
  name = mfile_pkg_name{i};
  silent_pkg_install ({'-local', mfile_pkg_zip{i}});
  unwind_protect
    pkg ('load', name);
    pkg ('unload', name);
  unwind_protect_cleanup
    pkg ('uninstall', name);
  end_unwind_protect
 endfor
!!!!! test failed
couldn't create installation directory /home/buildbotu/octave/mfile_basic_test-0.1.0 : File exists
shared variables   scalar structure containing the fields:

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = mfile_basic_test.zip
      [1,2] = mfile_minimal_test.zip
    }

    silent_pkg_install =

@(args) evalc (sprintf (['pkg install', repmat(' %s', 1, numel (args))], args {:}))

***** test
 silent_pkg_install ({'-local', mfile_pkg_zip{1}});
 [desc, flag] = pkg ('describe', mfile_pkg_name{1});
 pkg ('uninstall', mfile_pkg_name{1});
!!!!! test failed
couldn't create installation directory /home/buildbotu/octave/mfile_basic_test-0.1.0 : File exists
shared variables   scalar structure containing the fields:

    mfile_pkg_name =
    {
      [1,1] = mfile_basic_test
      [1,2] = mfile_minimal_test
    }

    mfile_pkg_zip =
    {
      [1,1] = mfile_basic_test.zip
      [1,2] = mfile_minimal_test.zip
    }

    silent_pkg_install =

@(args) evalc (sprintf (['pkg install', repmat(' %s', 1, numel (args))], args {:}))


However, I can't find the file pkg.tst in my local build directory.  Where are the buildbots getting this file?

Rik <rik5>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rik5
  • -email is unavailable- added by rik5 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 10 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-04-06 mtmiller StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2018-04-05 mtmiller StatusFixed Confirmed
        Open/ClosedClosed Open
    2018-04-05 rik5 StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2018-04-04 mtmiller StatusNone Confirmed
        Summarypkg.tst failing on some buildbots pkg.tst should not write to the test directory or the user's home directory
    2018-04-04 mtmiller Summarypkg.test failing on some buildbots pkg.tst failing on some buildbots
    2018-04-04 rik5 Carbon-Copy- Added juanpi

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code