bugGNU Octave - Bugs: bug #47405, pkg update errors out if one...

 
 

bug #47405: pkg update errors out if one package sees a dependency error

Submitter:  Nicholas Jankowski <nrjank>
Submitted:  Mon 14 Mar 2016 03:22:21 AM UTC
   
 
Category:  Octave Function Severity:  2 - Minor
Priority:  5 - Normal Item Group:  Installation Failure
Status:  Confirmed Assigned to:  None
Originator Name:  Nicholas Jankowski Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 10 Jul 2022 12:37:50 AM UTC, comment #8: 

I wrote a small helper whose code and output you can see here: https://octave.discourse.group/t/welcome-octave-packages/2954/17

Right now it only prints the commands but executing them is only an eval() away.

If the user wants only one package, say the econometrics one,  then the line for econometrics says to install econometrics only after installing io, statistics, struct, and optim in that order.

Where should that sort of dependency-scanning functionality be absorbed?

Arun Giridhar <arungiridhar>
Group Member
Thu 02 Dec 2021 01:08:05 AM UTC, comment #7: 

I'm cleaning up a patch to bug #60574 that will allow 'pkg update' to accept the same options as install, including -nodeps.  that will at least allow this bug to be sidestepped for now, but at the risk of updating something that breaks because of dependencies. but yes, a smarter system would be better.

Nicholas Jankowski <nrjank>
Group Member
Wed 01 Dec 2021 10:17:21 PM UTC, comment #6: 

I forgot a bit about my work on pkg.m a while ago (about dependencies and inverse dependencies, etc). At the time I did think about this, and from what I remember it shouldn't be to hard to more or less do the update work safely by implicitly following the dependency tree, w/o actually deriving it. In comment #1 I really over-engineered it.
The private functions installed_packages.m and get_inverse_dependencies.m do the required work. Here's what I was thinking:

Step 1:
'pkg update' could start with updating all packages that have no dependencies; easily derived from the info in the 'installed_pkgs_list' struct created by installed_packages.m. That'll take care of the ultimate "leafs" of the dependency tree.
Most packages have no inverse dependencies so for those the story ends there; they can be marked as 'done'.

Step 2:
Next, get_inverse_dependencies.m could find out which packages directly depend on the ones that are already updated in step 1. Updating those should be a safe action and would take care of a level higher up in the dependency tree. The packages updated in step 1 that hadn't yet been marked as 'done' can now be marked.

Steps 3 - ....
And so on, until all packages have been updated.

Of course there'll be gotchas but that's the fun part :-) For one, think of circular or even entangled dependencies - but do we have those? they'd be a pain when installing anew in the first place. For another, think of required libraries.
Maybe circular deps. can be detected and processed separately for each step 2 and up using the '-nodeps' option, or maybe they can only be updated in the end using '-nodeps', I'm unsure.

It all doesn't look hard to me, but I have no time now to set this up. Some more thinking is required about finding out which packages can be updated safely in the next round and what to do with all the corner cases & gotchas.
But all in all I'd be surprised if this were to lead to more that half a screenful of code. Some while or do...until loop seems ideal for the job.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 04 Apr 2016 01:57:47 PM UTC, comment #5: 

Recording some thoughts here for a workable hack vs a better fix:

most of pkg.m is a top level wrapper with work in the private functions. In particular, pkg update calls install.m for each package after generating a list to iterate through. install.m has 'unsatisfied dependency' as an error, so this aborts the whole update process. install.m is called without any expected return variables, so there's no feedback to pkg.m, and dependencies aren't checked by pkg.m, just inside install.m

1 - bare minimum fix: "don't abort the pkg.m update sequence when a dependency error is encountered."
--> is there a way for the error (or something similar) to only abort the current script and return control to the calling function pkg? Would there be a problem if this was replaced with a warning?  for a single pkg install, I believe either a warning or an error would be the last thing done by the script. for multiple installs, a warning would inform the user but still continue to try the other packages.

2 - #1 + inform the user in some way that they should re-run pkg update - if changed to a warning, there should be a way to check if a particular warning  was issued, change a flag to 'dependency_error_encountered', and then at the end of the update process, produce a 'you should re-run pkg update' message.

3 - if #2 is happening, other things could be triggered by the flag, such as logging the package(s) that caused the error, and retrying them once at the end of the update script.

4 - the most desirable option, that pkg checks for dependencies and re-orders the update list so that nothing occurs out of order, would require moving (copying) the dependency check from install to pkg. Not too difficult, but much more involved.

Nicholas Jankowski <nrjank>
Group Member
Tue 15 Mar 2016 08:21:19 PM UTC, comment #4: 

I don't know about the bigger pkg environment challenge, but I'll take a look at this specific bug. If I can wrap my head around what pkg is doing I'll see if any hacks (in the order specified in my first comment) are workable. Once we have this tackled, I'll start peeking at some of the other pkg project issues, and see if any sit within my current comfort zone. ;)

Nicholas Jankowski <nrjank>
Group Member
Tue 15 Mar 2016 07:49:39 PM UTC, comment #3: 

Oh, if you (or someone else) would pick this up that would be great :-)

This dependency issue has always been a stumbling block for novice users;  I didn't realize  it could also a stumbling block for all of us when upgrading all packages. As I upgrade Octave (on Windows) about twice a week and OF packages are automatically renewed in that process I never noted pkg.m's "upgrade" instruction.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 15 Mar 2016 04:57:48 PM UTC, comment #2: 

yes, I realized there has been quite a bit of discussion on modifications to the package handling.  I called this one out here mainly because it seems to be a specific bug with a possible shorter term fix (or hack) that it errors out and fails to continue updating other packages.

what you say sounds ideal. I didn't realize it was sitting in m-code. I may take a peek.

Nicholas Jankowski <nrjank>
Group Member
Tue 15 Mar 2016 04:50:27 PM UTC, comment #1: 

(bug report title could use some polishing :-) )

I'm afraid that for some packages several iterations are needed instead of just two (e.g., communications depends on signal that depends on control).
It is probably better if pkg.m would set up a dependency tree, descend into it and infer an optimal "route" for each OF package to upgrade it, rather than cycle through all OF packages again and again.
Actually, a case can be made that it should do so when installing packages as well, esp. with the -forge flag.

In fact similar ideas have been mentioned as a Project Idea here:
http://wiki.octave.org/Projects#Packaging

I think it probably isn't very hard to set up dependency trees for OF packages; parsing & interpreting the share/octave/octave-packages file would be a first start and much of that code is present in pkg.m and its companion functions.
Making pkg.m follow the optimized "route" through the package collection is probably the hardest part. But again, maybe not.

But admittedly pkg.m is quite a beast; I've tried improving it before (as regards what to do about shadowing core functions ad matching functions from other OF packages) and I quickly lost track :-(

(release => dev; OS => any)

Philip Nienhuis <philipnienhuis>
Group Member
Mon 14 Mar 2016 03:22:21 AM UTC, original submission:  

When running 'pkg update' it appears to work through the packages in some particular order (alphabetical order, or install order) When one of the packages has a dependency on a version of a package that has not been updated yet, pkg errors out back to the command line and does not continue checking any other packages for updates.

This can be seen in the windows version of 4.0.0 where the default package set is a bit out of date. Running the build_packages script followed by pkg update gives the following error:

error: the following dependencies were unsatisfied:
> >    optim needs struct >= 1.0.12


And pkg update cannot proceed to other packages until struct is done first. While this is not difficult to do, it may be confusing for a new user.

I would think the simplest solution would be for package to somehow catch the error and step over that package. Then running the update twice would result in everything being fully updated. Perhaps if it needed to step over a package it could tell the user he should probably rerun the update.

Alternatively it would be nice if it would just recognize that it had to step over one of the packages and to try it again at the end. If it failed again and could tell the user.

Really nice would be for it to recognize which package is needed, if it is already installed run that update first, if it's not installed ask the user whether it should be installed, and then after the dependency has been satisfied pick up the update process where it left off.

Nicholas Jankowski <nrjank>
Group Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by arungiridhar (Posted a comment)
  • -email is unavailable- added by nrjank (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 7 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2021-12-01 nrjank Severity3 - Normal 2 - Minor
        Summarypkg update errors out of one package sees dependency error pkg update errors out if one package sees a dependency error
    2021-05-10 mtmiller Carbon-CopyRemoved 80942 -
    2016-03-31 mtmiller CategoryNone Octave Function
        StatusNone Confirmed
    2016-03-15 philipnienhuis Release4.0.0 dev
        Operating SystemMicrosoft Windows Any

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code