bugGNU Octave - Bugs: bug #54420, fminsearch default TolFun is...

 
 

bug #54420: fminsearch default TolFun is different from Matlab

Submitter:  Arnaud Delorme <arnodelorme>
Submitted:  Wed 01 Aug 2018 05:39:33 PM UTC
   
 
Category:  Octave Function Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Matlab Compatibility
Status:  Fixed Assigned to:  None
Originator Name:  Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Sat 04 Aug 2018 04:23:22 AM UTC, comment #2: 

I changed fminsearch tolerances to match Matlab in this changeset (https://hg.savannah.gnu.org/hgweb/octave/rev/8eec0403a370).  Marking as fixed and closing report.

Rik <rik5>
Group administrator
Wed 01 Aug 2018 06:14:32 PM UTC, comment #1: 

I'm not sure there is anything to fix here.

First, fminsearch is included by default in core Octave in scripts/optimization so there is no need to load the 'optim' package.

Second, optimization routines are known to be very sensitive to the initial starting location.  In this case, it appears that the wobble function has at least two local minima that are quite close together and Octave chooses one while Matlab chooses a different one.  That is just a hazard of numerical analysis, but not what I would call a true incompatibility with Matlab.

Regarding tolerance, it appears that Matlab has changed the values of some of the default options they feed to the fminsearch algorithm.  I'm pretty sure Octave used to be in sync, but now Matlab passes


Display


Level of display (see Iterative Display):

    'notify' (default) displays output only if the function does not converge.

    'final' displays just the final output.

    'off' or 'none' displays no output.

    'iter' displays output at each iteration.

FunValCheck

Check whether objective function values are valid. 'on' displays an error when the objective function returns a value that is complex or NaN. The default 'off' displays no error.

MaxFunEvals


Maximum number of function evaluations allowed, a positive integer. The default is 200*numberOfVariables. See Tolerances and Stopping Criteria.

MaxIter


Maximum number of iterations allowed, a positive integer. The default value is 200*numberOfVariables. See Tolerances and Stopping Criteria.
OutputFcn

Specify one or more user-defined functions that an optimization function calls at each iteration, either as a function handle or as a cell array of function handles. The default is none ([]). See Output Functions.

PlotFcns


Plots various measures of progress while the algorithm executes. Select from predefined plots or write your own. Pass a function handle or a cell array of function handles. The default is none ([]):

    @optimplotx plots the current point.

    @optimplotfunccount plots the function count.

    @optimplotfval plots the function value.

For information on writing a custom plot function, see Plot Functions.

TolFun


Termination tolerance on the function value, a positive scalar. The default is 1e-4. See Tolerances and Stopping Criteria. Unlike other solvers, fminsearch stops when it satisfies both TolFun and TolX.

TolX


Termination tolerance on x, a positive scalar. The default value is 1e-4. See Tolerances and Stopping Criteria. Unlike other solvers, fminsearch stops when it satisfies both TolFun and TolX.


whereas Octave does


  if (nargin == 1 && ischar (fun) && strcmp (fun, "defaults"))
    x = optimset ("Display", "notify", "FunValCheck", "off",
                  "MaxFunEvals", 400, "MaxIter", 400,
                  "OutputFcn", [],
                  "TolFun", 1e-7, "TolX", 1e-4);


In particular, the TolFun argument is 1e-7 for Octave versus 1e-4 for Matlab.  That seems reasonable to make consistent so I have re-titled the bug report.

Rik <rik5>
Group administrator
Wed 01 Aug 2018 05:39:33 PM UTC, original submission:  

This call

newcenter = fminsearch(@(center,x,y,z)std(sqrt((x - center(1)).^2+(y - center(2)).^2+(z - center(3)).^2)-mean(x+y+z-center(1)-center(2)-center(3))),[0 0 0],struct('MaxFunEvals',[3000]),[0;2;0],[1.95;0;0],[0;0;2])

returns under Matlab

0.0149   -0.0354    0.0149

And, under Octave 4.4.0 (after running pkg install -forge optim) returns

0.10107   0.05302   0.10107

The local minimum is consistency different. The Matlab function has been returning the same values since Matlab 5.3 (up to 2018a tested above).

--- further investigation

wobble = @(center,x,y,z)std(sqrt((x - center(1)).^2+(y - center(2)).^2+(z - center(3)).^2)-mean(x+y+z-center(1)-center(2)-center(3)));

wobble(newcenter,[0;2;0],[1.95;0;0],[0;0;2])

returns 1.9971e-05 under Matlab
returns 0.00000018495 under Octave so the Octave error is lower

Not that anything should be changed after further inspection but it is worth documenting that the fminsearch function is performing better in Octave than in Matlab one in this specific instance.

It also points to the fact that different algorithms are used by default. Changing the number of iterations in Matlab does not change the function output.

Arno

Arnaud Delorme <arnodelorme>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by arnodelorme (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 6 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-08-04 rik5 StatusConfirmed Fixed
        Open/ClosedOpen Closed
    2018-08-02 rik5 Release4.4.0 dev
        Operating SystemMac OS Any
    2018-08-01 rik5 StatusNone Confirmed
        Summaryfminsearch innacuracy fminsearch default TolFun is different from Matlab

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code