BIOGEME Algorithms

Optimization algorithms

Biogeme can use five different optimization algorithms: CFSQP, DONLP2, SOLVOPT, BIO and BIOMC. Note that it is possible that each of them produce different solutions. Usually, the discrepancies are small, and due to numerical differences and various stopping criteria. Also, none of them identifies a global maximum of the likelihood function. Therefore, it may happen that one of them is caught in a local maxima, different from local maxima found by other algorithms.

Which one to choose is difficult to say. They all have advantages and disadvantages. BIO (which stands for BIerlaire's Optimization, like in Biogeme) has been specifically adapted for this software package, but it cannot accommodate nontrivial constraints yet. A version for simulated maximum likelihood estimation, called BIOMC, is also available. CFSQP is not free and, therefore, not included in the general distribution of Biogeme. DONLP2 is slower than CFSQP, but faster than SOLVOPT. SOLVOPT can sometimes be very slow. The algorithm is not to blame. It has not been originally designed for the kind of optimization problem involved in Biogeme. However, it seems robust on ill-specified models. When other algorithms fail to converge, SOLVOPT may succeed in finding a solution. Therefore, our recommendation would be:

  1. If there are no non-trivial constraint on the parameters, use BIO.
  2. If there are, or if BIO is very slow, use CFSQP if you have it.
  3. If not, use DONLP2.
  4. If all fail, try SOLVOPT.
  5. If it fails again, redefine your model.
It is always a good idea to solve the same problem with several algorithms.

BIO

BIO is a trust-region algorithm (see Conn et al, 2000) designed for problems with simple bounds constraints, using a truncated conjugate-gradient method to solve the trust-region subproblem. By default, it uses the BHHH (Berndt et al, 1974) matrix as an approximation of the second derivatives matrix. At each iteration, it displays
  1. The value used in the stopping criterion at iterate x, that is the infinite norm of the relative gradient, computed as

    γ = maxi|| |gi(xi)| max(1,|xi|) / max(f(x),tf) ||

    where g is the projection of the gradient of f onto the feasible set, tf is defined by the parameter BTRTypf (see Dennis and Schnabel, 1996).
  2. The iteration number.
  3. The radius of the trust region.
  4. The current value of the objective function. As it is a minimization algorithm, the objective function is the opposite of the log-likelihood and, therefore, is a positive value.
  5. The exit status of the subproblem solver.
  6. The value used to check if the radius of the trust region is appropriate, that is

    (f(x+) - f(xc))/(m(x+)-m(xc))

    where xc is the current iterate, x+ the new candidate, and m the quadratic model approximating f at xc (see Conn et al, 2000 for details).
  7. The number of variables not constrained to one of their bounds.
  8. The status of the iteration, that is very successful (++), successful (+) or unsuccessful (-).
  9. If the subproblem is preconditioned, a P is also displayed.

BIOMC

It is a version of BIO designed for simulated maximum likelihood. The idea is to use BIO with few draws in the beginning, in order to obtain a rough value of the parameters, and then to increase the number of draws until reaching the number required by the user (du). Let dk be the number of draws considered by BIOMC at iteraton k (where d0 is defined by the parameter BTRStartDraws). Then, algorithm BIO runs until the following condition is verified:

ln(γ) / ln(ε) ≥ dk / du,

where γ is defined above (see BIO) and ε is defined by the parameter BTRTolerance, so that the convergence requirements are not as strong when the number of draws is low. Then, the number of draws is increased:

dk+1 = min(λ dk,du),

where the factor λ is defined by the parameter BTRIncreaseDraws, and the process starts again. The algorithm stops when dk=du. In general, BIOMC is not faster than BIO. But it allows to obtain good approximations of the parameters pretty quickly.

CFSQP

CFSQP does not stand for Canadian Food Safety and Quality Program. It is the version written in C of the Feasible Sequential Quadratic Programming algorithm developed by E.R. Panier, A.L. Tits, J.L. Zhou, and C.T. Lawrence and distributed by aemDesign. The conditions for external use are the following
  1. The CFSQP routines may not be distributed to third parties. Interested parties shall contact AEM Design directly.
  2. If modifications are performed on the routines, these modifications shall be communicated to AEM Design. The modified routines will remain the sole property of the authors.
  3. Due acknowledgment shall be made of the use of the CFSQP routines in research reports or publications. Whenever such reports are released for public access, a copy shall be forwarded to AEM Design.
  4. The CFSQP routines may only be used for research and development, unless it has been agreed otherwise with AEM Design in writing.
If you have a CFSQP license, and need a Windows version of Biogeme with CFSQP, send an Email to michel.bierlaire@epfl.ch to receive the executable.

SOLVOPT

Solvopt is defined by its authors, Alexei V. Kuntsevich and Franz Kappel, as follows: The program SolvOpt (Solver for local optimization problems) is concerned with minimization resp. maximization of nonlinear, possibly non-smooth objective functions and with the solution of nonlinear programming problems taking into account constraints by the so-called method of exact penalization. The package implements a version of the minimization method with space dilation by Shor (1985). See Kuntsevich and Kappel (1997) for a tutorial of the package and the description of the algorithm.

DONLP2

DONLP2 is a sequential equality constrained quadratic programming method, developed by P. Spellucci. The algorithm is described by Spellucci, 1999. The conditions of use of the DONLP2 package are the following.
  1. donlp2 is under the exclusive copyright of P. Spellucci (spellucci@mathematik.tu-darmstadt.de) ``donlp2" is a reserved name
  2. donlp2 and its constituent parts come with no warranty, whether expressed or implied, that it is free of errors or suitable for any specific purpose. It must not be used to solve any problem, whose incorrect solution could result in injury to a person , institution or property. It is at the users own risk to use donlp2 or parts of it and the author disclaims all liability for such use.
  3. donlp2 is distributed ``as is". In particular, no maintenance, support or trouble-shooting or subsequent upgrade is implied.
  4. The use of donlp2 must be acknowledged, in any publication which contains results obtained with it or parts of it. Citation of the authors name and netlib-source is suitable.
  5. The free use of donlp2 and parts of it is restricted for research purposes commercial uses require permission and licensing from P. Spellucci.
Back
Biogeme