Note: versions 3.2.9 and 3.2.10 are identical. Therefore, version 3.2.9 has been removed from the official distribution platform.

New syntax for DefineVariable

DefineVariable actually defines a new column in the database. The old syntax was:

myvar = DefineVariable('myvar', x * y + 2, database)

The new syntax is:

myvar = database.DefineVariable('myvar', x * y + 2)

Likelihood ratio test

It is now possible to perform a likelihood ratio test directly from the estimation results. See documentation here. It relies on a function that can be used in more general context. See documentation here.

Comparing several models

It is now possible to compile the estimation results from several models into a single data frame. See documentation here.

Automatic segmentation

It is now possible to define a parameter such that it has a different value for each segment in the population. See the example 01logitBis.py.

Simulation of panel data

It is now possible to use Biogeme in simulation mode for panel data. See the following example: 13panel_simul.py.

Flattening panel data

This new feature transforms a database organized in panel mode (that is, one row per observation) into a database organized in normal mode (that is, one row per individual, and the observations of each individual across columns). See documentation here and here

Covariance and correlation matrix of the nested and the cross-nested logit models

These new functions calculate the covariance and the correlation matrix of the error terms of a cross-nested logit model from the estimated parameters. See documentation here, here, here and here.

Recycling estimation results

It is now possible to skip estimation and read the estimation results from the pickle file by setting the parameter recycle=True. See the online documentation [here].

The feature removing unused variables has been canceled.

The parameters removeUnusedVariables and displayUsedVariables in the BIOGEME constructor have been removed.

More functionalities for the mathematical expressions.

The expressions have now been designed to also be available outside of the BIOGEME class. A detailed illustration of the functionalities is available [Click here].

New syntax for the assisted specification algorithm

The new syntax involves NamedTuple to make the code more readable. Refer to the examples, such as optima.py.

Note that version 3.2.7 and 3.2.8 are almost identical. The description belows compares to version 3.2.6.

Assisted specification

The asssisted specification algorithm by Ortelli et al. (2021) is now available.

Optimization

The optimization algorithms have been organized into two modules. The module algorithms.py contains generic optimization algorithms. The module optimization.py contains the functions that can be called directly by Biogeme [Click here for the documentation of the estimate function]. [Click here for an example.]

CFSQP

The CFSQP algorithm has been removed from the distribution.

Null log likelihood

The log likelihood is calculated. The null model predicts equal probability for each alternative.

Saved iterations

Iterations are saved in a file with extension .iter. If the file exists, Biogeme will initialize the parameters from this files, and ignore the starting values provided. To turn this feature off, set biogeme.saveIterations=False

Random starting values

It is possible to modify the initial values of the parameters in all formulas, using randomly generated values. The value is drawn from a uniform distribution on the interval defined by the bounds (by default [-100, 100].) [Click here for the documentation].

Sensitivity analysis

The betas for sensitivity analysis are now generated by bootstrapping. [Click here for the documentation].

Box-Cox

The implementation of the Box-Cox transform was incorrect and has been corrected.

Validation

The out-of-sample validation has been improved. [Click here for the documentation]. It has to be compined with the split function of the database object.

Statistics about chosen alternatives

It is now possible to calculate the number of time each alternative is chosen and available in the sample. [Click here for the documentation].

Validity check for the nests

The validity of the specification of the nests for nested and cross nested logit models is new checked.

ALOGIT file

Output files in F12 format compatible with ALOGIT can now be produced. [Click here for the documentation.

Likelihood ratio test

A function to perform the likelihood ratio test has been implemented. [Click here for the documentation].

Optimization

New optimization algorithms are available for estimation See the documentation of the estimate function, and the optimization module. See also an example.

Stochastic log likelihood

It is now possible to calculate the log likelihood function on a sample (a batch) of the full data file. This is particularly useful with large databases. It can be used in the implementation of a stochastic gradient algorithm, for instance. See documentation.

User's notes

It is possible to include your own notes in the HTML file using the userNotes parameter of the biogeme object. See documentation. See example.

Scaling

It is possible to have Biogeme suggesting the scales of the variables in the database using the suggestScales parameter of the biogeme object. See documentation.

Estimation

A new function quickEstimate performs the estimation of the parameters, and skips the calculation of the statistics. See documentation.

Validation

A new function in the database module allows to split the database in order to prepare an estimation and a validation sets, for out-of-sample validation. See documentation. It is used by the new function validate in the biogeme module. See documentation. See example.

Messages

A new function allows to extract all the messages generated during a run. See documentation. See example. It is also possible to make the logger temporarily silent using the functions temporarySilence and resume.

In order to comply better with good programming practice in Python, the syntax to import the variable names from the data file has been modified since version 3.2.5. The file headers.py is not generated anymore. The best practice is to declare every variable explicity:

PURPOSE = Variable('PURPOSE')
CHOICE = Variable('CHOICE')
GA = Variable('GA')
TRAIN_CO = Variable('TRAIN_CO')
CAR_AV = Variable('CAR_AV')
SP = Variable('SP')
TRAIN_AV = Variable('TRAIN_AV')
TRAIN_TT = Variable('TRAIN_TT')

If, for any reason, this explicit declaration is not desired, it is possible to replace the statement

from headers import *

by

globals().update(database.variables)

where database is the object containing the database, created as follows:

import biogeme.database as db
df = pd.read_csv('swissmetro.dat', '\t')
database = db.Database('swissmetro', df)

Also, in order to avoid any ambiguity, the operators used by Biogeme must be explicitly imported. For instance:

from biogeme.expressions import Beta, bioDraws, PanelLikelihoodTrajectory, MonteCarlo, log

Note that it is also possible to import all of them using the following syntax

from biogeme.expressions import *

although this is not a good Python programming practice.

If you have the results of a previous estimation, it may be a good idea to use the estimated values as a starting point for the estimation of similar models. If not, it depends on the nature of the parameters:

• If the parameter is a coefficient (traditionally denoted by β), the value 0 is appropriate.
• If the parameter is a nest parameter of a nested or cross-nested logit model (traditionally denoted by μ), the value 1 is appropriate. Make sure to define the lower bound of the parameter to 1.
• If the parameter is the nest membership coefficient of a cross-nested logit model (traditionally denoted by α), the value 0.5 is appropriate. Make sure to define the lower bound to 0 and the upper bound to 1.
• If the parameter captures the membership to a class of a latent class model, the value 0.5 is appropriate. Make sure to define the lower bound to 0 and the upper bound to 1.
• If the parameter is the scale of an error component in a mixture of logit model (traditionally denoted by σ), the value must be sufficient large so that the likelihood of each observation is not too close to zero. It is suggested to try first with the value one. If there are numerical issues, try a larger value, such as 10. See Section 7 in the report Estimating choice models with latent variables with PandasBiogeme for a detailed discussion.

Yes. It is actually the default behavior. At each iteration, Biogeme creates a file __myModel.iter. This file will be read the next time Biogeme tries to estimate the same model. If you want to turn this feature off, set the BIOGEME class variable saveIterations to False.

Yes. See example 04validation.py on Github.

The issue is that in Python 3.8 and older on Windows, DLLs are loaded from trusted locations only (see this). It is necessary to add the path of the DLLs. Here is a way proposed by Facundo Storani, University of Salerno:

• Search the DLLs folder of anaconda3. It may be similar to: C:\Users\[USER_NAME]\anaconda3\DLLs or C:\ProgramData\Anaconda3\DLLs.
• Click the Start button, type "environment properties" into the search bar and hit Enter.
• In the System Properties window, click "Environment Variables."
• Select "Path" on the users' list, and modify.
• Add the path of the dlls folder to the list. It may be similar to: C:\Users\[USER_NAME]\anaconda3\DLLs or C:\ProgramData\Anaconda3\DLLs.

(credit: Facundo Storani)

On Mac OSX, the following error is sometimes generated:

ImportError:
dlopen(/Users/~/anaconda3/lib/python3.6/site-packages/biogeme/cbiogeme.cpython-36m-darwin.so,
2): Symbol not found:
__ZNSt15__exception_ptr13exception_ptrD1Ev

It is likely to be due to a conflict of versions of Python packages. The best way to deal with it is to reinstall Biogeme using the following steps:

• First, make sure that you have the latest version of pip:

  pip install --upgrade pip
  

• Uninstall biogeme:

  pip uninstall biogeme
  

• Install cython:

  pip install —-upgrade cython
  

• Reinstall biogeme, without using the cache:

  pip install biogeme -—no-cache-dir
  

If it does not work, try first to install gcc:

conda install gcc

If it does not work, try creating a new conda environment:

conda create -n python310 python=3.10 pip
conda activate python310
pip install biogeme

If it does not work... I don't know :-(

On Mac OSX and Windows, the procedure is designed to install from binaries, not sources. If you get messages that look like the following, it means that pip is trying to compile from sources. And it will most certainly fail as the environment must be properly configured.

Running setup.py install for biogeme ... error
Complete output from command
c:\users\willi\anaconda3\python.exe -u -c "import setuptools,
tokenize;
__file__='C:\Users\willi\AppData\Local\Temp\pip-install-iaflhasr\biogeme\setup.py';
f=getattr(tokenize, 'open', open)(__file__);
code=f.read().replace('\r\n', '\n');
f.close();
exec(compile(code, __file__, 'exec'))" install --record C:\Users\willi\AppData\Local\Temp\pip-record-v6_zn0ff\install-record.txt --single-version-externally-managed --compile:
Using Cython
Please put "# distutils: language=c++" in your .pyx or .pxd file(s)
running install

It means that there is no binaries available for your version of Python. To check which versions are supported, go to the repository

pypi.org/project/biogeme/

For instance, the following files are available for version 3.2.10:

biogeme-3.2.10.tar.gz

biogeme-3.2.10-cp310-cp310-win_amd64.whl

biogeme-3.2.10-cp310-cp310-macosx_10_9_x86_64.whl

biogeme-3.2.10-cp39-cp39-win_amd64.whl

biogeme-3.2.10-cp39-cp39-macosx_10_9_x86_64.whl

biogeme-3.2.10-cp38-cp38-win_amd64.whl

biogeme-3.2.10-cp38-cp38-macosx_10_9_x86_64.whl

biogeme-3.2.10-cp37-cp37m-win_amd64.whl

biogeme-3.2.10-cp37-cp37m-macosx_10_9_x86_64.whl

biogeme-3.2.10-cp36-cp36m-macosx_10_9_x86_64.whl

It means that you can use Python 3.7, 3.8 and 3.9 on both platforms, while the version for Python 3.6 is only available on MacOSX.