Principles and testing#

Durbin and Koopman as the source of truth#

Decision. The algorithms, their names and their defaults follow Durbin and Koopman (2012), Part I. Where statsmodels differs, we follow DK.

Why. One reference keeps the library consistent: the recursions, the initialization, the transforms and the diagnostics come from the same derivations, and a disagreement has one arbiter. DK are also the authors of most of the methods.

Alternatives. Matching statsmodels in every detail would ease comparisons, but would carry over its choices where DK differ (the variance transform, the damped cycle) and its known errors (see Differences from statsmodels).

Where. Comments in the source cite DK sections and equations, as do these documents.

statsmodels as a reference, not an authority#

Decision. statsmodels’ tsa.statespace provides test data: the fixtures in test/fixtures are its output for the same models, written by test/fixtures/make_fixtures.py. Where it follows DK the results must agree, typically to 1e-9 in relative terms; where it does not, the tests check our results another way.

Why. An independent implementation catches errors that internal checks miss. Using it as the only check would import its errors: three were found this way (see Differences from statsmodels).

Checks where no reference exists#

Several algorithms have no counterpart in statsmodels. Each is checked against another route to the same answer:

  • the dense matrix form of the model (DK §4.13) against the recursions;

  • the exact diffuse filter against the approximate one as \(\kappa \to \infty\), and against the augmented filter (DK §5.7);

  • the univariate against the conventional filter;

  • the square root, fast, classical and two-filter smoothers against the state smoother;

  • collapsing against the full model;

  • the marginal likelihood against the invariance and the examples of Francke, Koopman and de Vos (2010);

  • the analytic score against finite differences;

  • EM at the maximum likelihood estimate as a fixed point;

  • the worked examples of DK §5.6 in closed form;

  • the smoothing spline against SciPy’s, and least squares residuals against ordinary least squares;

  • the illustrations of DK ch. 8 against DK’s printed values.

Completeness over convenience#

Decision. Every algorithm of DK Part I is implemented, including those DK present for insight rather than use: the Whittle recursion (DK §4.6.3) loses accuracy on long series and the dense form (DK §4.13) costs \(O((nm)^3)\). They are documented as such.

Why. The library serves as a companion to the book; the extra algorithms also check the main ones.