The Analog Kalman Filter

From Hooke's law to Bayes' rule in five consolidation moves.

1. A Hookean universe

Work in one dimension and imagine a slightly unusual physics: a body of mass $k$ sitting at position $x$ attracts a unit test particle at position $z$ with a force that grows linearly with separation,

$$ F(z) \;=\; k\,(x - z), \qquad U(z) \;=\; \tfrac{1}{2}\,k\,(z - x)^2 . $$

Whether you call $k$ a mass in linear gravity or the stiffness of a spring tying the particle to an anchor at $x$ makes no difference — the algebra is Hooke's law either way. Now compare the potential $U$ with a Gaussian density $\mathcal{N}(x, k^{-1})$ evaluated at $z$:

$$ -\log p(z) \;=\; \tfrac{1}{2}\,k\,(z - x)^2 + \text{const}. $$

The two are identical up to a constant. So in this universe: energy is negative log-likelihood, mass is precision (inverse variance), and force is the score $\tfrac{d}{dz}\log p(z)$. Minimising energy is maximising likelihood, and a particle released to find equilibrium is performing maximum a posteriori estimation by analog computation. (The full dictionary has more entries.)

2. Fusion: the centre of mass is Bayes' rule

Attach the test particle to two bodies: a prior belief of mass $P^{-1}$ anchored at $m$, and an observation of mass $R^{-1}$ anchored at $y$. The net force vanishes where the pulls balance:

$$ P^{-1}(m - z) + R^{-1}(y - z) = 0 \;\;\Longrightarrow\;\; z^\ast \;=\; \frac{P^{-1}m + R^{-1}y}{P^{-1} + R^{-1}} . $$

That is the centre of mass of the two bodies — and it is also, exactly, the Kalman measurement update. In gain form, with $K = P/(P+R)$, the same point is $z^\ast = m + K(y - m)$. The posterior precision is the total mass, $(P')^{-1} = P^{-1} + R^{-1}$: masses attached in parallel add, which is the mechanical face of the rule that independent precisions add.

This licenses the first consolidation move: as far as the test particle is concerned, the two bodies may be replaced by a single body of mass $P^{-1}+R^{-1}$ sitting at $z^\ast$. No experiment performed by the particle can tell the difference. Consolidation is what makes the analogy computational rather than decorative — it is the mechanical form of “the posterior is again Gaussian.” Watch it settle in the demo.

3. Chains: series springs are the reduced mass

Now make the model hierarchical, as in the article. The unknown location $\mu$ has a Gaussian prior; an intermediate quantity $x_2$ scatters around $\mu$; and the measurement $y$ is a noisy reading of $x_2$:

$$ \mu \sim \mathcal{N}(m, P), \qquad x_2 \mid \mu \sim \mathcal{N}(\mu, \rho^{-1}), \qquad y \mid x_2 \sim \mathcal{N}(x_2, \phi^{-1}). $$

Mechanically, the body at $y$ no longer connects to the test particle directly: its spring of stiffness $\phi$ ends at a free intermediate joint, and a second spring of stiffness $\rho$ continues to the particle. Two springs in series have effective stiffness

$$ \tilde{\phi} \;=\; \left( \frac{1}{\phi} + \frac{1}{\rho} \right)^{-1} \;=\; \frac{\phi\,\rho}{\phi + \rho}, $$

which the physicists among us recognise as the reduced mass formula $\mu_{12} = m_1 m_2 / (m_1 + m_2)$. Statistically it is nothing but “variances add along a chain”: marginally $y \mid \mu \sim \mathcal{N}(\mu,\, \phi^{-1} + \rho^{-1})$. The second consolidation move follows: replace the chain by a single body of mass $\tilde{\phi}$ at $y$ (watch the chain and its consolidation settle on the same line), then apply the centre-of-mass rule from step 2:

$$ m' \;=\; \frac{P^{-1}m + \tilde{\phi}\,y}{P^{-1} + \tilde{\phi}}, \qquad (P')^{-1} \;=\; P^{-1} + \tilde{\phi}. $$

4. The Kalman filter as a limit

Stiffen the intermediate link. As $\rho \to \infty$ the joint is pinned — $x_2$ coincides with $\mu$ — and the reduced mass tends to $\tilde{\phi} \to \phi$, recovering the standard update of step 2. The textbook Kalman measurement step is the rigid-rod limit of a hierarchical model; keeping $\rho$ finite is the hierarchical model, at the price of one harmonic mean. The fusion demo has a $\rho$ dial so you can feel the observation's effective mass drain away as the channel gets sloppier.

5. The whole filter: alternate parallel and series

A full filtering cycle needs one more ingredient: time. Between observations the state diffuses with process noise of variance $Q$, so the predictive variance is $P' + Q$ — which in mass terms is another series composition, this time with the process-noise body of mass $q = Q^{-1}$:

$$ p_{\text{pred}} \;=\; \frac{p'\,q}{p' + q} \;<\; p'. $$

So the scalar Kalman filter is an alternation of the two composition rules: measurement updates add mass in parallel (evidence accretes), time updates bleed mass through a series spring (staleness erodes). The state estimate is a body whose weight waxes with every observation and wanes with every tick of the clock.

6. Beyond one dimension

Nothing above is really scalar. In $n$ dimensions mass becomes a precision matrix, springs pull along vectors, and the centre-of-mass and series rules become $\Sigma$-weighted averages and matrix harmonic means. Conditioning a multivariate Gaussian on some of its coordinates — pinning part of the spring network to the wall and asking what the rest feels — produces a Schur complement of the precision matrix (try it on a small graph). That thread is pulled at the sister site, schur.microprediction.org.

Caveats

The correspondence is exact precisely when energies are quadratic, i.e. when everything is Gaussian and linear. Non-Gaussian likelihoods are non-Hookean springs: the picture of forces balancing at a mode survives (that is just MAP estimation as equilibrium), but the consolidation moves — the licence to replace a subsystem by one equivalent body — do not. That failure is instructive too: it is the mechanical face of non-conjugacy.

Next: the dictionary collects the correspondences in one table, and the demos let you drag the masses yourself.