Spring Theory
Visualizing common statistical procedures with exact physical analogs.
Stiffen the chain and it pulls taut toward a constant (all evidence pooled); slacken it and each bead chases its own observation. Drag an observation and watch the disturbance propagate a few springs down the chain and die out. Most fun: add an observation and watch the whole posterior path re-equilibrate to accommodate it. This is the smoother — the whole path relaxing at once. For the filter — one body running forward through time, tick by tick — see the filter-through-time demo.
What you are looking at: the Kalman smoother as a bead chain
The local-level model — a random-walk state observed in noise, $x_{t+1} = x_t + \mathcal{N}(0, q^{-1})$, $y_t = x_t + \mathcal{N}(0, r^{-1})$ — is a chain of beads. Each green bead $x_t$ rides a frictionless vertical rod at its time slot, hangs from its observation (dark) by a measurement spring of stiffness $r$, and is tied to its neighbours by process-noise springs of stiffness $q$. The rods matter: they stop the chain contracting sideways, and because the springs have zero natural length the diagonal neighbour spring's energy splits exactly into a constant (the fixed horizontal span) plus $\tfrac12 q\,(x_{t+1}-x_t)^2$ — so the constant drops out of the mechanics and the total stored energy is, exactly, the negative log posterior of the entire path,
The chain's resting shape therefore is the Kalman smoother — equally, Whittaker–Henderson graduation with a first-difference penalty, the Hodrick–Prescott filter's older sibling. The dashed curve is the exact smoother from a tridiagonal solve; the beads just obey $F = ma$ with damping. That they agree is the point: this is an exact physical analog, not an illustration. The smoother's influence is local — a dragged observation disturbs only a few springs' worth of chain — which is why filtering and smoothing agree except near the ends.
Every Gaussian belief is a spring: its negative log-density is a quadratic potential well, so precision is mass, energy is negative log-likelihood, and estimation is mechanical equilibrium. Each link of the chain above is one series spring (the prediction step bleeding precision); each measurement spring adds its mass in parallel; run left to right doing those two moves and you have the Kalman filter. The introduction derives the correspondence, the dictionary tabulates it, and the demos page has more exact analogs: evidence fusion, reduced mass, the filter through time, conditioning a Gaussian graph, Schur damping, least squares, minimum variance, PCA, bid-ask bounce, and where the analogy breaks — Ridge vs Lasso.
Bibliography
Works on, behind, or adjacent to the mechanical view of Gaussian inference. Many of these were collected by Loftus, whose post revived the analogy.
The core
- Cotton, P. (2021). “The Analog Kalman Filter.” Geek Culture (Medium). The article this site is loosely based on: precision as mass, reduced-mass consolidation, and the Kalman filter as a limiting hierarchical model.
- Loftus, J. (2020). “Least Squares as Springs.” The regression half of the story, and the direct inspiration acknowledged in the article.
- Levi, M. (2009). The Mathematical Mechanic: Using Physical Reasoning to Solve Problems. Princeton University Press. A book-length case that mechanics proves theorems, springs-for-least-squares included.
- Wickham, H., Navarro, D., and Pedersen, T. L. ggplot2: Elegant Graphics for Data Analysis, “A case study: drawing springs”. Hadley Wickham's worked example of rendering springs as a ggplot2 geom — how to draw these diagrams properly in R.
Antecedents
- Pearson, K. (1901). “On Lines and Planes of Closest Fit to Systems of Points in Space.” Philosophical Magazine 2(11), 559–572. PCA, born as an orthogonal-fitting problem — the rod-on-collars of the PCA demo.
- Whittaker, E. T. (1923). “On a New Method of Graduation.” Proceedings of the Edinburgh Mathematical Society 41, 63–75. The penalised smoother that the bead chain above relaxes to.
- Dwight, T. W. (1937). “The Fitting of Linear Regression Lines by the Method of Least Squares.” The Forestry Chronicle 13(4), 509–519. A forester fitting regressions with literal springs, ninety years ago.
- Kalman, R. E. (1960). “A New Approach to Linear Filtering and Prediction Problems.” Journal of Basic Engineering 82(1), 35–45.
- West, M., and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models, 2nd ed. Springer. The dynamic linear model treatment in which precision coordinates are natural.
- Roll, R. (1984). “A Simple Implicit Measure of the Effective Bid-Ask Spread in an Efficient Market.” Journal of Finance 39(4), 1127–1139. The negatively autocorrelated errors of the bid-ask bounce demo.
Energy and its relatives
- Székely, G. J., and Rizzo, M. L. (2017). “The Energy of Data.” Annual Review of Statistics and Its Application 4, 447–479. Statistics built from a potential-energy functional of distances.
- Sejdinovic, D., Sriperumbudur, B., Gretton, A., and Fukumizu, K. (2013). “Equivalence of Distance-Based and RKHS-Based Statistics in Hypothesis Testing.” Annals of Statistics 41(5). Energy distance and maximum mean discrepancy are the same thing.
- LeCun, Y. Energy-based models lecture, NYU Deep Learning course. The same energy = negative log-likelihood identification, driving modern machine learning.
- Grathwohl, W., et al. (2020). “Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One.” ICLR 2020.
- Sohl-Dickstein, J., et al. (2015). “Deep Unsupervised Learning using Nonequilibrium Thermodynamics.” arXiv:1503.03585. Diffusion models: thermal jitter as a generative device — the dictionary's last row, industrialised.
Physical computation and visualisation
- Mount, J. (2019). “Lord Kelvin, Data Scientist.” Kelvin's tide-predicting machines: serious inference performed by pulleys and wire.
- Core77 (2016). “When Splines Were Physical Objects.” Draughtsmen's splines: bent wood held by weights, minimising bending energy — smoothing before software.
- Simpson, G. (2016). “Soap Film Smoothers.” Two-dimensional smoothing as a physical minimal surface.
- Strang, G. (2008). MIT OCW 18.085, Computational Science and Engineering I, Lecture 1. Spring-mass chains and their stiffness matrices — the linear algebra under every demo here.
- Healy, K. Animation of least squares as springs (Cross Validated). · Goesh, T. Interactive Desmos spring regression. · Davis, J. 3D spline springs.
- Friendly, M. A bibliography of physics-inspired statistical models. The wider literature, collected.
Know another Gaussian–Hookean correspondence, or a reference that belongs here? Open an issue on the springtheory repo.