Robust smoothing: bounded influence vs Kalman

One dial, from a bounded-influence robust fit to the ordinary Kalman smoother.

The smoother's measurement spring has energy $\tfrac12\phi(x_t-y_t)^2$. The shape is fixed and quadratic. Generalise it to a power law,

$$ E \;=\; \sum_t \frac{\phi}{p}\,|x_t - y_t|^p \;+\; \sum_t \tfrac12\,q\,(x_{t+1}-x_t)^2, \qquad F_{\text{meas}}(t) \;=\; \phi\,\mathrm{sign}(y_t-x_t)\,|y_t-x_t|^{p-1}. $$

At $p=2$ the force is $\phi(y_t-x_t)$. This is an ordinary Hookean spring, its pull growing linearly with the residual. That gives the Kalman smoother, as before. At $p=1$ the force is $\phi\,\mathrm{sign}(y_t-x_t)$. The magnitude is constant, whatever the size of the residual. It is the same constant-tension device as the Lasso demo's pulley. The measurement pull is now bounded. A single point tugs its bead with force at most $\phi$, however far off it sits, so no outlier can drag the fit far. Each bead settles where that bounded pull balances the quadratic springs from its neighbours. Between $1$ and $2$ the influence function $|r|^{p-1}$ grows sub-linearly in the residual $r$. The exponent is a continuous dial between the two.

Point 4 below is a planted outlier. Drag $p$ from $2$ down to $1$ and watch the green curve peel away from the grey dashed line. The dashed line is the ordinary Kalman fit, and the outlier drags it noticeably. The green curve near that point barely moves once $p$ nears $1$. Its influence has saturated. Drag any point yourself to plant a bigger or smaller outlier and see the gap widen or close.

exponent $p$ chain stiffness $q$ obs. mass $\phi$

$p$ reads as max deviation from Kalman fit

This is the “sub-linear gain” framing of robust M-estimation. The influence function $\psi(r)=|r|^{p-1}\mathrm{sign}(r)$ is linear at $p=2$, so a point's influence is unbounded. At $p=1$ it is constant, so influence is bounded. The $p=1$ end is old. Boscovich fitted by least absolute deviations in 1757, working on the shape of the earth, and Laplace took up the method soon after. Minimising absolute error alone lands on the median. The smoothing springs here turn that into the bounded-influence fit on screen. The Huber loss demo takes the other common route to the same place. It keeps the spring quadratic near zero and caps its force beyond a threshold, rather than reshaping the whole curve.

The usual explanation the textbook derivation, in full

Read the power-law loss as an M-estimator. The fitted path minimises a total loss, the reshaped measurement term plus the quadratic smoothing penalty:

$$ E \;=\; \sum_t \frac{\phi}{p}\,|x_t - y_t|^p \;+\; \sum_t \tfrac12\,q\,(x_{t+1}-x_t)^2. $$

Set the gradient to zero at each bead. With the influence function $\psi(r) = |r|^{p-1}\,\mathrm{sign}(r)$, the stationarity condition is a set of estimating equations, one per time slot:

$$ -\phi\,\psi(y_t - x_t) \;+\; q\,(2x_t - x_{t-1} - x_{t+1}) \;=\; 0. $$

At $p=2$ these are linear and collapse to the tridiagonal solve $(\phi I + qL)x = \phi y$. For $1 \le p < 2$ they are nonlinear, and there is no closed form. The standard remedy is iteratively reweighted least squares. Split the influence function into a weight times the residual, $\psi(r) = w(r)\,r$ with $w(r) = |r|^{p-2}$, and each estimating equation becomes a weighted least-squares normal equation:

$$ w_t = |y_t - x_t|^{p-2}, \qquad \big(\mathrm{diag}(\phi\,w) + q\,L\big)\,x = \mathrm{diag}(\phi\,w)\,y. $$

Iterate. The current residuals set the weights, the weighted tridiagonal system is solved, the residuals are recomputed, and the loop repeats to convergence. For $1 < p < 2$ the problem is convex and the iteration settles. At $p=1$ the weight $|r|^{-1}$ diverges as a residual approaches zero, so it needs regularising, and the minimiser need not be unique. The robust fit is the fixed point of these reweighted solves, not a formula.

The physics proof the same fit, read off one reshaped spring

Only one spring on the page changed shape. The neighbour springs stay quadratic, the same chain as the Kalman smoother. The measurement spring is reshaped by the exponent. At $p=1$ its pull stops growing with the residual and saturates at $\phi$. Each bead still rests where its pulls cancel:

$$ \phi\,\mathrm{sign}(y_t - x_t) \;+\; q\,(x_{t-1} - x_t) \;+\; q\,(x_{t+1} - x_t) \;=\; 0 \qquad (p=1). $$

A far outlier tugs with force at most $\phi$, so it cannot drag the bead far. The bead settles where that bounded pull balances its two quadratic neighbours. It is a force balance with a capped measurement pull. It is a bounded-influence fit, not a running median.