Evidence fusion

The posterior mean is where the pulling stops.

The dark body is the prior. The green bodies are observations. Each carries a mass equal to its precision and is tied by a spring to the white test particle. The particle relaxes to the centre of mass, which is the posterior mean. The bowl above is the potential energy, the negative log posterior, and the particle is a ball rolling to its bottom. Drag any body, or add an observation, and watch the equilibrium re-form. Every observation is another spring in parallel, so the masses, which are the precisions, simply add.

prior position $m$ prior mass $P^{-1}$ new obs. mass $\phi$

posterior mean posterior precision (total mass) particle at

Everything on screen is computed twice. The readout uses the closed-form update $m' = \big(P^{-1}m + \sum_i \phi_i y_i\big)\big/\big(P^{-1} + \sum_i \phi_i\big)$. The particle just integrates $F = ma$ with damping. The two agree. Weighting each observation by its precision is an old habit. Gauss combined observations this way through least squares in the early 1800s, and the inverse-variance weighted average is the one with the least variance. Derivation in the introduction.

The usual explanation the conjugate-Gaussian proof

Proposition. Let the prior be $\mu\sim\mathcal N(m, P)$ and the observation $y\mid\mu\sim\mathcal N(\mu,\phi^{-1})$, with precision $\phi$. Then the posterior is Gaussian, $\mu\mid y\sim\mathcal N(m', P')$, with

$$ (P')^{-1} = P^{-1} + \phi, \qquad m' = \frac{P^{-1}m + \phi\,y}{P^{-1} + \phi}. $$

Proof. By Bayes the posterior is the prior times the likelihood, up to a constant that normalisation fixes at the end:

$$ p(\mu\mid y) \;\propto\; \exp\!\Big[-\tfrac12\big(P^{-1}(\mu-m)^2 + \phi\,(\mu-y)^2\big)\Big]. $$

A sum of two quadratics in $\mu$ is again a quadratic in $\mu$, and the exponential of a quadratic is a Gaussian. So the posterior is Gaussian, and only its mean and precision remain to be found. Gather the powers of $\mu$:

$$ P^{-1}(\mu-m)^2 + \phi\,(\mu-y)^2 = (P^{-1}+\phi)\,\mu^2 - 2(P^{-1}m + \phi\,y)\,\mu + \text{const}. $$

Match this against the standard Gaussian exponent $\lambda(\mu-\nu)^2 = \lambda\mu^2 - 2\lambda\nu\,\mu + \text{const}$. The coefficient of $\mu^2$ is the posterior precision; the coefficient of $\mu$ fixes the mean:

$$ (P')^{-1} = P^{-1} + \phi, \qquad m' = \frac{P^{-1}m + \phi\,y}{P^{-1} + \phi}. \qquad \blacksquare $$

Several observations multiply in several likelihood factors. Nothing changes but the count. The precisions all add, and $m' = \big(P^{-1}m + \sum_i \phi_i y_i\big)\big/\big(P^{-1} + \sum_i \phi_i\big)$.

The physics proof energy, force, and curvature

The negative log posterior is a sum of quadratics, so it is a potential energy. Each belief is a spring whose stiffness is its precision, anchored at its centre:

$$ U(\mu) = \tfrac12 P^{-1}(\mu-m)^2 + \tfrac12 \phi\,(\mu-y)^2. $$

The estimate is the equilibrium, where the net force is zero. Differentiate and solve:

$$ U'(\mu) = P^{-1}(\mu-m) + \phi\,(\mu-y) = 0 \;\Longrightarrow\; m' = \frac{P^{-1}m + \phi\,y}{P^{-1}+\phi}. $$

The precision is the stiffness that resists a nudge, the curvature of the well:

$$ (P')^{-1} = U''(\mu) = P^{-1} + \phi. $$

The mean is a force balance and the precision is a stiffness. Add another observation and it is one more spring, so the stiffnesses add.

Spring Theory the whole trick, drawn

Draw the system and simplify it. One move, and no algebra to speak of.

The prior and the observation pull the particle in parallel. Springs in parallel add their stiffnesses, so the two become one, and the particle rests at their centre of mass. That rest point is the posterior:

$$ z^\ast = \frac{P^{-1}m + \phi\,y}{P^{-1} + \phi}, \qquad (P')^{-1} = P^{-1} + \phi. $$

Each new observation is one more spring in the bundle. Its precision joins the sum.