PCA as a rod on sliding collars
Total least squares is the same rod as regression — with different spring attachments.
In the least squares demo each spring is pinned to the rod directly above its data point, so it stretches vertically and the rod relaxes to the regression line. Attach each spring instead through a frictionless collar that slides freely along the rod, and the spring swings perpendicular, storing energy $\tfrac12\sum_i d_i^2$ in the squared orthogonal distances. The resting configuration — centre at the centroid, axis along the direction of maximal variance — is the first principal component, exactly Pearson's 1901 “line of closest fit.”
The rod's torque balance is pendulum-like with period $\pi$: there are two equilibria. The first principal component is the stable one; the second is a balance point — try the button that stands the rod on PC2 and watch it teeter before swinging down to PC1, slowly when the cloud is nearly round (weak restoring torque, eigenvalues nearly tied) and violently when it is elongated. Principal components are the normal modes of the point cloud; eigenvalues are their stiffnesses. Drag points, reshape the cloud with the sliders, or spin the rod.
Vertical springs give regression; sliding collars give PCA; and a spring with anisotropic stiffness would interpolate the two (errors-in-variables). The energy here is the negative log-likelihood of an isotropic Gaussian around the line, which is why the collar construction finds the top eigenvector of the covariance — and why, for a nearly round cloud, the axis is barely identified: the bowl is nearly flat in $\theta$ and the rod wanders. See Pearson (1901) in the reading list.