Motivation
An edge can move visibly across itself while its motion along itself remains invisible. A corner contains gradients in more than one direction, so a small window around it can provide more information. This distinction is not a matter of how many pixels are counted: repeated measurements in one direction do not measure a second direction. We will make that statement precise with a finite weighted model, then examine how noise changes the usefulness of an otherwise unique answer.
The model below describes a selected local calculation, not a guarantee that real motion obeys brightness constancy. A single velocity is assumed to apply across the chosen window. Occlusion, deformation, illumination changes, and motion boundaries may invalidate that assumption.
Definitions
Let be jointly differentiable in space and time near each . Suppose that, for each , a differentiable path passes through at time and preserves brightness. The chain rule gives , where is that path’s velocity at . Under the common-velocity approximation , these constraints are combined as follows.
Definition 7.2.1
(Windowed flow energy and structure tensor)
Write and for samples , and choose weights . The windowed flow energy is
Let have rows and have entries , so that . The structure tensor and right-hand side are
and is the fitted (normal) equation.
Weights may describe a spatial window or confidence; zero weight discards a sample. The units of depend on the units of the spatial and temporal derivatives: with pixels and frames, is in pixels per frame. Changing the sign convention for the temporal difference changes , so the convention must be reported.
Theorem/proposition
Theorem 7.2.2
(Existence, uniqueness, and unseen directions)
For finite real data and nonnegative weights, attains its minimum. The minimizers are precisely the solutions of . A minimizer is unique exactly when the gradients with positive weights span . If is any minimizer, all minimizers are with .
The same tensor describes the shape of the energy around a minimizer, which is a second, quantitative reading of the theorem.
Concept lensGeometric
Eigenvalues as curvatures of the fitting energy
The symmetric positive-semidefinite has orthonormal eigenvectors with eigenvalues . Since at a minimizer , moving by raises the energy by exactly . Zero means an exactly invisible direction, while a small positive eigenvalue means a shallow, noise-sensitive direction. The size of an eigenvalue depends on intensity scale, derivative units and filters, weights, and window size; a numerical threshold must be calibrated to those choices.
A noise model turns the curvature picture into a variance statement. Suppose the linear observations satisfy , where the gradients and positive weights are fixed; all samples included in this probabilistic statement have , and zero-weight samples are omitted. The sample set and weights are fixed independently of the errors; confidence weights calculated from the same noisy observations need separate analysis.
Proposition 7.2.3
(Unbiasedness and covariance of the windowed estimate)
Let with every , assume and , and suppose is invertible. Then is unbiased for , and
In particular the variance of is .
This formula requires the stated noise covariance; spatial window weights alone do not imply it. It also treats gradients as exact, whereas noisy derivative estimates create an errors-in-variables problem. It does not account for systematic brightness changes or approximation error from a nonconstant velocity.
Proof sketch or proof idea
The first proof is the geometry of least squares; the only point needing care is the singular case.
Proof
(Projection, normal equations, and a shared kernel)
The range of is a closed subspace of the finite-dimensional data space. Orthogonally project onto it and choose whose image is that projection. The Pythagorean theorem shows this minimizes . A vector minimizes exactly when is orthogonal to the range, which is equivalent to , or . For every ,
If the identity gives ; conversely gives . Consequently consists exactly of directions perpendicular to every positively weighted gradient. If solves the normal equations, their full solution set is . It is a singleton exactly when that kernel is zero, equivalently when the weighted gradients span the plane.
Proof X-ray
Why a singular tensor still has a solution
The proof never inverts . Existence comes from the projection of onto , which exists for any matrix, so the normal equations are consistent even when is singular; only an inverse is unavailable. The single identity does the remaining work twice: it shows , so the ambiguity of the normal equations is exactly the ambiguity of the data, and it expresses that kernel through the samples, as the directions perpendicular to every gradient with . Nonnegativity of the weights is what makes real; a negative weight would break the factorization and with it the claim that every solution of minimizes .
The covariance statement is a direct linear computation.
Proof
(Error representation and covariance)
Let have rows , so and . Hence , whose expectation is zero. Multiplying covariance matrices,
For a unit eigenvector, .
Worked examples
Worked example 7.2.4
(Two gradient directions and a noisy observation)
Take unit weights, , , , and . Then
The two horizontal equations disagree; least squares averages their targets. The residuals are , so the minimum objective is , not zero. A positive-definite tensor establishes a unique fitted velocity even when the equations cannot all be satisfied exactly.
Worked example 7.2.5
(Full rank can still be fragile)
Let , with unit weights. Then is invertible, but a perturbation of in changes by and the vertical estimate by one. With independent observation noise of variance , the vertical variance is . Algebraic uniqueness alone is not a reliability certificate.
Worked example 7.2.6
(A zero weight removes a direction)
Take , with , and , with weight . If , then and , so the minimizers are for every real . For any , and , so the unique minimizer is . Uniqueness switches on at any positive weight, but under the matched noise model the vertical variance is , which grows without bound as . The boundary between “unique” and “invisible” is sharp algebraically and gradual statistically.
Common mistakes
Counterexample mode
More pixels do not necessarily resolve the aperture problem
For any number of samples with and , every velocity fits perfectly. Adding another identical sample increases horizontal weight but leaves the vertical eigenvalue zero. The minimum-norm answer is a convention, not evidence that vertical motion vanished.
A ridge penalty with replaces by and produces the unique estimate . Under the linear model with , even when is singular, and so the expected coefficient satisfies . The penalty can stabilize a weak direction, but introduces bias and cannot manufacture a measurement in a null direction.
Common mistake
(Concluding that a singular tensor means no solution)
When the inverse does not exist, but is still consistent, because always lies in . The correct conclusion is a whole line (or plane) of minimizers, , not an empty solution set. Reporting “no flow” at an edge discards the component that the data do determine.
Common mistake
(Using a fixed eigenvalue threshold across intensity scales)
Multiplying all intensities by multiplies every by and by . Converting an image from the range to therefore multiplies both eigenvalues by , while the fitted velocity is unchanged. A threshold such as is meaningful only together with the intensity scale, derivative filter, weights and window size it was chosen for.
Summary
A window accumulates directional evidence in a positive-semidefinite tensor. Its kernel describes exact ambiguity, and its small eigenvalues identify directions in which noise can be strongly amplified. A unique linearized fit, a reliable fit under a specified noise model, and a correct physical motion estimate are three different claims.
Exercises
Quick check
(With unit weights, take , and , . Find , , and the fitted velocity.)
Solution · Answer
and , hence . Both equations are satisfied: and .
Quick check
(Window data produce and . Give all minimizers of and the ridge estimate for , and explain its second coordinate.)
Solution · Answer
The solutions of are for arbitrary . The ridge estimate is . The zero second coordinate is chosen by the penalty; the observations supply no information in that direction.
Quick check
(Under the covariance assumptions of the proposition, compare the variances along the two eigenvectors if the eigenvalues are and . Does this alone prove the common-velocity assumption?)
Solution · Answer
The variances are and , respectively: the weaker direction has nine times the variance. No: this calculation is conditional on the assumed linear model and does not test common velocity, brightness constancy, or occlusion.