Motivation
Two consecutive images can reveal how projected features move, but image motion is not automatically the same as three-dimensional object velocity. The registration unit considered one global periodic translation; this unit estimates a local two-dimensional image velocity and examines when that estimate is identifiable.
Brightness constancy and its differential form
Definition 7.1.1
(A local motion model)
Let be a differentiable scalar image intensity, and let a visible image feature follow . Assume its intensity remains constant over a short interval and define image velocity . Differentiating gives
This is one scalar equation for two velocity components at one location. It relies on the assumed feature correspondence and constant intensity; occlusion, lighting changes, reflection and motion blur can violate those assumptions.
For two sampled frames separated by a time interval, derivatives are estimated from data. A first-order Taylor approximation is useful only when the displacement and local variation are suitably small; the residual also contains model and derivative-estimation error. Velocity units must match the derivative convention, such as pixels per frame or pixels per second. A single equation does not determine a general two-component motion vector.
Counterexample mode
An edge cannot reveal tangential motion
If locally , and , brightness constancy gives but places no restriction on . Every satisfies that one equation. A visible edge can constrain motion across itself without revealing motion along itself. This is an identifiability problem, not a reason to select as if it were measured.
Estimating one motion over a patch
Suppose a patch contains observations and the same velocity is appropriate throughout it. Put the spatial gradients in the rows of and set . The least-squares problem is
The shared-velocity assumption can fail across an object boundary. Weighted and robust variants require separately specified weights or losses; the calculation below uses equal weights.
Theorem 7.1.2
(When local least squares has a unique answer)
The normal equations are , where . They have a unique least-squares solution exactly when has two linearly independent columns, equivalently when is positive definite. Then .
Proof
(Expanding the squared error)
For any and any increment ,
If solves , the middle term vanishes, so with equality exactly when . Conversely a minimizer must have a vanishing middle term for every , that is . So the minimizers are exactly the solutions of the normal equations, and two minimizers differ by an element of .
Since , is positive definite exactly when , that is when the columns of are independent. In that case is invertible and is the only solution. If the columns are dependent, a nonzero exists, is singular, and is a minimizer for every because it gives the same predictions .
Proof X-ray
Two independent gradient directions are needed
The only non-obvious move is the expansion with an arbitrary increment . The cross term is exactly what the normal equations kill, and what remains is . Hence the loss grows by in every direction, and the only directions along which it fails to grow are those with .
Independence of the columns of enters at exactly one point, the step from to . Geometrically, the rows of are gradient vectors, and means is perpendicular to all of them. If every gradient in the patch is parallel to one direction , every is invisible. This is the aperture problem in matrix form, and it is the situation of the edge counterexample above. A numerical solver should solve the system without explicitly forming an inverse when appropriate.
Worked example 7.1.3
(Three gradients and a consistent velocity)
Take gradient rows , and , with temporal derivatives . Thus
Here , so and . Both components are identifiable under the patch model. If only the first gradient direction were present repeatedly, vertical motion would remain unconstrained.
Worked example 7.1.4
(A patch whose gradients are all parallel)
Take gradient rows and with temporal derivatives and . Then
The matrix is singular, and the normal equations reduce to the single condition . Both observations are satisfied exactly by every on that line, for example , and . Only the component of the velocity along the gradient direction , equal to , is determined. The shortest solution is the normal flow, which is a convention for the unobserved part rather than a measured motion.
Common mistake
(Reading a zero residual as proof of the motion)
In the parallel-gradient patch above, the residual is exactly zero for every point of the line , yet these velocities differ: and point in different directions. A zero residual says the equations are consistent, not that the velocity is identified. Only a positive definite , together with a motion model that actually holds, identifies it.
Common mistake
(Dropping the sign of the temporal derivative)
The right-hand side is . Using solves the problem with replaced by , and by linearity returns . In the three-gradient example the temporal derivatives give , while using them without the minus sign would give , the opposite direction of motion.
Residuals and conditioning
Worked example 7.1.5
(A noisy patch and its residual)
Replace the last temporal derivative in the first example by , so . Then and the normal equations , give . The predictions are , so the three residuals are , with squared norm . A nonzero residual warns that the equations are not exactly consistent, but a zero residual does not prove that brightness constancy or the common-motion model is physically correct.
The eigenvalues of the example's are and . A very small positive smallest eigenvalue indicates that one direction is weakly constrained, even if is technically invertible. For the specific statistical model with independent zero-mean errors of common variance and fixed full-rank ,
Indeed . The covariance statement requires these assumptions; real derivative errors can be correlated, biased or present in itself.
Practical limits
A textureless region has near-zero spatial gradients, and a straight edge supplies mainly one gradient direction. Corners or textured patches can supply multiple directions, but do not remove occlusion or illumination changes. Enlarging a patch adds observations while also increasing the risk that its pixels have different motions. Coarse-to-fine warping and robust losses can help in some settings, but require their own models and validation; they are not proved by the local equation.
Exercises
Quick check
(For gradients and with temporal derivatives and , find the velocity when the two equations are exact.)
Write the two equations separately.
Solution · Answer
The equations are and , so .
Quick check
(If two rows of are both and both entries of are , describe every exact solution.)
Which component of do the two equations constrain?
Solution · Answer
Every is a solution. The vertical component is not identifiable.
Quick check
(Verify the noisy worked example's normal equations and squared residual.)
Compute for and substitute .
Solution · Answer
Here and . The residual is and its squared norm is .
Quick check
(Under the stated independent-noise model, what are the covariance eigenvalues in the example with eigenvalues and ?)
The covariance is .
Solution · Answer
The eigenvalues of are and . These quantify uncertainty only under the stated noise model.
Further study
Discrete derivative design, pyramidal warping, robust motion fields and occlusion detection require additional analysis. This unit does not implement an optical-flow system or certify the motion in any particular video.