Motivation
A homography estimator starts with proposed correspondences; it does not explain where those proposals come from. A well-chosen local feature should be locatable in both images and distinguishable from other patches. An edge may be visible but poorly located along its own direction, while a repeated window can be sharply localized yet look like many other windows. These are different failures. Matching and robust model fitting must address them before blending can produce a coherent mosaic.
After pairwise alignment, a chain of small errors can accumulate into a visibly broken loop. We study a finite translation model to see exactly what global adjustment accomplishes. It is deliberately narrower than a full panorama: rotation, projective warps, exposure, parallax and moving objects require richer models and cannot be repaired by this translation solve alone.
Local structure and normalized patches
Definition 1.30.1
(Structure tensor of a window)
For a differentiable intensity image and a finite window of sample locations , let nonnegative weights be fixed. For a small displacement , Taylor expansion gives . Its weighted squared change is approximately , with the symmetric positive-semidefinite structure tensor
The eigenvalues measure change in the corresponding local directions, in intensity-squared per coordinate-unit squared. If both are small, the patch is flat. If one is large and the other near zero, an edge constrains motion across it but barely constrains motion along it: the aperture ambiguity. Two substantial eigenvalues mean the linearized local displacement is constrained in two directions; they do not prove that a visually similar patch elsewhere is absent. Thresholds depend on intensity scale, window, smoothing and noise. The expansion assumes the same local image structure and a sufficiently small displacement; occlusion, large displacement and nonrigid change violate it.
Proposition 1.30.2
(Identifiability in the quadratic model)
For weights and every , . Moreover exactly when for every with , and is positive definite exactly when the gradients at those locations span .
Proof
(Expanding the quadratic form)
Since , summing with weights gives the stated formula. A sum of nonnegative terms vanishes only when every term does, and a term with vanishes only when . Hence is positive definite exactly when no nonzero is perpendicular to all the positive-weight gradients. That fails to happen exactly when those gradients are not all contained in one line through the origin, that is, when they span the plane.
Proof X-ray
Why an eigenvalue is only a local statement
The proof needs only two things: the rewriting of the quadratic form as a sum of squares, and the nonnegativity of the weights. The rewriting is the key move. It turns the algebraic condition "both eigenvalues are positive" into the geometric condition "the gradients are not all parallel". In the first worked calculation every gradient lies on the -axis, and is perpendicular to all of them, which is exactly the aperture ambiguity.
The result concerns the linearized displacement of one window. It does not assert that the same patch is absent elsewhere in the image, and it needs the small-displacement expansion to be valid.
Definition 1.30.3
(Normalized correlation of two patches)
A separate descriptor compares the pixel vectors of two fixed, corresponding-size patches . Write and . If both centered vectors are nonzero, their normalized correlation and squared normalized distance are
Both quantities are defined only when both centered vectors are nonzero. A constant patch has zero centered norm, so the descriptor is undefined there.
Proposition 1.30.4
(Normalized correlation and distance)
Let and be nonzero. Then and . Moreover exactly when for some and real , and exactly when this holds with .
Proof
(Expanding the square and Cauchy–Schwarz)
For unit vectors , . Cauchy–Schwarz gives , with equality exactly when for some real , and then has the sign of . If , then . Conversely, gives and , so .
Normalization suppresses an affine brightness offset and positive gain under this model, but discards that information and does not make a patch invariant to arbitrary viewpoint, occlusion or saturation.
For each detected point one can compare descriptors in a search region, retain a best candidate only if it is sufficiently distinct from alternatives, and optionally require mutual nearest neighbors. A nearest/second-nearest distance ratio must handle zero or tied distances explicitly; repeated textures are exactly where ambiguity matters. None of these local rules certifies a match. Fit a geometric model with the earlier robust-sampling procedure, inspect inlier residuals and spatial support, and use independent checks when possible. A large consensus on a repeated facade can still be systematically wrong.
An anchored global translation model
Definition 1.30.5
(Anchored global translation objective)
Let images be vertices of a finite undirected overlap graph. Choose an orientation for each edge and a measured translation , interpreted as in a common coordinate convention. Give each edge positive weight . Anchoring , minimize
This is a model for measurements already expressed in compatible coordinates, not a general formula for composing arbitrary image homographies. Anchor one vertex in each connected component if the graph is disconnected. If a component has no anchor, adding the same vector to all its leaves unchanged.
Theorem 1.30.6
(Existence and uniqueness after anchoring)
If the finite overlap graph is connected and all edge weights are strictly positive, fixing gives a unique minimizer. Exact satisfaction of every edge is possible exactly when the signed sum of measurements around every cycle is zero.
Proof
(Uniqueness from positive definiteness)
To prove uniqueness, write for the oriented incidence matrix with the anchor column removed, and for the diagonal matrix of weights. For each coordinate, . Its normal equations are . If , every edge difference of the vertex values is zero. Connectivity makes the values constant, and the anchored value forces this constant to zero. Thus is positive definite, so the normal equations have a unique solution and completing the square shows it minimizes .
Proof
(Cycle sums and exact consistency)
For exact consistency, edge differences telescope around each directed cycle, with a minus sign when an edge is traversed against its orientation. Conversely choose a path from the anchor to each vertex and sum signed measurements. If every cycle sum is zero, the path sum is independent of the path: a difference of paths is a closed walk, decomposable into cycles and backtracked edge pairs that cancel. The resulting vertex values satisfy every edge. A tree has no cycle test and always admits an exact assignment, even when its measured translations are inaccurate; external evidence is still needed.
Counterexample mode
Anchoring one vertex does not fix a disconnected graph
Take three images with a single measured edge , in one dimension, and anchor . Then does not involve . Both and give , so the minimizer is not unique: the isolated image can be placed anywhere. The repaired statement needs a connected overlap graph, or one anchor in each connected component.
Worked calculations
Worked example 1.30.7
(A window with a visible but ambiguous edge)
Suppose two unit-weight gradients are both . Then and . The local quadratic predicts no change for : the edge locates horizontal displacement but not vertical displacement. Replacing the second gradient by gives , which is positive definite. It makes both small displacement directions observable in this model, though a repeated corner elsewhere can still create a false match.
Worked example 1.30.8
(Normalize two patches)
For and , the centered vectors are and . Both normalized vectors are , so and normalized squared distance is zero. This is consistent with . It also shows the cost: the descriptor cannot distinguish those two absolute brightness patterns.
Worked example 1.30.9
(Distribute a loop discrepancy)
Take three images with one-dimensional edge measurements , , , all with unit weight, and set . The two-edge chain suggests , while the direct edge requests . Instead minimize
The normal equations are and , giving , . The signed residuals on edges are and the minimum sum of squares is . The cycle discrepancy is ; no adjustment can satisfy all three measurements exactly. The optimum distributes the inconsistency according to this objective and its weights, not according to knowledge of which measurement is wrong. The same calculation applies independently to each coordinate of two-dimensional translations.
Counterexample mode
A low local error is not a correct panorama
Imagine identical windows on successive floors. A one-dimensional model shows the failure exactly. Let the row be and use windows of length three. The windows starting at positions are all , with centered vector , so each has normalized distance from the others. A patch at position therefore has three candidates at distance zero. The nearest/second-nearest ratio is , and a mutual nearest-neighbor rule can select any of them. A geometric consensus built on the offset-two choice would be perfectly self-consistent and wrong by two positions. Resolve ambiguity using additional distinctive texture, spatial and multi-view constraints, and independent visual or scene evidence; none guarantees correctness in every scene.
Concept lensAlgorithmic
Local matching and global adjustment answer different questions
The tensor tests small changes around one patch, the descriptor ranks possible patch partners, robust fitting selects a model-consistent subset, and global adjustment reconciles accepted edges. Each stage inherits errors from the previous one. Recompute residuals after adjustment and inspect disconnected overlaps, repeated structures and violations of the shared projection model before blending.
Common mistakes
Common mistake
(A contrast-reversed patch is not a match)
For and the centered vectors are and , so and the normalized squared distance is , the largest possible. A matcher that uses accepts such a pair as a perfect match. Only certifies a positive-gain relation .
Common mistake
(Normalizing a constant patch)
For the centered vector is zero, so has a zero denominator. Adding a small constant to avoid the division produces a number that carries no information about the patch. Detect flat patches first and exclude them from matching.
Common mistake
(Expecting the optimum to name the bad measurement)
In the loop example the residuals on the three edges are , all of the same size. The solve spreads the inconsistency according to the weights; it does not say which of the three measurements is wrong, and a tree of edges exposes no inconsistency at all.
Practice and solutions
Quick check
(Two unit-weight gradients are and . Compute and its eigenvalues.)
Solution · Answer
Adding the two outer products gives , with eigenvalues 2 and 2.
Quick check
(Compare and using normalized correlation. Is this a positive-gain match?)
Solution · Answer
The centered vectors are and , so . They differ by negative gain, not positive gain.
Quick check
(For a graph with edges and only, measurements are 4 and 7. With , solve the exact translation model. Can the graph expose an erroneous first measurement from cycle inconsistency?)
Solution · Answer
and give zero residual. The graph is a tree, so it has no cycle to expose an error.
Quick check
(In the three-edge example replace by 5. Find the minimum and explain why.)
Solution · Answer
The edge measurements now sum consistently: . The unique anchored solution is , and its nonnegative objective is zero.