Why some panoramas align and others double
Imagine taking two photographs while turning on the spot. Distant buildings often align well, but a nearby railing may appear twice. It is tempting to blame the feature matcher or to make the blending region wider. Neither action addresses the first mathematical question: does one image transformation actually describe every visible point? A homography is an exact common model under particular geometric assumptions. Its eight free projective parameters do not give it permission to fit an arbitrary three-dimensional scene.
This note separates two exact models: a camera that rotates about its optical center, and a translated camera viewing one plane. It then constructs a scene for which translation produces incompatible mappings. We assume ideal pinhole cameras, corrected lens distortion, a rigid stationary scene, known nonsingular intrinsic matrices, instantaneous exposure, and correspondences of the same physical points. Occlusion can remove a correspondence; no warp reconstructs an unseen surface merely by aligning its neighbors.
Coordinates and the direction of the map
Definition 1.32.1
(Relative pose and calibrated ray)
Let be a scene point in the first camera frame. Its second-frame coordinates are
Thus map camera 1 coordinates into camera 2 coordinates. A pixel is a homogeneous column and satisfies , where means equality up to a nonzero scalar. The calibrated vector is a ray direction. There is an unknown nonzero scalar with . With the standard intrinsic normalization, and is first-camera depth. For a visible point require positive depth in both cameras and finite image coordinates.
Substituting rather than guessing the warp gives
The last term is the central issue: a translation is divided by depth. Different depths generally produce different image displacements.
Theorem 1.32.2
(Exact pure-rotation homography)
If the camera centers coincide, so in the stated relative-pose convention, every common visible scene point satisfies with . The matrix is invertible. No planarity or common-depth assumption is needed, but the dehomogenizing denominator must be nonzero for the particular point.
Proof
(Depth cancels for a common camera center)
Write , where is the first-camera depth. For zero translation , so . Homogeneous equality removes the scalar , giving . Each factor is invertible, so is invertible.
Proof X-ray
Where the unknown depth disappears
The decisive step is that multiplies the whole vector , so projective equality can discard it. With translation present, the vector is and only the first term carries . Dividing by the common depth leaves , which depends on the scene point and cannot usually be absorbed into a single matrix. The hypothesis is therefore used exactly once. The proof is silent about the converse: the plane case below shows that a homography can be exact for a translated camera, so a fitted homography need not have arisen from pure rotation. Invertibility of the factors gives invertibility of only; whether the denominator of a particular point is nonzero is a separate condition.
“Turning on the spot” is only an approximation to this hypothesis. A camera held at arm's length moves its optical center along an arc as the photographer turns. A tripod whose rotation axis misses the optical center also introduces translation. Rolling shutter, lens distortion, scene motion and exposure changes are separate departures from the assumptions.
A rotation gives a rational pixel map
Worked example 1.32.3
(A 3–4–5 rotation with a checkable pixel prediction)
Use identical cameras with and zero principal point. Take
For a first-image pixel , matrix multiplication gives the homogeneous second-image vector . Division by its third entry yields , approximately . The first-camera ray is and its rotated direction is , confirming the same answer. Putting the point at depth or multiplies all three camera coordinates by that depth and changes neither projected coordinate. The positive value also confirms positive second-camera depth for both choices.
The displacement is not the constant throughout the image. Rotation changes the denominator with . A translation-only image model would lose this perspective effect even though camera translation is zero.
The source-image line , equivalently , consists of rays with zero second-camera depth. These directions lie in the second camera's principal plane; sends the source-image line to the line at infinity. At such a ray the inhomogeneous formula has a zero denominator. Invertibility of as a projective map does not imply that every point stays in a finite rectangular image or in front of both cameras.
Common mistake
(Using R instead of its transpose for the reverse map)
The map from image 2 back to image 1 is . In the 3–4–5 example the ray returns to under , which is the first-image pixel . Under it becomes , a pixel near , far from where the point started.
Why a single plane is another exact case
Proposition 1.32.4
(Homography induced by a plane)
Suppose the observed scene points lie on a plane with , written in first-camera coordinates. Then every such point satisfies with .
Proof
(Eliminating the depth with the plane equation)
Because , the plane equation gives , so . Therefore , and the substitution gives the stated matrix.
The plus sign follows from our plane equation and pose convention. Writing the plane as changes the displayed sign; copying a formula without its conventions is unsafe. The plane must not pass through either camera center for this to be an invertible plane-to-plane image homography. Indeed the determinant lemma gives ; its vanishing means the second center, whose first-frame coordinates are , lies on the scene plane. Rays parallel to the plane do not provide finite intersections.
Concept lensGeometric
Rotation as the limit of a receding plane
Fix , and the unit normal , and let the plane recede, . Then , so , the pure-rotation homography. The reason is the term in the derivation: for points far away the depth is large and the translation shift vanishes. The rotation case is the limiting member of the plane family.
Worked example 1.32.5
(A plane at depth two and a sideways translation)
Take , , and the plane , so and . Then
which sends to . This agrees with the direct computation at used in the counterexample below. The translation is not zero, yet one homography is exact on this plane.
Worked example 1.32.6
(A plane through the second camera center)
Keep , and , but take the plane , so and . In first-camera coordinates the second camera center is , which lies on this plane. The matrix is
which is singular, in agreement with . Every point of the plane has and , so the whole plane projects into the line and there is no invertible plane-to-plane image homography.
Common mistake
(A good homography fit does not prove a pure rotation)
In the plane example above the camera moved one unit sideways, yet a single homography is exact. Fit quality alone cannot separate a rotation from a translated planar scene. The scene assumptions, or additional views, must decide.
Counterexample mode
Two depths demand two different shifts
Set both intrinsics to the identity, , and . The second camera center has moved one unit along the first camera's axis, to the right under the usual image-axis convention. A point has normalized image coordinates , , and second-image coordinate , .
Consider four isolated feature points on , without an opaque planar surface filling the space between them, whose first-image coordinates are . They are a projective frame: no three are collinear. Their exact correspondences uniquely determine the homography . Now add the point , whose first-image location is . It actually moves to . The homography determined by the four near points predicts . The horizontal residual is , so no single homography can satisfy these five correspondences. The fifth ray differs from the four foreground rays, so this counterexample does not require two visible surfaces to occupy the same source pixel.
The uniqueness used here is stronger than counting equations. Two invertible homographies agreeing on a projective frame have a quotient fixing that frame. Choose three frame vectors as a basis; fixing their projective directions makes this quotient diagonal in that basis, with entries . The fourth vector has three nonzero coordinates in that basis, and fixing its projective direction forces all three diagonal scales to agree. The quotient is a scalar matrix, so the projective maps coincide.
From geometry to an honest pipeline
A robust estimator may choose the dominant plane and reject the fifth point. That can be appropriate when the plane is the intended subject; it does not make the entire scene obey the fitted model. Plot residual vectors against image location and scene depth when available. Consistent depth-dependent residuals suggest model error rather than merely poor optimization. Comparing transfer errors requires the same coordinate units and valid denominators.
For approximately distant scenes, translation may be negligible at the required output scale. To quantify one exact case, assume identical intrinsics with horizontal focal length in pixels, no relative rotation (), and . Let the two surfaces be frontoparallel depth planes and in camera 1. If the plane at is aligned exactly, the horizontal residual for a point at , defined as actual minus predicted position, is . This formula does not apply unchanged under general rotation or forward translation, which alter the projection denominator. Its magnitude is the useful tolerance comparison. With , , , , the mismatch is pixels. “Distant” therefore depends on baseline, focal length and the permitted error; it is not a universal distance threshold.
Cylindrical or spherical output coordinates can reduce perspective stretching for wide rotational panoramas. They do not remove depth-dependent parallax caused by translation. A seam can hide some disagreement and blending can soften intensity changes, but neither recovers a geometrically consistent common point map. Local warps, depth-aware reconstruction or multiple planes change the model and require their own assumptions.
Practice with solutions
Quick check
(For a pure rotation find the image-2-to-image-1 homography.)
Solution · Answer
Invert the factors in reverse order: . Using instead of applies the camera-1-to-camera-2 rotation again rather than undoing it.
Quick check
(Two frontoparallel surfaces have depths and meters. With pixels, what horizontal baseline makes the relative mismatch at most pixels?)
Solution · Answer
Require , hence meters. This bound assumes the stated translation geometry, not arbitrary motion and slanted surfaces.
Quick check
(A nonsingular candidate maps between cameras with known . What necessary algebraic test distinguishes a pure rotation candidate from a generic homography?)
Solution · Answer
Let . Because has arbitrary scale, require for some . If it holds, divide by and choose its projective sign so the determinant is . This yields a rotation matrix; also check positive depth on the actual observed rays. Passing this algebraic test establishes compatibility with a rotation model, not proof that a real translated planar scene could not have generated the same observations.
Quick check
(Why did the derivation exclude ?)
Solution · Answer
The equation no longer determines depth. Points on this plane project to the image line , so the plane has collapsed to a line rather than a two-dimensional image region. Dividing by would conceal this geometric loss of information.