A photograph turns a three-dimensional scene into a two-dimensional array, so some information must be lost. The pinhole model identifies that loss precisely: points along the same camera ray have the same image position. A change of viewpoint can separate those points, producing depth-dependent motion called parallax.
A single matrix can nevertheless describe the image-to-image motion of an entire plane. This is the basis of many geometric image-warping and mosaicing constructions. We will derive that matrix, track the coordinate conventions that determine its signs, and identify the geometric condition under which it has an inverse. A correct matrix relation is still separate from whether a point is in front of a camera or visible behind another surface.
Project onto a virtual image plane
Use camera coordinates , with the camera center at the origin and the viewing direction along positive . In the ideal model, light follows the straight line joining the center and the scene point. Its intersection with the virtual plane has coordinates , provided .
The virtual plane is placed in front of the center so the formula has no image-inversion minus sign. A physical sensor can be behind the pinhole; its orientation and coordinate convention must then be handled consistently rather than silently changing signs halfway through a calculation.
Definition 14.1.1
(Intrinsics and homogeneous image coordinates)
Represent a finite pixel by the nonzero homogeneous vector . Two nonzero vectors represent the same projective image point when they differ by a nonzero scalar; write for that relation. A vector gives a finite pixel only if . The zero vector is never a projective point.
For an ideal calibrated pinhole camera, let
Here convert normalized image coordinates into pixel units, is the principal point, and is skew. Projection is
The camera matrix is not a rule for declaring every projected point physically visible. Positive depth is required for the forward-looking camera convention. Negative-depth points still have algebraic projective coordinates, but are behind the camera. Points with have no finite image coordinate in this chart; at the center itself the homogeneous result is zero and projection is undefined. Lens distortion and the finite image boundary are also outside this ideal model.
Distinguish the camera center from its translation column
Let a world point be , and map world coordinates into camera coordinates using
Thus is a proper rotation and is the translation in this world-to-camera map. The camera center is the world point that maps to the camera origin. Solving gives
The last formula combines a three-dimensional affine map with homogeneous projection. Its camera matrix has a nontrivial kernel; it is not an invertible change between four-dimensional and three-dimensional coordinates.
Worked example 14.1.2
(Moving the camera right can move the image left)
Take and camera center . Its translation column is , not . A fixed world point therefore has camera coordinates .
With , zero skew and principal point , its pixel is . A camera of the same orientation and intrinsics at the world origin instead sees pixel . The camera moved right by one world unit, while this point moved left by half a pixel. Confusing with would predict the opposite shift.
Invert a pixel into a ray, not a three-dimensional point
Because is invertible, a finite pixel determines . The last coordinate of is 1 for the normalized representative . The corresponding world points are
Indeed, , and multiplying by gives , the same image point. Conversely, a world point of positive camera depth that projects to this pixel has exactly this form. The parameter is its camera-coordinate depth , not generally its Euclidean distance from the center, because need not be a unit vector.
Without extra information, all positive values of are possible. An inverse calibration matrix therefore recovers a direction, not the missing depth. A known surface, a second view, or another geometric constraint may supply that depth; the pixel alone does not.
Use a known plane to eliminate depth
Now take camera 1's coordinates as the reference coordinates . Camera 1 projects using and camera 2 uses
Suppose the scene points lie on the known plane , where and . The normal need not be a unit vector. Scaling and by the same nonzero constant does not change the plane. The condition excludes a plane through camera 1's center, which would itself project to a line rather than fill a two-dimensional image region.
For , put . A point on its ray is . The plane equation yields . If , the ray has no finite intersection with this plane; otherwise its intersection has depth . For physical use we also require this depth, and the resulting camera-2 depth, to be positive.
Theorem 14.1.3
(The plane-induced image map)
For the valid plane correspondences described above,
This matrix is invertible exactly when camera 2's center , expressed in the reference frame, does not lie on the plane: . An invertible matrix of this kind represents a planar projective transformation, or homography, up to nonzero overall scale.
Proof
(Substitute the plane equation before removing homogeneous scale)
Using , write
Multiplication by and removal of the common nonzero scale gives the stated image relation. The plus sign follows from our plane convention . If the plane were written , then and the corresponding formula would have a minus sign in front of .
Proof
(Invertibility holds exactly off the plane)
The calibration matrices are invertible, so is invertible exactly when is. Look for a null vector with . Multiplying the equation by gives
If , the scalar is nonzero, and . Applying yields , hence . Conversely, if , then . The vector is nonzero because , so this is a genuine null vector. Therefore is singular exactly when camera 2's center lies on the plane.
In that degenerate case, every point of the plane satisfies
Thus its nonzero image vectors lie on the projective image line with coefficient vector . The plane is seen edge-on. In particular, points on one line through camera 2's center can collapse to one second-image position. The center itself has no image: the nonzero homogeneous input is sent to zero by .
Proof X-ray
How the invertibility argument is built
Three facts carry the argument, and each enters once. Invertible reduce the question to the single matrix . The orthogonality lets us multiply by and meet the center . The hypothesis allows division by , and it also makes when , so the null vector found in the converse is genuine.
The non-obvious move is to notice that has rank one with image spanned by . Any null vector of must therefore be a multiple of , that is, of . A search over a three-dimensional space collapses to testing one candidate direction, and the single scalar equation decides it. The argument closes with the explicit computation . If the plane passes through camera 1's center and is not even defined, which is why that case was excluded before the theorem.
This invertibility test is algebraic. It does not guarantee positive depth, membership in the finite image bounds or absence of occlusion. Even an invertible homography can send some projective points to infinity; valid physical correspondences still require the separate conditions stated above.
Concept lensStructural
A homography as a rotation map plus a rank-one term
Expanding the product gives
The first term is the depth-independent pure-rotation map of the later section. The second has rank one and carries all the dependence on translation and plane. It vanishes when , and also tends to zero as the plane recedes, with fixed.
Work out a plane warp, then change the depth
Worked example 14.1.4
(One plane translates uniformly in the image; another depth does not)
Use the same intrinsics in both cameras:
For the plane , take and . The formula gives
Thus every valid point of this plane shifts left by half a pixel. Point has , and the homography gives , agreeing with direct camera projection.
Now take . It lies on the same camera-1 ray as , so it has the same . Direct projection in camera 2 gives , not . The homography cannot also describe this depth-2 point. These points can be compared as separate possible scene points; if opaque surfaces put both on the same ray in one scene, visibility must additionally select what the first camera actually sees.
Counterexample mode
One source pixel cannot have two outputs under one homography
The previous example gives the same source pixel two different target positions when its unknown depth changes. Any single-valued image map, including a homography, must assign that source position only one result. This proves that the plane-specific map cannot describe arbitrary three-dimensional depths under that translation.
More generally, in the same parallel-camera setup a point at positive depth shifts horizontally by . Different depths produce different shifts. This is parallax, not an interpolation error that a more accurate resampling kernel can remove.
Pure rotation is an important depth-independent exception
If the camera centers coincide, the relative translation is . Then and
There is no plane parameter or depth in this formula. It holds for points at arbitrary depths, provided they have valid projections in both views. The matrix is invertible because are invertible. This explains why an ideal panorama made by rotating about the camera center can use one projective warp without requiring a flat scene.
Worked example 14.1.5
(Pure rotation: finite, behind the camera, and at infinity)
Take , , and a quarter turn about the vertical axis,
so and . Three source pixels show three different behaviors.
- gives , the target pixel . Every scene point with is sent by to , which has depth and the same pixel. There is no parallax.
- gives , the target pixel . But has camera-2 depth . The algebra returns a finite pixel, yet these points lie behind camera 2, so this is not a physical correspondence.
- gives . The third coordinate is zero, so there is no finite target pixel: the ray is parallel to camera 2's image plane.
The matrix is invertible in all three cases; invertibility says nothing about which of these behaviors occurs.
A real movement that also translates the camera center reintroduces depth dependence. A plane model can still be useful for a dominant wall or sufficiently distant scene, but those are scene assumptions or approximations; they should not be confused with the exact pure-rotation result.
Apply a homography and sample an image in separate steps
Given a pixel , first compute . It is a valid projective result only when , and a finite target pixel only when . Divide its first two coordinates by . A homography is not generally an affine map: its denominator can depend on the source location.
For inverse image resampling, an invertible allows a target location to be mapped with to a source location before interpolation. That still does not decide whether the source location is inside the image, whether a plane intersection has positive depth, or whether one surface hides another. Geometry supplies coordinates; boundary rules, visibility and the image reconstruction model supply other parts of a complete rendering or mosaicing method.
Common mistakes
Common mistake
(Changing the plane convention but not the sign)
The formula belongs to the plane written . If the plane is entered as with , then , and using gives the shifted-right matrix with entry instead of . The example above would then predict that moves right, the opposite of direct projection. Either convert or flip the sign in front of .
Common mistake
(Applying where is needed)
When warping an image by inverse resampling, each target pixel asks which source location to read. In the example above shifts source pixels left by , so the target pixel must read the source at , which is applied to it. Applying again would read the source at , a shift in the wrong direction.
Exercises with solutions
Quick check
(Let , and camera center . For pixel , find the world ray and its intersection with . State the translation column and the role of the ray parameter.)
Solution
The world-to-camera translation is . The calibrated direction is , so the world ray is , with . At , and the point is . The parameter is camera depth, not Euclidean distance; the vector from the center to the intersection has length .
Quick check
(Use , , and plane . Derive and the normalized target pixel for source . Which source pixels meet this plane at positive depth?)
Solution
Here , , and the third row of is ; its first two rows are and . Applying it to gives target coordinates whenever .
The reference ray meets the plane at depth , which is positive exactly when . On this physical domain, camera 2's depth is and the displayed denominator cannot vanish. Outside it, the projective formula alone does not establish physical visibility. The location-dependent denominator also shows why the map is generally not an affine shift.
Quick check
(In normalized cameras , let plane and . Find , its invertibility, and the two image pairs for , .)
Solution
With and , the matrix is . The second camera center lies on the plane, so the matrix is singular.
Camera 1 sees at and at . Camera 2 sees both at : subtracting its center gives and . The plane collapses to the image line ; the two distinct reference pixels cannot be recovered from their shared target pixel.
Quick check
(A programmer normalizes the top two entries of Hp by the original pixel's third coordinate, which is 1, and uses one plane homography for foreground and background under camera translation. Identify the two separate errors and explain when one depth-independent matrix is exact.)
Solution
Homogeneous normalization must divide by the result's third entry, , which need not equal 1. Omitting that division mistakes a projective map for an affine one.
A plane homography also depends on the plane depth/orientation and generally cannot model a translating camera's arbitrary foreground and background together. Better interpolation does not repair that geometric mismatch. With coincident camera centers, gives the exact depth-independent matrix for points with valid projections in both views; an actual planar scene is the other setting proved in this lesson, subject to the stated nondegeneracy conditions.