Motivation
A photograph of a rectangular noticeboard usually contains a quadrilateral. Straight edges remain straight, yet equal lengths on the board need not have equal lengths in the image. Before trying to straighten the photograph, we need a model that explains which geometric properties survive imaging. Perspective projection supplies such a model. It also explains why a single correction cannot generally align a nearby tree and a distant building photographed from two different positions.
Computational imaging joins several questions that are easy to confuse. Where should an image point move? What continuous intensity should be assigned at that location? Which discrete output pixels should be recorded? The first is geometry; the second is reconstruction from samples; the third is sampling. Solving the first question exactly does not solve the other two. A perfectly estimated geometric transformation can still yield jagged edges or blurred texture.
We develop an ideal pinhole model and use homogeneous coordinates to express planar mappings. Real lenses also have distortion, finite apertures and imperfect calibration. Those effects matter in practice, but introducing them before understanding perspective can hide the reason the basic equations work. The objective here is a careful mathematical baseline, with explicit limits on when its conclusions can be trusted.
Definitions
Definition 1.1.1
(Perspective projection and the camera matrix)
Use a camera coordinate system with optical center at the origin and a virtual image plane at , where . A visible scene point , with , projects to
These coordinates follow by intersecting the ray with the plane: . A physical sensor behind the pinhole gives an inverted image; the virtual plane convention avoids carrying that sign through every equation. Pixel coordinates additionally incorporate focal scaling, a principal point, and possibly skew. In homogeneous form,
Here is a world point, is a rotation, is a translation expressed in camera coordinates, and is an invertible intrinsic calibration matrix. The symbol means equality up to a nonzero scalar; it never means equality component by component. For with , dehomogenization gives .
Definition 1.1.2
(Planar homography and homogeneous lines)
A planar homography is a projective mapping with an invertible matrix , itself defined only up to nonzero scale. Finite image coordinates exist only where the third coordinate of is nonzero. A homogeneous line is represented by a nonzero vector , and its points satisfy . The representation includes points at infinity, which organize directions but are not finite sensor locations.
Definition 1.1.3
(Bilinear reconstruction)
A digital image is an array . To evaluate it away from integer coordinates we choose a reconstruction rule, such as bilinear interpolation. For a location with , the interpolated intensity is
This is a weighted combination of four samples, not a newly measured scene value. Boundary handling and color representation must also be specified. To model physical averaging, intensities should be in a linear light representation rather than arbitrary nonlinear display codes.
Theorem/proposition
Theorem 1.1.4
(Why a plane induces a homography)
Parameterize a scene plane by , where is a matrix and are homogeneous plane coordinates. Let two camera matrices be . If and are both invertible, corresponding plane images obey
The invertibility assumptions exclude a plane viewed degenerately, for example a plane containing a camera center.
Theorem 1.1.5
(Lines and cross-ratios survive a homography)
An invertible homography maps projective lines to projective lines, with . For four distinct collinear points with finite affine coordinates , their cross-ratio
is preserved wherever the displayed expressions are defined. Lengths, angles and ordinary ratios of distances are not generally preserved.
A further useful special case is a camera that rotates about a fixed optical center. With compatible camera coordinates and no translation, corresponding rays satisfy . This homography applies across scene depths. Rotation about the center is the relevant condition; casually rotating a handheld camera may also translate it and introduce parallax.
Proof sketch or proof idea
Proof
(Eliminating the plane coordinates)
On the plane, camera one gives . Since is invertible, . Substituting into camera two gives the claimed mapping. This eliminates plane coordinates rather than reconstructing every point in three dimensions. If is singular, that elimination fails: distinct plane points may project to the same image point, so an inverse image-to-plane mapping does not exist.
Proof X-ray
The hypothesis that makes one matrix possible
Both invertibility hypotheses are used, but at different moments. is spent once, to solve : it lets the first image be decoded back to the plane's own coordinates. The non-obvious move is to route the argument through , which both cameras share, instead of through three-dimensional points. plays no part in deriving the relation. It matters only afterwards, because is invertible exactly when is, and only then is the map a homography. The argument closes because the same three homogeneous coordinates feed both cameras.
A general three-dimensional scene has an additional depth freedom: four homogeneous coordinates replace the plane's three. A second camera position can reveal the missing freedom as parallax, which no single image-plane homography can remove for all points.
Proof
(Lines and cross-ratios under a homography)
For the line statement, write up to scale. Then , so the transformed line is represented by . For cross-ratios, a projective mapping restricted to a line has the form with . Differences transform as
Substituting four such differences cancels every determinant and denominator factor. The cross-ratio remains. Cases involving infinity are handled by limits or homogeneous determinants; the finite formula should not be evaluated by dividing by zero.
Four point correspondences can determine a homography when both point quadruples are in general position, meaning no three are collinear. Each match supplies two independent homogeneous linear constraints on nine matrix entries. Scale leaves eight degrees of freedom. The degree count alone is insufficient: collinear or repeated points produce dependent constraints. With more noisy matches, one commonly estimates a least-squares algebraic fit after coordinate normalization, then assesses geometric reprojection error. The algebraic residual depends on coordinate scaling and is not itself a pixel distance.
Worked examples
Worked example 1.1.6
(Depth changes apparent size)
Take in pixel units. The scene points and project to horizontal coordinates and . A horizontal segment of world length one at constant depth four therefore spans pixels, while the same length at depth eight spans pixels. This is not a change in focal length. It is the division by depth. A pixel displacement alone cannot identify a world displacement without depth or another geometric constraint.
Worked example 1.1.7
(Rectifying a simple projective distortion)
Let
Then maps to . The square corners become . The inverse matrix changes the bottom-left entry to . Applying it to the four transformed points recovers the square. The line maps to infinity, so the forward formula cannot be used there as a finite image warp. In a real rectification one must choose an output domain that avoids such a singular boundary.
Counterexample mode
Midpoints are not preserved
A tempting claim is that a homography sends the midpoint of a segment to the midpoint of its image. For the matrix above, the segment from to has midpoint , which maps to . The image segment runs from to , whose midpoint is . Since , the claim fails. What survives is the cross-ratio of four collinear points, not ordinary ratios.
Worked example 1.1.8
(Inverse warping and interpolation)
Suppose a source image is translated by into the output. For output pixel , inverse mapping requests the source at . Its four bilinear weights are on . If those intensities are , the result is . Mapping source samples forward and rounding their destinations instead could leave some output pixels empty and send several samples to one pixel. Inverse warping assigns each valid output location a source query, while still requiring an explicit rule for queries outside the source domain.
The following is a separate scaling example, with a different source image from the translation example above. Its source values are , arranged in two rows. Scaling horizontally by and vertically by sends the output point back to . Bilinear interpolation therefore uses weights and returns . Compare the forward writes with the inverse query; then try the boundary and collision cases. A missing forward write is not the same thing as a written value of zero.
An implementation should therefore separate three functions: an output-domain mask, an inverse geometric map, and a reconstruction filter. An invertible guarantees a projective inverse, but does not guarantee that every output query lies in the photographed rectangle. Image validity is a separate issue from matrix invertibility. Uncovered locations should remain marked as unknown until another source image or a stated filling model supplies them.
Concept lensAnalytic
A warp changes the sampling footprint
Locally, the Jacobian of the inverse map describes which source neighborhood contributes to an output pixel. If one output pixel covers many source pixels, evaluating a single bilinear point can miss alternating texture. Prefiltering should average over a suitable source footprint before downsampling. Interpolation fills between samples; antialiasing controls which variations survive a coarser sampling grid.
For a one-dimensional illustration, the signal sampled at integer locations cannot be distinguished from there. At , both give the same integer samples as frequency . A later warp cannot decide which continuous signal was originally present. The remedy is to restrict the signal bandwidth before sampling, using optical or digital filtering appropriate to the acquisition or resampling step.
Common mistakes
Counterexample mode
A homography cannot generally remove parallax
Two points on the first camera's optical ray are and . Both project to with unit focal length. Translate the second camera center to without rotating it. Their second-camera coordinates are and , giving image points and . One input point would need two different outputs. No single-valued image warp, and hence no homography, can satisfy both correspondences. The occluded point may not be simultaneously visible, but the example isolates the depth ambiguity exactly.
Common mistake
(A homogeneous third coordinate is not depth)
After a general projective transformation, the third coordinate of is only the divisor of the chosen representation: and are the same point. Depth belongs to the scene point in camera coordinates and cannot be read off .
Common mistake
(Normalizing by an entry that may be zero)
Setting is a common convention, but it is valid only when . The invertible matrix with rows has : it sends to , so it cannot be rescaled to have . Choose a different entry, or normalize the Frobenius norm instead.
Common mistake
(Judging a model by one well-fitting patch)
A wrong model can match a small region and leave systematic error elsewhere. A homography fitted to points on one wall can overlay that wall closely while an object off the wall, such as a tree in front of it, drifts by parallax. Check the overlay across the whole image, and especially on points not used for the fit.
Finally, repeated image resampling causes cumulative filtering. Composing geometric maps first and resampling once usually preserves more detail than repeatedly warping already interpolated images. This does not eliminate the need for antialiasing, nor does it repair an incorrect camera model. Geometry, photometry and sampling errors should be diagnosed separately because they require different remedies.
Summary
Perspective projection divides lateral position by depth. Homogeneous coordinates make planar projective mappings linear up to scale, and expose their exceptional points. Planes and pure camera rotation give exact homographies under the stated idealizations; arbitrary translated views of a three-dimensional scene do not. A practical warp also needs an output domain, interpolation and antialiasing. Those numerical decisions have information limits that geometric accuracy cannot remove.
Exercises
Quick check
(A camera has and principal point , with square pixels and zero skew. Project in camera coordinates. If depth doubles while stay fixed, find the new pixel and explain which displacement halves.)
Solution · Answer
The pixel is . At depth ten it is . The displacement relative to the principal point changes from to . The absolute pixel coordinates do not halve because the principal point remains fixed.
Quick check
(For the homography in the rectification example, map the line . Give its finite Cartesian equation in the output, and verify it using two mapped points.)
Solution · Answer
A source line vector is . Multiplying by gives , hence . The source points and map to and ; both satisfy that equation. Perspective preserves straightness here, but changes the line's slope.
Quick check
(A horizontal source row has samples at positions . Translate by to the right. Find the inverse-warped value at output location under linear interpolation. Explain why the value at output location needs a boundary rule.)
Solution · Answer
Output location two requests source location . The weights are and on the samples with values and , giving . Output zero requests , outside the sampled row. Zero padding, reflection and an invalid-data mask give different answers. None follows solely from the translation matrix.
Quick check
(Four proposed matching source points are . Explain why they cannot uniquely determine a two-dimensional homography, even with exact matches. Construct two distinct invertible homographies that agree on the source line .)
Solution · Answer
The matches constrain only the restriction of the transformation to one projective line. Identity and both fix every point while acting differently off the line. Both matrices are invertible and are not scalar multiples. Thus even infinitely many exact correspondences on this line would leave the action in the second image direction unresolved.