A surface for a wide field of view
A perspective photograph records rays on a plane. As a ray approaches a direction parallel to the image plane, its image coordinate grows without bound, so one flat perspective canvas is awkward for a wide sweep. A cylinder instead records the horizontal direction as an angle. The picture is still made of samples from the input photographs: the cylinder changes the coordinate chart, not the scene or the camera motion.
Definition 1.31.1
(Pixel rays in the panorama frame)
Assume an ideal pinhole camera, corrected lens distortion, and a static scene. Let the panorama frame have axes right, down and forward. The camera frame uses the same handedness, and a proper rotation maps camera-frame directions into panorama-frame directions. For exact rotational registration, all input cameras share one optical center, chosen as the panorama origin. The panorama axis is the chosen cylinder or panning axis; it need not coincide with gravity. Pixel coordinates increase right and down; for zero skew let
Here denotes a ray, not a scene point at unit physical depth. The chosen rotation direction matters: a panorama direction is brought back to the camera frame by , not by applying a second time. For each input image use its own and intrinsics.
From a ray to a cylinder
Definition 1.31.2
(Cylindrical coordinates of a ray)
Take a unit-radius cylinder around the vertical axis. For , intersect the ray , , with . The unique positive intersection parameter is , hence its height is . Its angular coordinate, measured from toward , is
Choose a branch : reduce the output of modulo into this interval, mapping (including a signed-zero seam result) to . Choose scales in pixels per radian and pixels per unit height. With an origin , the continuous output coordinates are and . The map does not in general preserve input-image area or solid angle: sample density changes across directions. The panorama may cover only a subset of this coordinate range.
Inverse mapping for sampling
Forward mapping tells us where an input ray goes, but rounding every source pixel to an output pixel can leave holes. Instead, for each output pixel center first compute and , reducing the angle modulo into the chosen branch if needed. A representative panorama-frame ray is
If , project it into the input image:
Sample only when lies in the image's valid domain, including whatever neighborhood the interpolation rule requires. If the ray is parallel to the image plane; if it points behind the camera center. Neither direction can be sampled by this forward-facing pinhole image. At division is undefined, even if the cylinder coordinate itself is finite. Select another image if it sees that direction; otherwise leave the output unsupported. Interpolation, filtering before downsampling and exposure compensation address sampling or appearance, not the validity of the geometry.
Concept lensAnalytic
The warp as a change of sampling density
Take and the row . Then , so and . This is the number of input pixels per radian. With the warp is one-to-one at , but at one output column covers input pixels, so reducing sampling error there means filtering before downsampling.
Theorem 1.31.3
(The two coordinate formulas are inverse on their domains)
For any nonvertical ray with , the cylinder construction gives and . Therefore , a positive multiple of the original ray. Conversely, for any finite and angle , this representative has horizontal radius one, and its cylinder height is . Its angle equals modulo . A camera sample additionally requires positive camera-frame depth and a valid input-image location.
Proof
(Round trip between rays and cylinder coordinates)
Fix a ray with . For any scale , the vector has horizontal radius , so and . Every point of the same forward ray has the same cylinder coordinates, so they describe the ray and carry no depth.
By the definition of , and . Hence , a positive multiple of .
Conversely, let be finite, let be any angle, and put . Its horizontal radius is and its cylinder height is . Its angle equals modulo , and reducing into the chosen branch returns the representative of there.
Proof X-ray
Where each hypothesis enters the round trip
Each hypothesis is spent at one place. The condition is used when dividing by ; it excludes vertical rays, for which has no value. Positive scaling is what turns the round trip into "same ray": is a positive multiple of , and the cylinder coordinates cannot tell positive multiples apart. A negative multiple would give , a different pair, because it is the opposite ray.
The angle is recovered only modulo , which is why the theorem compares angles modulo and a branch is fixed separately. The theorem also says nothing about the camera. Positive camera-frame depth and a valid image location are extra conditions for sampling, and they can fail for a ray whose cylinder coordinates are perfectly finite.
A numerical round trip
Worked example 1.31.4
(A 45-degree ray)
Let , , and input pixel . Then , , and . If , pixels per radian and pixels per height unit, its continuous output location is , approximately . It need not land at a pixel center.
For the inverse, . Its positive depth is , and division by it returns . This verifies the geometry before any discrete interpolation. For an arbitrary output pixel, the reprojected location and interpolation footprint must still be checked against the image bounds.
Worked example 1.31.5
(Accepting and rejecting output pixels)
Let , , , with a input image. Take and . Four output locations behave differently.
- gives , , so . Then and , inside the image.
- gives and , outside the image: the ray is in front of the camera but beyond its field of view.
- gives , so and . The division is undefined; the ray is parallel to the image plane.
- gives , so . The ray points behind the camera.
Only the first location is sampled from this input. The other three stay unsupported unless another image sees them.
Worked example 1.31.6
(Two rays on either side of the branch cut)
Take and with . Since , we get and . The two rays are only about radians apart, about pixels, yet their raw output columns differ by about pixels. A routine that takes the extent of an image as the range of raw angles would claim pixels of coverage. Adding to gives , and then the extent is , as it should be. The same unwrapping tells a blending routine that the columns at are neighbors.
Seam, vertical directions and parallax
The rays at and coincide. A full-turn panorama must treat its horizontal ends as neighbors when filtering or blending; otherwise the choice of branch can create an artificial seam. The half-open interval records that direction only once. An image may cross the branch cut, so its angular coordinates must be unwrapped consistently before finding its extent. This coordinate seam is distinct from a chosen compositing seam between photographs.
At the direction is vertical and is undefined. Approaching a vertical direction gives unbounded , so no finite cylinder canvas includes either vertical pole. Cropping the cylinder selects a finite vertical field of view; it does not fill the poles. A spherical chart uses latitude in place of , compressing the poles to finite limiting latitudes. Because points down, positive and positive point below the horizontal. Longitude remains undefined at the poles and requires special handling there.
Counterexample mode
Changing coordinates cannot remove parallax
Take two identical cameras with , normalized focal length one and camera 2 translated right by one unit. A point projects to and . Take two isolated points and . Their camera-1 directions are and , whose angle has cosine . Camera 2 sees directions and , whose angle has cosine . A rotation preserves angles, so no single rotation maps both first-camera rays to their second-camera counterparts. The points occupy different source rays and need not occlude each other. Their horizontal shifts are respectively and ; unequal shifts alone would not prove the result because a rotation also causes position-dependent shifts. A cylinder can give a convenient wide output canvas, but it cannot make a single rotation align this translated, multi-depth scene. A seam or blend can conceal some disagreement, not recover the missing depth geometry.
Common mistakes
Common mistake
(Applying where is needed)
The matrix carries camera directions into the panorama frame. Inverse mapping starts from a panorama ray, so it must return to the camera frame with . Using again rotates twice in the same sense, and only when is the identity or a half-turn, so the sample comes from the wrong place.
Common mistake
(Dividing by without checking its sign)
With and the representative ray is . The formula returns the principal point , as though the camera looked straight at that direction, but puts the ray behind the camera. Test before dividing, and treat as unsupported.
Practice with solutions
Quick check
(Let map camera rays to panorama rays. Which matrix maps back, and why?)
Solution · Answer
, because a proper rotation satisfies . Applying twice generally rotates farther rather than undoing the first rotation.
Quick check
(With , consider . Can the camera sample it?)
Solution · Answer
The representative is , up to the usual floating-point representation of . Its depth is negative, so the camera cannot sample that direction even though the cylinder coordinate is finite. The excluded endpoint represents the same ray.
Quick check
(For , find the corresponding spherical latitude and say whether the ray is vertical.)
Solution · Answer
Since and for , , or below horizontal. It is not vertical: the representative has horizontal radius one. Only the limiting values approach the poles.
Quick check
(Why can a forward scatter of integer input pixels leave gaps, and what does inverse mapping change?)
Solution · Answer
Several scattered pixels can round to the same output pixel while others receive none. Inverse mapping visits every chosen output pixel and asks each visible input for a continuous sample. It does not create data where no input camera sees the ray, and interpolation still requires valid neighbors.