Alignment does not equal appearance
A panorama can be geometrically aligned yet show a bright vertical band where one exposure ends and another begins. Before choosing a seam or blending pixels, we can estimate a multiplicative correction for each image from its overlaps with other images. This note gives a finite, reproducible version of that idea. It does not claim that a single gain can remove vignetting, motion, local illumination changes, or a wrong geometric warp.
Assume one linear-light channel, already registered on the regions compared. For a scene point seen in image , write , where the unknown exposure factor is constant over that image and is the common scene intensity. Ignore saturated, black-level-clipped, moving, occluded, and unsupported samples. This ideal model assumes that camera response and black level have already been corrected; gamma-encoded pixel values generally do not obey it. If the camera response is unknown, estimating gains this way is a modeling approximation, not a physical calibration.
Definition 1.20.1
(Pairwise log exposure ratio)
We seek gains such that agrees across overlaps. Let . For an oriented overlap with valid matched samples and positive weights , define
The equalized log intensities satisfy in the ideal model. The sign is worth checking: if image is twice as bright as , then and the correction on must be twice that on . Reversing an edge negates . A weighted mean in log space corresponds to a weighted geometric mean of intensity ratios; it is not generally the logarithm of the ratio of arithmetic means.
Common mistake
(Using the wrong sign for an oriented edge)
Since and , the darker image receives the larger gain. Suppose image 1 is twice as bright as image 2, so on the overlap, and fix . The correct choice gives . Writing the constraint as instead gives and : the corrected intensities now differ by a factor of four instead of two.
Worked example 1.20.2
(A weighted log ratio of overlap samples)
Let the overlap contain three matched samples. Sample A has with weight , sample B has with weight , and sample C has because the second camera clipped to black. Sample C is excluded: is undefined, and the sample carries no usable information about the ratio. The remaining log differences are and , so
This is the logarithm of the weighted geometric mean of the intensity ratios. The ratio of weighted arithmetic means is , whose logarithm is about . The two estimates differ because arithmetic means are dominated by the brighter sample B.
Concept lensStructural
Why only relative brightness is observable
Multiplying every gain by the same positive constant preserves all corrected ratios. The data alone cannot select an absolute output brightness. Fixing chooses image 1 as a reference; it does not discover a physically privileged exposure. A later global tone or exposure choice changes all gains together.
The graph least-squares problem
Definition 1.20.3
(Overlap graph, incidence matrix and objective)
Construct a graph whose vertices are images and whose edges are overlaps with at least one valid match. Give each edge a finite weight , expressing its chosen reliability, and form the incidence matrix : row has in column , in column , and zero elsewhere. The objective is
where is diagonal.
The normal equations are together with the anchor. In computation remove column 1 of and solve the resulting full-column-rank least-squares problem by QR; forming the normal equations is convenient for proof but squares the condition number. Edge weights must be specified: a large number of correlated pixels does not automatically provide that many independent observations.
Theorem 1.20.4
(Existence and uniqueness after anchoring)
For a finite connected overlap graph with positive finite edge weights, the anchored objective has exactly one minimizer. Without an anchor, its minimizers differ by a common additive constant. If the graph is disconnected, one independent constant remains on each connected component.
Proof
(Positive definiteness of the anchored normal matrix)
Let be with its first column removed. If , extend by to a vector . Then along every edge, so connectivity forces every . Thus has full column rank, and for every nonzero
Hence is positive definite and invertible, which gives a unique stationary point of the anchored objective. Because the objective is a quadratic with Hessian proportional to and vanishing gradient at , , so every other point has a strictly larger value. For the unanchored problem, exactly when is constant on each connected component, so adding those constants leaves the objective unchanged. An anchor removes the constant of its own component only.
Proof X-ray
Where each hypothesis enters the uniqueness argument
Connectivity enters once: it turns into , because a vanishing difference along every edge carries the anchor value to every vertex. Positive weights enter once: they make strictly positive whenever some . Dropping connectivity breaks the first step concretely. If images 1, 2 overlap and images 3, 4 overlap with no other edge, the vector with and satisfies , so has a nonzero kernel and is singular. The argument never uses the sizes of the weights: uniqueness does not depend on how they are chosen, although the minimizer itself does.
Proposition 1.20.5
(Exact fit and cycle sums)
Let the overlap graph be finite, with oriented edges and measurements . There is a vector with for every edge if and only if, around every cycle, the sum of over edges traversed along their orientation minus the sum over edges traversed against it is zero.
Proof
(Telescoping and path independence)
Necessity: along a cycle the differences telescope to zero, and each equals or according to whether the edge is traversed along or against its orientation. Sufficiency: in each connected component choose a base vertex, give it the value zero, and define the value at any other vertex as the signed sum of measurements along a walk from the base to it. Two such walks ending at the same vertex form, with one reversed, a closed walk. A closed walk decomposes into cycles and pairs of the same edge traversed forth and back, which cancel, so its signed sum is zero and the value does not depend on the walk. Extending a walk to vertex by the edge gives a walk to , whence for every edge.
A tree has no cycle test, and any edge values fit exactly even if measurements are wrong.
Real measurements need not obey the ideal forward model: noise, motion and model error can make cycle measurements inconsistent. The following example therefore relaxes exact model agreement.
A three-image loop
Worked example 1.20.6
(Distributing an inconsistent cycle error)
Suppose three equally weighted measurements are , , and , in natural-log units. The first two suggest a total difference of 2, while the third says 3. With , minimize
Setting derivatives to zero gives and . Thus and . The three predicted-minus-observed residuals are , , and , and the objective is . The gains are . They are relative gains, not three estimates of radiance. Reversing the last edge changes both its row and measurement sign but leaves the objective and solution unchanged.
Worked example 1.20.7
(A tree fits exactly, even when a measurement is wrong)
Take three images joined by the edges and only. With and measurements , , the choice makes both residuals zero. If the second overlap contained a misregistered object and produced , the choice would again give objective zero. A zero residual here only says that a tree has no cycle in which an inconsistency could appear; it does not say that the measurements are right. Adding the edge with creates a loop whose signed sum is , and the error becomes visible as a nonzero objective.
Counterexample mode
A gain cannot repair a local mismatch
For two corresponding pixels let image 1 have values and image 2 have . Equalizing the first requires ; equalizing the second requires . No constant gains satisfy both. A least-squares result is a compromise, and a low average residual can conceal a serious localized error. Inspect residual maps and validity masks, not merely the fitted gains. A moved object or a misregistration can create the same symptom without any exposure change.
From a model to a usable mosaic
Choose a channel and working color space first; applying unrelated gains to color channels can alter color balance. Warp each image to a common domain with an explicit interpolation and validity mask. Reject clipped samples and choose conservative intensity thresholds, since logarithms are undefined at zero and amplify relative uncertainty near black. Estimate pairwise log ratios only from the remaining same-scene samples. A robust statistic or robust optimization can reduce outlier influence, but its objective and tuning must be stated; the quadratic uniqueness theorem above does not automatically transfer to every robust loss.
Common mistake
(Keeping saturated samples in the overlap)
Let and , so and and every unclipped ratio is . If the sensor saturates at and the scene intensities are , image 1 reads , and , the last being the clipped value of . The three ratios are , and , and their equally weighted log mean is , a ratio of about instead of . A clipped sample pulls the estimate toward zero, so exclude such samples before forming .
Build the overlap graph and inspect its components. Anchor one image per component; their relative brightness remains undetermined without extra information. Solve, exponentiate to obtain gains, and inspect both edge residuals and spatial difference images in the overlap. An edge can average opposing local errors to zero. Large cycle residuals are a diagnostic of inconsistency, not proof that a particular image is faulty. Finally, choose seams and blending, with exposure compensation applied in a suitable linear representation before display encoding. Gains may cause values outside the display range; clipping or tone mapping is a separate, potentially information-losing operation.
Practice with solutions
Quick check
(Two images have and at every valid match. Set and find .)
Solution · Answer
, so and ; both corrected intensities equal .
Quick check
(Images 1 and 2 overlap; images 3 and 4 overlap; there are no other edges. How many anchors are needed to make the solution unique?)
Solution · Answer
Two, one per component. Fixing only leaves the common offset of free.
Quick check
(Edges , and have measurements , and . Is an exact anchored fit possible?)
Solution · Answer
Yes: . With , choose and , giving zero objective.
Quick check
(Two equally weighted matched pairs have log differences and . What is , and does it imply that the pixels agree?)
Solution · Answer
The average is zero, but the two differences are nonzero and opposite. A single gain of one matches neither pair, so inspect the individual residuals.
Quick check
(An anchored solution has . What solution uses as the anchor instead?)
Solution · Answer
Subtract from every entry: . Every difference and residual is unchanged, while all gains are multiplied by .