Motivation
A wavelet transform reorganizes an image into averages and details; it does not remove noise by itself. Thresholding detail coefficients can suppress fluctuations, but it also removes or shrinks genuine image features. How can we compare these two effects when the clean image is unknown? A bound on the largest noise coefficient addresses a different question from the expected squared error of the reconstructed image. We now derive an estimate of that error which uses the observed coefficients and a known noise variance.
The estimate is unbiased for each threshold fixed before observing the data. This statement is about an average over repeated noisy observations, not a certificate for one image. If we try many thresholds and choose the smallest displayed estimate, we have used the data twice; the selected estimate need not retain the same unbiasedness. Understanding that distinction is as important as evaluating the formula.
Definitions
Definition 4.2.1
(Soft-threshold estimator and its risk)
Let be an real orthogonal transform, for example the finite Haar transform, and suppose with deterministic unknown and , where is known. In transform coordinates , with and independent standard normal entries in . Choose a fixed set of detail coordinates in advance. For a fixed threshold , estimate them by
Unpenalized coordinates, such as the coarsest average, are left equal to their observations. Orthogonality makes squared image error equal the sum of squared coefficient errors. The expected squared error in the selected detail coordinates is . Each untouched coordinate contributes to total expected error. We use the sum, not the per-coordinate average; dividing both risk and its estimate by gives mean squared error per image coordinate.
Theorem/proposition
The estimate rests on one identity for Gaussian variables.
Lemma 4.2.2
(Gaussian integration by parts)
Let and let be a continuous function made of finitely many continuously differentiable pieces with bounded derivative. Then
where may be given arbitrary values at the finitely many joining points.
Applied in each detail coordinate, the lemma turns the unknown risk into an observable quantity.
Theorem 4.2.3
(Stein estimate for a fixed soft threshold)
For every deterministic , the quantity
has expectation . Equalities at the threshold have probability zero under the stated model. Add for each untouched coordinate to estimate total image risk. A realized estimate may be negative, although actual squared error and its expectation are nonnegative.
Proof sketch or proof idea
Proof
(Integrating against the Gaussian density)
The Gaussian density satisfies . Integrate on each smooth interval. Boundary terms at infinity vanish, and the terms at the finitely many joining points cancel because is continuous. The derivative can be assigned arbitrary values at the joining points, which have zero probability.
The theorem follows by applying the lemma to the shrinkage , which is continuous with kinks at .
Proof
(Expanding the error around the observation)
Set . Expanding , taking expectations and applying the identity gives
Here and for , zero for . Summing yields the formula. When , almost everywhere and the expression reduces almost surely to , as it should for the identity estimator. Orthogonality transfers this coefficient risk to image space. No independence of the coordinates is needed to sum these marginal expectations, but the stated white Gaussian model supplies the required marginals and independence.
Proof X-ray
How the unknown signal disappears from the estimate
The proof has one non-obvious move. The error involves the unknown , and the expansion around the observation isolates it in the single cross term . The lemma converts that term into , which depends on alone. Everything else is observable or known: has mean , and is computed from the data. Using the observable residual in place of the error would rate the identity estimator as perfect, although it keeps all the noise.
The hypotheses enter at identifiable places. Gaussianity is used only through the density identity . Continuity of makes the boundary terms at the kinks cancel; hard thresholding, which jumps at , would leave extra terms there. The threshold must be fixed in advance because the lemma is applied to one fixed function ; a threshold computed from makes depend on the data in a way the lemma does not cover. Independence of the coordinates is never used, since the risk is a sum of marginal expectations.
Worked examples
Worked example 4.2.4
(Two observed details)
Take , , , and a threshold fixed in advance. The estimated details are and
The actual squared error is , which cannot be computed from alone. If a third, coarse coefficient is untouched, add one for its expected error: the total estimate is . The negative contribution from the thresholded coefficient is allowed in an unbiased estimate.
Worked example 4.2.5
(A negative estimate is possible)
With one detail, , fixed and observed , the estimate is . The estimate is negative whenever , an event of positive probability. This is not a negative realized loss. Replacing negative estimates by zero may be useful for display, but it changes the random quantity and generally destroys exact unbiasedness.
Worked example 4.2.6
(A threshold above every observation)
Return to , and , but fix in advance. Both observations satisfy , so both estimated details are and
This edge case is the zero estimator. Its loss is , and indeed , which is unbiased for that loss because . Any gives the same estimate. Comparing with the value found for after seeing the data is exactly the selection examined next.
Common mistakes
Counterexample mode
Selecting the smallest estimate can introduce optimism
Consider one coordinate with and . Compare the identity estimator, with SURE equal to , against the zero estimator, with SURE . Both estimates are unbiased for their respective fixed estimators. Select the estimator with the smaller SURE: choose zero when , and identity otherwise. The selected loss is zero in the first region and in the second. Direct Gaussian integration gives
where is the standard normal density. Thus fixed-estimator unbiasedness does not survive selection automatically. The zero estimator is the limiting soft threshold ; this example compares two explicitly defined estimators and does not require computing an infinite threshold numerically.
To verify the last identity, use and symmetry. Selection may still give a useful practical rule, but its risk must be assessed as a data-dependent procedure. Searching more thresholds also creates ties and implementation choices; no fixed-threshold unbiasedness theorem supplies an unbiased estimate for the selected rule without further analysis.
Common mistake
(Applying the formula outside its model)
Known white Gaussian noise and an orthogonal transform are assumptions, not automatic properties of a photograph. Unknown variance must be estimated with its own assumptions and uncertainty. A redundant transform, colored noise, clipping, or signal-dependent noise changes the calculation; blindly inserting the same formula need not estimate image risk. Nor is the risk of one selected coefficient set a guarantee about visual quality or edge preservation.
Common mistake
(Leaving out the untouched coordinates)
SURE estimates the risk of the thresholded detail coordinates only. Each coordinate left equal to its observation still carries its full noise and contributes to the expected image error. In the first worked example the detail estimate is but the estimate for the three-coefficient image is . Comparing a detail-only estimate with a full-image error, or with an estimate that includes the coarse coefficients, mixes two different quantities.
Summary
Soft thresholding trades noise suppression for signal shrinkage. Gaussian integration by parts yields a computable estimate of its expected squared error for a fixed threshold and known variance. Orthogonality connects coefficient and image errors. Data-driven threshold selection is a different estimator whose apparent minimum SURE can be optimistic.
Exercises
Exercise 4.2.7
(Let , , and fixed . Compute the estimate and SURE.)
Foundation
Solution(2)
- 1
Apply the soft threshold
The coordinates and have magnitude at most and become ; the coordinate becomes . The estimate is .
- 2
Evaluate the three terms
and two coordinates satisfy , so .
Exercise 4.2.8
(For , find the estimator and its detail risk for arbitrary deterministic . What is its realized SURE almost surely?)
Foundation
Solution(2)
- 1
Identify the estimator
, so the estimator is itself and its error is , with risk for every .
- 2
Evaluate SURE
With every vanishes, and only when , an event of probability zero. Hence almost surely, matching the risk exactly.
Exercise 4.2.9
(Two fixed thresholds have unbiased estimates of risks . Prove . Does this inequality alone identify the risk of the selected estimator?)
CoreProof
Solution(3)
- 1
Compare pointwise
For every outcome, for and .
- 2
Take expectations
Taking expectations gives for both , hence .
- 3
Interpret the bound
∎The inequality compares the selected estimate with the smaller fixed risk. It does not compute the loss or risk of the data-selected estimator, which the counterexample shows can exceed the expected selected estimate.
Exercise 4.2.10
(Show that for orthogonal , . Explain how two untouched coefficients change the estimated total risk when .)
Foundation
Solution(2)
- 1
Use orthogonality
Since , we have , and multiplication by preserves the Euclidean norm.
- 2
Account for untouched coefficients
Each untouched coordinate has expected squared error , so the total risk estimate is the thresholded-coordinate SURE plus .