A random input can have the same mean as a constant input while producing a different average output. If energy consumption is quadratic in load, for example, alternating between two loads generally costs more on average than keeping the load fixed at its mean. Jensen's inequality makes this observation precise. It also explains why replacing a random quantity by its mean inside a nonlinear function can be misleading.
Convexity and supporting lines
Definition 26.1.1
(Convex, strictly convex and concave functions)
A function on an interval is convex if, for and ,
It is strictly convex if the inequality is strict whenever and . The function is concave (strictly concave) if is convex (strictly convex).
Geometrically, the graph lies on or below each chord. For a twice differentiable function, on the interval is a useful sufficient condition for convexity; is sufficient for strict convexity. A zero second derivative at isolated points does not preclude strict convexity: is strictly convex on .
The proof of Jensen's inequality rests on one geometric fact: at an interior point, a convex graph lies above some straight line through that point.
Lemma 26.1.2
(Supporting line at an interior point)
Let be convex on an interval and let be an interior point of . There is a real number such that
If is differentiable at , the choice works, so every tangent line at an interior point is a global lower bound.
Proof
(Compare chord slopes on the two sides of the point)
Take in and write with , so that . Convexity gives . Subtracting , and separately subtracting this inequality from , yields
Hence every left chord slope at is at most every right chord slope. Because is interior, both families are nonempty, so and are finite with . Choose any . For , the bound rearranges to ; for , the bound is the same inequality; at there is nothing to prove.
If exists, it is the limit of the left slopes, each at most , and of the right slopes, each at least . Thus , and is admissible.
Differentiability is convenient, not essential: has a corner at , and every gives a supporting line there. Interiority, by contrast, is needed. The function is convex on , but its chord slopes from the endpoint tend to , so no line of finite slope supports it at .
Jensen's inequality
Theorem 26.1.3
(Jensen's inequality)
Let be an integrable real random variable taking values in an interval , and suppose is convex and is integrable. Then and
For concave the inequality reverses. If is strictly convex, equality holds exactly when is almost surely constant.
Integrability of means ; the separate assumption on matters. For instance, a variable may have a finite mean but an infinite second moment, so its variance cannot be treated as a finite number. The theorem above deliberately uses finite expectations rather than requiring a convention for infinite values.
Proof
(Take expectations of a supporting line at the mean)
Write . Since almost surely, lies between the endpoints of . Suppose equals an endpoint, say the left endpoint . Then almost surely and , so almost surely; in particular , and both sides of the inequality equal . The right endpoint is handled in the same way.
Otherwise is an interior point of , and the supporting-line lemma gives a slope with
Both sides are integrable. Taking expectations and using gives . For concave , apply this to .
For the equality case, let . Equality in Jensen means , hence almost surely. If is strictly convex, the supporting line touches the graph only at : if it also touched at , then at every point strictly between and the graph would lie on or below the chord joining the two touching points, which is a segment of the supporting line, and on or above that line, contradicting strict convexity. Thus forces almost surely. The converse is immediate.
Proof X-ray
Where each hypothesis enters
The decisive move is to choose the supporting line at the mean and nowhere else. An affine function commutes with expectation, so the line contributes exactly ; at any other base point the linear term would not vanish and the bound would not close.
Integrability of and of is used once, to take expectations of both sides of a pointwise inequality and to subtract finite numbers. Interiority is the reason for the case split: the lemma can fail at an endpoint, as shows, but at an endpoint the mean pins down completely. Strict convexity plays no role in the inequality. It enters only in the equality case, where it shrinks the set on which the line touches the graph to the single point . For a merely convex function, equality can hold for nonconstant when is affine on an interval containing its values.
Worked example 26.1.4
(Quadratic cost and the Jensen gap)
Let be or with probability each. Then , but . More generally, for any square-integrable ,
Here the Jensen gap is exactly the variance. A larger variance has this interpretation for a quadratic cost, but for a general convex function the gap is not determined by variance alone.
Worked example 26.1.5
(Equality for a nonconstant variable when the function has a flat piece)
Let , which is convex but not strictly convex. If is or with probability each, then while : the inequality is strict.
If instead is or with probability each, then and . Equality holds although is not constant, because both values lie in , where is affine. The gap is positive exactly when puts positive probability on both sides of the kink at .
Concept lensGeometric
The center of mass of the graph lies above the graph
Plot the random point . Its mean is , and Jensen says exactly that this mean lies in the epigraph , a set that is convex precisely because is convex. If takes values with probabilities , the mean is the center of mass of points on the graph, and convexity, extended by induction to finitely many points, places it in the epigraph. The theorem extends this picture to every integrable .
Useful consequences
Corollary 26.1.6
(Moment and logarithmic bounds)
- If is square-integrable, then , with equality exactly when is almost surely constant.
- If is integrable, then .
- If almost surely, and is integrable, then
with equality exactly when is almost surely constant.
Proof
(Apply Jensen to three specific functions)
Item 1 is Jensen for the strictly convex function on . Item 2 is Jensen for the convex function . Item 3 is Jensen for the strictly concave function on ; exponentiating, which is increasing, gives the second form.
The absolute-value function is convex but not strictly convex, which is why item 2 has no simple equality condition: if is nonnegative almost surely, equality holds even when is random.
For a finite equally likely sample , the last inequality becomes the arithmetic–geometric mean inequality. Equality requires all to be equal. Positivity and integrability are not decorative: is not a finite real number, and a positive random variable can have a nonintegrable logarithm.
Worked example 26.1.7
(Why average log growth differs from log average growth)
A multiplicative factor is with probability and with probability . Its arithmetic mean is , while its mean log factor is zero. The geometric mean is therefore , strictly below . If independent factors with this law are multiplied over many periods, the product's expected value is , whereas its log divided by converges almost surely to zero by the strong law. These are different summaries; a large expected product can coexist with zero typical long-run log growth.
Counterexample mode
Nonlinearity cannot be moved through an expectation
The tempting claim is that for any reasonable function . For the preceding factor and , , but . This does not contradict Jensen: is strictly convex on , so the correct inequality is when both expectations are finite, and equality would require to be almost surely constant. The interchange is always valid for affine ; for other functions it can fail, as it does here.
Common mistakes
Common mistake
(Reversing the direction for a concave function)
Jensen's inequality for a concave function bounds the expectation from above. If is or with probability each, then while . The correct statement is ; writing because “Jensen goes that way” is wrong.
Common mistake
(Using a function that is convex only on separate pieces)
The function is convex on and concave on , but it is not convex on any interval containing both signs. If is or with probability each, then while , so the “Jensen inequality” fails. The hypothesis is convexity on one interval containing all values of .
Practice
Quick check
(A variable takes values and with equal probability. Find , and the Jensen gap.)
Compare the gap with the variance.
Solution · Answer
and , so . The gap is the variance.
Quick check
(Suppose almost surely, , and . What lower bound can you prove for ? When can equality hold?)
Decide whether is convex or concave on .
Solution · Answer
Since is strictly convex on , Jensen yields . Equality holds precisely when almost surely.
Quick check
(Let equal or with equal probabilities. Verify Jensen for and explain whether equality must hold for every nonconstant .)
Try a nonconstant variable that never changes sign.
Solution · Answer
, so equality does not hold here. Equality is not ruled out for every nonconstant variable either: a nonconstant nonnegative variable gives equality, since almost surely. Strict convexity is the hypothesis that would exclude this, and lacks it.
Exercise 26.1.8
(Let sum to one and let for . Prove , and state the equality condition for indices with positive weight.)
CoreProof
Hint(1)
Turn the weights into a probability distribution and take logarithms.
Solution(3)
- 1
Model the weights as a distribution
Define with probability . Then , and and take finitely many values, so both are integrable.
- 2
Apply Jensen to the logarithm
Key step
Because: is strictly concave on .
- 3
Exponentiate and read off equality
∎Exponentiating gives . Equality holds exactly when is almost surely constant, that is, when all with are equal; zero-weight values have no effect.