Motivation
A multilayer network is a composition of simple operations, but a parameter may influence the loss along many routes. A hidden value can feed several later units, and one weight can be reused at several positions. Differentiating only the most visible route misses real contributions. Backpropagation is the systematic accumulation of every such contribution, organized so that shared work is performed once.
It helps to separate three objects: the function computed by the graph, its derivative at the current parameters, and the optimization algorithm using that derivative. A correct gradient does not prove that a chosen step decreases the loss, and decreasing training loss does not prove good predictions on new cases. Here we derive the gradient exactly and follow finite updates whose forward values can be checked.
Definitions
Definition 30.1.1
(Computational graph)
A finite computational graph is a directed acyclic graph with a topological ordering. Source nodes are inputs or parameters. Each remaining scalar node is a differentiable function of specified earlier nodes; vector operations can be read componentwise. A scalar loss L is the final node. Differentiability is required at the values used. A ReLU at zero has no ordinary derivative, so a software convention there defines an update rule rather than proving classical differentiability.
Definition 30.1.2
(Forward pass and adjoints)
A forward pass evaluates nodes in topological order and stores the values needed for local derivatives. The adjoint is the derivative of the final loss with respect to an independent perturbation of the intermediate value , holding earlier computed quantities fixed at that boundary. Reverse accumulation starts with and adds downstream contributions:
The sum is over argument occurrences. If a node uses the same parent twice, both partial contributions count. A parameter represented by one source node receives contributions from every child using it. Creating two independent parameter copies changes the model unless their gradients are subsequently summed for the shared parameter.
Definition 30.1.3
(Softmax and cross-entropy)
For logits , softmax probabilities are . A fixed target distribution q satisfies , . A class label corresponds to one-hot q. Cross-entropy is
All finite logits give strictly positive probabilities. The loss uses natural logarithms, so changing the logarithm base rescales derivatives.
Definition 30.1.4
(Simultaneous gradient step)
Gradient descent with step replaces every parameter by its old value minus times its gradient evaluated at the same old parameter vector. Sequentially updating a weight and then using its new value elsewhere is a different algorithm.
Theorem/Proposition
Theorem 30.1.5
(Reverse accumulation is the complete chain rule)
On a finite acyclic graph of differentiable operations, reverse accumulation computes the exact derivative of L with respect to every source parameter. Each local edge contribution is processed once after its child adjoint is available. Its arithmetic work is proportional to the local derivative work over the graph, rather than the possibly exponential number of source-to-loss paths.
Theorem 30.1.6
(Softmax cross-entropy derivative and curvature)
For finite logits and a fixed target distribution q,
The Hessian is positive semidefinite, so the loss is convex as a function of logits. Adding one common scalar to every logit leaves the probabilities and loss unchanged. Convexity in logits does not imply convexity in all parameters of a multilayer network.
Proof sketch or proof idea
Proof
(Substituting differentials backwards)
For a perturbation, the local differential relation is . Start at and substitute these relations backwards in reverse topological order. When a node is eliminated from the expression, every term involving its differential is already present, since all its children occur later in the forward ordering. Collecting their coefficients gives exactly the adjoint sum. At the end only source differentials remain, with coefficients equal to the derivatives by the total differential definition. This proves the algorithm, including the addition of contributions at branches and shared parameters.
Equivalently, repeated chain-rule expansion gives one product of local derivatives for every directed source-to-loss path. The graph is finite and acyclic, so this is a finite sum. Reverse accumulation factors common suffixes of those products. It therefore changes the organization of the computation, not the derivative being computed. A node with no path to the loss has zero adjoint.
Proof X-ray
Why a branch requires addition
If a value v feeds two children, perturbing v changes both children simultaneously. Their downstream effects add in the total differential. Multiplying those two effects would describe neither the chain rule along one route nor the sum over routes. The required multiplication occurs along a route; addition combines distinct routes.
Proof
(Differentiating log-sum-exp)
Differentiate log-sum-exp: its i-th derivative is , giving . Differentiating with respect to gives , yielding the Hessian. For any vector a, its quadratic form is , the variance of a random value taking with probabilities . It is nonnegative. The all-ones vector has zero quadratic form, matching the common-shift invariance. The logit gradient also sums to zero because p and q each sum to one.
Convexity does not guarantee that a finite minimizer exists. For a one-hot target, cross-entropy is minus the logarithm of the correct-class probability. Finite logits give that probability strictly below one when there are at least two classes, so the loss is positive. Sending the correct logit increasingly far above the others makes the loss approach zero without attaining it. A vanishingly small gradient can therefore accompany logit gaps growing without bound rather than convergence to a finite logit optimum. Regularization or other restrictions change that optimization problem and must be included in its objective.
For a target q with every entry positive, minimizing over positive probability vectors gives p equal to q. Indeed cross-entropy equals the entropy of q plus its Kullback–Leibler divergence from p; the latter is nonnegative by the logarithm inequality and vanishes at equality. Even then, logits are not unique because a common shift changes nothing. Distinguish uniqueness of a probability vector, uniqueness of logits, and uniqueness of underlying network parameters. This also explains why two optimization runs can return different parameter vectors with identical predictions, without either numerical result being wrong.
In numerical evaluation, subtract before exponentiation and compute . Every exponential is at most one. This algebra preserves the function and avoids exponentiating large positive logits. It does not authorize replacing the sum by its largest term; that would change both loss and derivative.
For a finite batch-average objective, differentiation commutes with its finite sum, so the gradient is the average of per-example gradients. Every example uses the same current parameter vector, while its hidden activations may differ. For a weighted average the weights appear in the gradient; confusing a sum with a mean changes the effective step size. A stopping rule should also distinguish a small gradient from a good objective value: nonconvex parameterizations may have stationary points that are not global minima.
When the computation "perturbs an intermediate value independently," the purpose is to measure its downstream influence, not to cut its relation to earlier parameters permanently. The downstream influence is first collected into an adjoint, then passed upstream through local derivatives, and only at the end does one obtain the total derivative with respect to a parameter. Treating an intermediate value as an independent source and never passing its adjoint back undercounts the dependencies of the original model. This boundary view also explains why the same node must both store its forward value and accumulate a reverse coefficient.
The values stored during the forward pass also fix the point at which differentiation happens. For example, the local derivative of tanh must use the old activation, and multiplication by an output weight must use the old weight. If parameters are updated while the backward pass is still running, later steps may mix local derivatives taken at two different points, and they no longer combine into the total differential of one function at one point. The requirement of a simultaneous update is therefore not a formatting convention; it is part of the mathematical content of the algorithm being claimed.
Inputs and parameters are both sources of the graph, but they play different roles in training. Input derivatives can be used to analyze sensitivity; parameter derivatives are used to update the model. Computing an input gradient does not mean that training changes the observed data. Conversely, if a preprocessing step contains trainable parameters, that step must be placed in the graph as well; otherwise the reported gradient applies only to a different model in which the preprocessing is held fixed.
The value of the loss and the classification decision must also be kept apart. Cross-entropy can keep improving while the most probable class stays the same, and a very small change of logits near a tie can change the predicted class. The derivative describes the continuous loss, not an ordinary derivative of the discrete accuracy. Record both separately during evaluation, and do not claim that every step improves classification accuracy just because the gradient direction is correct.
Concept lensAlgebraic
The adjoint as a row vector times Jacobians
For a layered chain with Jacobians , the loss gradient is the row vector . Reverse accumulation multiplies from the left, so every step is a vector–matrix product. With all widths equal to , this costs about multiplications per layer, whereas first forming the Jacobian product from the right costs about per layer. The adjoint is precisely this running row vector; the two descriptions give the same gradient and differ only in the order of multiplication.
Worked examples
Worked example 30.1.7
(A shared intermediate contributes through two branches)
Let , , . The forward values are . The two branches give , hence . With , update w to . The new v is , and the new loss is . Differentiating the expanded expression also gives 14 at w equal to one, independently checking the shared-graph calculation.
Worked example 30.1.8
(A complete two-layer classification step)
Take , hidden , logits and target class one. Start with . Then and loss . Gradients in order are because . A simultaneous step gives . The new first logit is , and the new loss is exactly . Every parameter update uses the old output weight 2 and old hidden value zero.
Worked example 30.1.9
(Large logits require stable algebra)
For logits and target class one, subtracting 1000 gives probabilities without computing . The loss is and the gradient is . If logits themselves are parameters, a step produces . Their difference is , so the new loss is exactly . The common offset 1000 carries no class information.
Common mistakes
Counterexample mode
Convex logits can have nonconvex parameter loss
A tempting claim is that every stationary point of the parameter loss is a global minimum, because cross-entropy is convex in the logits. With one positive binary example, use first logit and second logit zero. The loss is . At both parameter derivatives vanish, yet taking strictly lowers the loss. Thus a stationary parameter point need not minimize a loss that is convex in its logits. The composition with a nonlinear parameter map changes the optimization geometry.
Common mistake
(A correct gradient is not a useful training signal)
Cross-entropy's derivative avoids explicitly multiplying a tiny softmax derivative by a huge reciprocal probability, but hidden layers can still have small derivatives. A saturated tanh can attenuate the signal before it reaches early weights. Gradient cancellation at shared parameters is also possible even when each local derivative is nonzero. A correct graph derivative does not promise a large or useful training signal.
Common mistake
(Differentiating something other than the computation actually used)
Targets must be treated as fixed in this derivation. If a target is itself produced by a parameter-dependent procedure, differentiating through it defines a different objective unless it is intentionally held constant. Similarly, clipping probabilities before taking logarithms changes the implemented function. Derive the gradient of the computation actually used and state any nondifferentiable conventions explicitly.
Summary
Backpropagation is the chain rule organized over a finite dependency graph, with sums at branches and shared parameters. Softmax cross-entropy gives the exact logit gradient and a positive-semidefinite Hessian. Stable algebra preserves these identities, while simultaneous updates preserve the specified gradient-descent rule. Parameterization, initialization and generalization remain separate questions from derivative correctness.
Exercises
Quick check
(Let , , and with . Compute the gradient with respect to the shared w by reverse accumulation and by direct expansion.)
Apply the adjoint rule at , then at , and add the two routes into .
Solution · Answer
Forward gives . Reverse gives , . The direct occurrence of w in b contributes 6, and its route through a contributes , giving 18. Directly , so and its derivative at one is 18. Both routes are necessary.
Quick check
(Three logits are all zero and the target is class two. Find probabilities, loss, logit gradient and the updated logits for step . Verify their sum is unchanged.)
With equal logits every probability is ; subtract the one-hot target.
Solution · Answer
Probabilities are , loss , and gradient . Updated logits are , whose sum is zero as before. A common shift would give the same probabilities, so the zero-sum gradient is consistent with the invariance.
Quick check
(In the two-layer example, instead initialize . Find every gradient. Explain why repeated updates on this same input cannot make the hidden unit active through these gradients.)
With the hidden value is ; check which factors vanish.
Solution · Answer
The hidden value is zero, and the gradients for are . Only c changes. As long as w,b,v remain zero, h remains zero, the v gradient remains proportional to h, and the incoming gradients remain proportional to v. Induction shows this hidden part stays at zero for this training setup, although the output bias learns. This is a property of the specified initialization and data, not a theorem that every zero parameter prevents learning.
Quick check
(A two-example weighted loss is , with parameter gradients and . Compute and one step from with . Explain why averaging the two gradients equally is incorrect.)
Differentiate the weighted sum term by term.
Solution · Answer
The weighted gradient is . The new parameter vector is . Equal averaging would give , the derivative of a different objective. Finite-sum linearity preserves the authored weights, not a convenient replacement for them.