Backpropagation

Backpropagation computes gradients through a composed computation graph using the chain rule. It is reverse-mode Automatic differentiation applied to a scalar training loss.

For a composition:

$$y=f(g(x))$$

the chain rule is:

$$\frac{\partial y}{\partial x}=\frac{\partial f}{\partial g}\frac{\partial g}{\partial x}$$

For model training:

$${\nabla }_{\theta }\mathcal{L}=\frac{\partial \mathcal{L}}{\partial f_{\theta }(x)}\frac{\partial f_{\theta }(x)}{\partial \theta }$$

More generally, each node $h_i$ in the graph receives an adjoint:

$${\bar{h}}_i=\frac{\partial \mathcal{L}}{\partial h_i}$$

and sends it backward through local Jacobians:

$${\bar{h}}_{i-1}={\left(\frac{\partial h_i}{\partial h_{i-1}}\right)}^{\top }{\bar{h}}_i$$

This is usually implemented as vector-Jacobian products rather than explicit Jacobian matrices.

Reverse-Mode View

For an intermediate node:

$$h_i=f_i(h_{p_1},h_{p_2},\dots )$$

the backward pass accumulates gradient contributions from all downstream users:

$${\bar{h}}_i=\sum _{j\in \mathrm{children}(i)}{\left(\frac{\partial h_j}{\partial h_i}\right)}^{\top }{\bar{h}}_j$$

This accumulation is why reused tensors, shared parameters, and residual branches require careful gradient accounting.

For a parameter $\theta$ used in multiple places:

$${\nabla }_{\theta }\mathcal{L}=\sum _{u\in \mathrm{uses}(\theta )}{\left(\frac{\partial h_u}{\partial \theta }\right)}^{\top }{\bar{h}}_u$$

The optimizer updates the parameter after gradients from the chosen mini-batch and accumulation steps have been combined.

Loss Scaling

If the training loss is averaged over valid elements:

$$\mathcal{L}=\frac{1}{|\Omega |}\sum _{i\in \Omega }{\ell }_i$$

then masked or padded elements must be removed before the reduction. Otherwise the gradient scale depends on padding length:

$${\nabla }_{\theta }\mathcal{L}\propto \frac{1}{N_{\mathrm{tokens}}}\sum _i{\nabla }_{\theta }{\ell }_i$$

instead of the intended valid-token count.

What Can Go Wrong

• A tensor is detached or converted outside the differentiable graph.
• A mask is applied after loss reduction instead of before it.
• A custom operation has a backward rule that does not match the forward computation.
• Gradient accumulation unintentionally scales the effective loss.
• Shared parameters receive multiple gradient contributions that are not expected.
• In-place operations overwrite values needed for backward.
• Mixed-precision scaling hides overflow or underflow if skipped silently.

Checks

• Are invalid or padded elements masked before loss reduction?
• Are gradients zeroed before the next step unless accumulation is intentional?
• Are frozen parameters excluded from gradient updates?
• Are gradient norms monitored when training is unstable?
• Does Gradient checking pass for custom differentiable code?
• Does the reported loss reduction match the intended example/token/object unit?
• Are gradient accumulation, distributed averaging, and batch-size scaling documented?

Related

• Automatic differentiation
• Gradient checking
• Gradient descent
• Gradient accumulation
• Gradient clipping
• Loss function
• Training loop
• Jacobian and Hessian
• GPU memory