Bookkeeping partials on a small lattice of variables

Backprop at scale is still the chain rule; the hard part is bookkeeping. Assign a symbol to every intermediate value on a small graph. Label each edge with the local partial derivative of the child w.r.t. the parent. To find $\partial C/\partial x$, multiply factors along every path from $x$ to $C$ and sum when paths reunite at fan-out nodes .

If $z = f(x,y)$ and $C = C(z)$, then $\partial C/\partial x = (dC/dz)(\partial z/\partial x)$ when the univariate chain applies along that edge. When $x$ feeds two consumers, $\partial C/\partial x$ receives contributions from both branches .

Matrix-valued nodes introduce layout conventions: is the gradient a row vector or column vector? Does the Jacobian use numerator or denominator notation? Textbooks warn because a silent transpose in code changes every downstream shape .

Disciplined subscripts prevent off-by-one index errors that compile yet train wrong networks .

When you write $\partial C/\partial w_{jk}^{(\ell)}$, specify which layer $\ell$ and which edge connects neuron $j$ to neuron $k$. The lattice diagrams are small precisely so you can read every label without scrolling .

When translating to code, match tensor shapes to these symbols before trusting autograd: a silent transpose in a manual implementation is equivalent to swapping indices in the lattice .