Automated Gradient Computation
Overview of Techniques
Reader level: Beginner Essentials of Chain Rule for Differentiation This section describes some of the basics of chain rule for differentiation that we will use to describe automatic differentiation.
$$ \dfrac{ df(a \cdot b) }{dx} = b\dfrac{da}{dx} + a\dfrac{db}{dx} $$
For a function ‘f’ that is dependent on variables ‘ni(x)‘, the partial derivative w.r.t an independent variable ‘x’ can be written as shown below
$$ \dfrac{\partial f}{ \partial x} = \sum \dfrac{\partial f}{\partial n_{i} } \cdot \dfrac{ \partial n_i }{\partial x} $$
[Read More]