Differentiation

← Differentiability · Contents · Directional Differentiation →

Rules of Differentiation

You may recall from single variable calculus that we often didn't need use the definition of differentiability to actually differentiate functions. Instead we used a handful of simple rules of differentiation, as it is easier and gets the job done provided the function isn't too weird. As a refresher, here they are.

There are other rules, but they're often just consequences of these. You'll be happy to know that, just like how the definition of differentiability is basically the same in both the single-variable and multivariable case, the rules of differentiation are also basically the same in both cases, and so are the proofs that they work. We will prove them in the order shown above, except for the constant rule, which is an exercise.


Theorem (Sum Rule): Let $u, v: \mathbf{R}^n \rightarrow \mathbf{R}$ be differentiable. Then $u+v$ is differentiable at $a$, and $\nabla(u+v)(a) = \nabla u(a) + \nabla v(a)$:

Proof: Since $u$ and $v$ are differentiable, we know there must exist functions $\chi$ and $\psi$ continuous at $0$ such that $\chi(0) = \nabla u(a)$, $\psi(0) = \nabla v(a)$, and:

$$\begin{align*} (u+v)(a+h) &= (u(a) + \chi(a)\cdot h) + (v(a) + \psi(a)\cdot h)\\ &= u(a) + v(a) + \chi(h)\cdot h + \psi(h)\cdot h\\ &= u(a) + v(a) + (\chi(h) + \psi(h))\cdot h \end{align*}$$

Since continuous functions are closed under addition, $u + v$ is differentiable at $a$, with $\phi(h) = \chi(h) + \psi(h)$ defining the necessary continuous function $\phi$. Thus, $\nabla (u + v)(a) = \phi(0) = \nabla u(a) + \nabla v(a)$. $\square$


Theorem (Product Rule): Let $u: \mathbf{R}^n \rightarrow \mathbf{R}$ and $v: \mathbf{R}^n \rightarrow \mathbf{R}$ be differentiable at $a \in \mathbf{R}^n$. Then $uv$ is differentiable at $a$, and: $$(\nabla uv)(a) = u(a)\nabla v(a) + v(a)\nabla u(a)$$

Proof: Since $u$ and $v$ are differentiable, we know there must exist functions $\chi$ and $\psi$ continuous at $0$ such that $\chi(0) = \nabla u(a)$, $\psi(0) = \nabla v(a)$, and:

$$\begin{align*} (uv)(a+h) - uv(a) &= u(a+h)v(a+h) - u(a)v(a)\\ &= (u(a) + \chi(h)\cdot h)(v(a) + \psi(h)\cdot h) - u(a)v(a)\\ &= u(a)\psi(h)\cdot h + v(a)\chi(h)\cdot h + (\psi(h)\cdot h)(\chi(h)\cdot h)\\ &= (u(a)\psi(h) + v(a)\chi(h) + (\psi(h)\cdot h)\chi(h))\cdot h\\ \end{align*}$$

Since continuous functions are closed under scalar multiplication, addition, and multiplication, we see that $uv$ is differentiable, with $\phi(h) = u(a)\psi(h) + v(a)\chi(h) + (\psi(h)\cdot h)\chi(h)$ defining the necessary continuous function $\phi$. Thus, $(\nabla uv)(a) = \phi(0) = u(a)\nabla v(a) + v(a)\nabla u(a)$. $\square$


Theorem (Chain Rule): Let $v: \mathbf{R} \rightarrow \mathbf{R}$ be differentiable at $u(a)$ and $u: \mathbf{R}^n \rightarrow \mathbf{R}$ be differentiable at $a$. Then $v \circ u$ is differentiable at $a$, and $\nabla(v \circ u)(a) = v'(u(a))\,\nabla u(a)$.

Proof: Since $u$ is differentiable at $a$, we know there must exist some vector-valued function $\chi$ continuous at $0$ such that $u(a + h) = u(a) + \chi(h)\cdot h$. Let $k_h = \chi(h)\cdot h$. Then, since $F$ is differentiable at $u(a)$, there must exist some scalar-valued function $\psi$ continuous at $0$ such that $\psi(0) = F'(u(a))$, and: $$\begin{align*} F(u(a + h)) - F(u(a)) &= F(u(a) + \chi(h)\cdot h) - F(u(a))\\ &= F(u(a) + k_h) - F(u(a))\\ &= \psi(k_h)k_h = \psi(\chi(h)\cdot h)\chi(h)\cdot h\\ \end{align*}$$

Since continuous functions are closed under composition and multiplication, $F \circ u$ is differentiable, with $\phi(h) = \psi(\chi(h)\cdot h)\chi(h)$ defining the necessary continuous function $\phi$. Thus, $\nabla(v \circ u)(x) = \psi(0)\chi(0) = v'(u(a))\nabla u(a)$. $\square$


Remark: Since the differential is so closely related to the gradient, it should come as no surprise that it has the same rules of calculation as the gradient. The only difference, of course, is that computing the gradient gives you a vector, while computing the differential gives you a linear form.


Exercises