We will start by discussing what a differential is. In some ways this is familiar, but in other ways it might not be.

Let us start by revisiting an example in the one-dimensional case. We proceed informally for now. Intuitively, a differential is just a very small change in some function $\varphi$ caused by some very small change in its input. That is, when $\varphi$ is a function evaluated at $x$ and when $dx$ is very small, we have that:

$$\varphi(x + dx) - \varphi(x) \approx d\varphi_x$$

Where the subscript $x$ specifies the point at which we are differentiating. Let us work out a simple example. Suppose we have the function $\varphi(x) = x^2$. It is easy to compute the differential using the binomial theorem. Since $dx \approx 0$ and the squares of small numbers, less than 1, are much smaller than the original number, higher order $dx$ terms are negligible, so:

$$ \begin{align*} \varphi(x + dx) - \varphi(x) &= (x + dx)^2 - x^2\\ &= x^2 + 2x\,dx + dx^2 - x^2\\ &= 2x\,dx + dx^2\\ &\approx 2x\,dx \end{align*} $$

Therefore $d\varphi_x = 2x\,dx$. What it means for a function to be differentiable is that when we do this sort of thing, we always end up with a sensible result like this one for the differential: it is just some number, the derivative $\varphi'(x)$, multiplied by the size of the tiny nudge $dx$, or $d\varphi_x = \varphi'(x)\,dx$

Let's now think about how this generalizes. Imagine that rather than being a number, $2x$ is a 1x1 matrix, and that rather than being a small nudge number, $dx$ is a small nudge vector. Then $d\varphi_x$ is a linear transformation that takes in a little nudge and has a matrix representation of $\varphi'(x) = 2x$, which is the derivative of the function.

With that out of the way, we can finally ask: what does it take to generalize the derivative and differential to higher dimensions? The answer is basically nothing. A computational example should help illustrate this. Consider a function $\varphi: \mathbb{R}^3 \rightarrow \mathbb{R}^2$ defined by:

$$\varphi\begin{pmatrix}x \\ y \\ z \\ \end{pmatrix} = \begin{pmatrix}xy\\ yz\end{pmatrix}$$

We can find the differential by fixing $a = (x, y, z)$ and considering a nudge vector $h = (dx, dy, dx)$, then doing the same thing we did last time.

$$ \begin{align*} \varphi(a + h) - \varphi(a) &= \begin{pmatrix} (x + dx)(y + dy) \\ (y + dy)(z + dz) \end{pmatrix} - \begin{pmatrix}xy\\ yz\end{pmatrix}\\ &= \begin{pmatrix} xy+x\,dy + y\,dx + dy\,dx \\ yz + z\,dy + y\,dz + dy\,dz \end{pmatrix} - \begin{pmatrix}xy\\ yz\end{pmatrix}\\ &= \begin{pmatrix} x\,dy + y\,dx + dy\,dx \\ z\,dy + y\,dz + dy\,dz \end{pmatrix}\\ &\approx \begin{pmatrix} y\,dx + x\,dy + 0\,dz \\ 0\,dx + z\,dy + y\,dz \end{pmatrix}\\ &= \begin{pmatrix} y & x & 0 \\ 0 & z & y \end{pmatrix}\begin{pmatrix} dx \\ dy \\ dx \end{pmatrix}\\ &= \varphi'(a)\,h \end{align*} $$

We once again get that $d\varphi_a$ is a linear transformation that takes in a little nudge and has a matrix representation of $\varphi'(x)$, which is the derivative of the function. The only difference is that this time, the matrix is 2x3 rather than 1x1, because we're going from $\mathbb{R}^3$ to $\mathbb{R}^2$ rather than $\mathbb{R}$ to $\mathbb{R}$.

For our purposes, we're mostly only going to be interested in the following cases: the case of functions from $\mathbb{R}^n$ to $\mathbb{R}^n$, and from $\mathbb{R}^n$ to $\mathbb{R}$.