Differentiability

Differentiable functions

We will start from the one-variable definition of the derivative at a point p, namely

$\lim _{x\rightarrow p}{f(x)-f(p) \over x-p}=f'(p)$

Let's change above to equivalent form of

$\lim _{x\rightarrow p}{f(x)-f(p)-f'(p)(x-p) \over x-p}=0$

which achieved after pulling f'(p) inside and putting it over a common denominator.

We can't divide by vectors, so this definition can't be immediately extended to the multiple variable case. Nonetheless, we don't have to: the thing we took interest in was the quotient of two small distances (magnitudes), not their other properties (like sign). It's worth noting that 'other' property of vector neglected is its direction. Now we can divide by the absolute value of a vector, so lets rewrite this definition in terms of absolute values

$\lim _{x\rightarrow p}{\frac {\left|f(x)-f(p)-f'(p)(x-p)\right|}{\left|x-p\right|}}=0$

Another form of formula above is obtained by letting $h=x-p$ we have $x=p+h$ and if $x\rightarrow p$, the $h=x-p\rightarrow 0$, so

$\lim _{h\rightarrow 0}{\frac {\left|f(p+h)-f(p)-f'(p)h\right|}{\left|h\right|}}=0$,

where $h$ can be thought of as a 'small change'.

So, how can we use this for the several-variable case?

If we switch all the variables over to vectors and replace the constant (which performs a linear map in one dimension) with a matrix (which denotes also a linear map), we have

$\lim _{\mathbf {x} \rightarrow \mathbf {p} }{|\mathbf {f} (\mathbf {x} )-\mathbf {f} (\mathbf {p} )-\mathbf {A} (\mathbf {x} -\mathbf {p} )| \over |\mathbf {x} -\mathbf {p} |}=0$

or

$\lim _{\mathbf {h} \rightarrow \mathbf {0} }{|\mathbf {f} (\mathbf {p} +\mathbf {h} )-\mathbf {f} (\mathbf {p} )-\mathbf {A} \mathbf {h} | \over |\mathbf {h} |}=0$

If this limit exists for some f : R^m → Rⁿ, and there is a linear map A : R^m → Rⁿ (denoted by matrix A which is m×n), we refer to this map as being the derivative and we write it as D_p f.

A point on terminology - in referring to the action of taking the derivative (giving the linear map A), we write D_p f, but in referring to the matrix A itself, it is known as the Jacobian matrix and is also written J_p f. More on the Jacobian later.

Properties

There are a number of important properties of this formulation of the derivative.

Affine approximations

If f is differentiable at p for x close to p, |f(x)-(f(p)+A(x-p))| is small compared to |x-p|, which means that f(x) is approximately equal to f(p)+A(x-p).

We call an expression of the form g(x)+c affine, when g(x) is linear and c is a constant. f(p)+A(x-p) is an affine approximation to f(x).

Jacobian matrix and partial derivatives

The Jacobian matrix of a function is in the form

$\left(J_{\mathbf {p} }\mathbf {f} \right)_{ij}=\left.{\partial f_{i} \over \partial x_{j}}\right|_{\mathbf {p} }$

for a f : R^m → Rⁿ, J_p f' is a n×m matrix.

The consequence of this is that if f is differentiable at p, all the partial derivatives of f exist at p.

However, it is possible that all the partial derivatives of a function exist at some point yet that function is not differentiable there, so it is very important not to mix derivative (linear map) with the Jacobian (matrix) especially in situations akin to the one cited.

Rules of taking Jacobians If f : R^m → Rⁿ, and h(x) : R^m → R are differentiable at 'p':

• $J_{\mathbf {p} }(\mathbf {f} +\mathbf {g} )=J_{\mathbf {p} }\mathbf {f} +J_{\mathbf {p} }\mathbf {g}$
• $J_{\mathbf {p} }(h\mathbf {f} )=hJ_{\mathbf {p} }\mathbf {f} +\mathbf {f} (\mathbf {p} )J_{\mathbf {p} }h$
• $J_{\mathbf {p} }(\mathbf {f} \cdot \mathbf {g} )=\mathbf {g} ^{T}J_{\mathbf {p} }\mathbf {f} +\mathbf {f} ^{T}J_{\mathbf {p} }\mathbf {g}$

Important: make sure the order is right - matrix multiplication is not commutative!

Chain rule

The chain rule for functions of several variables is as follows. For f : R^m → Rⁿ and g : Rⁿ → R^p, and g o f differentiable at p, then the Jacobian is given by

$\left(J_{\mathbf {f} (\mathbf {p} )}\mathbf {g} \right)\left(J_{\mathbf {p} }\mathbf {f} \right)$

Again, we have matrix multiplication, so one must preserve this exact order. Compositions in one order may be defined, but not necessarily in the other way.

Continuity and differentiability

Furthermore, if all the partial derivatives exist, and are continuous in some neighbourhood of a point p, then f is differentiable at p. This has the consequence that for a function f which has its component functions built from continuous functions (such as rational functions, differentiable functions or otherwise), f is differentiable everywhere f is defined.

We use the terminology continuously differentiable for a function differentiable at p which has all its partial derivatives existing and are continuous in some neighbourhood at p.

Resources

• Partial Derivatives, WikiBooks: Calculus/Multivariable Calculus
• Chain Rule, WikiBooks: Calculus/Multivariable Calculus
• Tangent Planes and Differentials, Mathematics LibreTexts

Licensing

Content obtained and/or adapted from:

• Partial Derivatives, WikiBooks: Calculus/Multivariable Calculus under a CC BY-SA license
• Chain Rule, WikiBooks: Calculus/Multivariable Calculus under a CC BY-SA license