Maxima and Minima Problems

The four types of extrema.

Maxima and minima are points where a function reaches a highest or lowest value, respectively. There are two kinds of extrema (a word meaning maximum or minimum): global and local, sometimes referred to as "absolute" and "relative", respectively. A global maximum is a point that takes the largest value on the entire range of the function, while a global minimum is the point that takes the smallest value on the range of the function. On the other hand, local extrema are the largest or smallest values of the function in the immediate vicinity.

In many cases, extrema look like the crest of a hill or the bottom of a bowl on a graph of the function. A global extremum is always a local extremum too, because it is the largest or smallest value on the entire range of the function, and therefore also its vicinity. It is also possible to have a function with no extrema, global or local: $y=x$ is a simple example.

At any extremum, the slope of the graph is necessarily 0 (or is undefined, as in the case of $-|x|$), as the graph must stop rising or falling at an extremum, and begin to head in the opposite direction. Because of this, extrema are also commonly called stationary points or turning points. Therefore, the first derivative of a function is equal to 0 at extrema. If the graph has one or more of these stationary points, these may be found by setting the first derivative equal to 0 and finding the roots of the resulting equation.

The function $f(x)=x^{3}$ , which contains a saddle point at the point $(0,0)$.

However, a slope of zero does not guarantee a maximum or minimum: there is a third class of stationary point called a saddle point. Consider the function

$f(x)=x^{3}$

The derivative is

$f'(x)=3x^{2}$

The slope at $x=0$ is 0. We have a slope of 0, but while this makes it a stationary point, this doesn't mean that it is a maximum or minimum. Looking at the graph of the function you will see that $x=0$ is neither, it's just a spot at which the function flattens out. True extrema require a sign change in the first derivative. This makes sense - you have to rise (positive slope) to and fall (negative slope) from a maximum. In between rising and falling, on a smooth curve, there will be a point of zero slope - the maximum. A minimum would exhibit similar properties, just in reverse.

Good (B and C, green) and bad (D and E, blue) points to check in order to classify the extremum (A, black). The bad points lead to an incorrect classification of A as a minimum.

This leads to a simple method to classify a stationary point - plug x values slightly left and right into the derivative of the function. If the results have opposite signs then it is a true maximum/minimum. You can also use these slopes to figure out if it is a maximum or a minimum: the left side slope will be positive for a maximum and negative for a minimum. However, you must exercise caution with this method, as, if you pick a point too far from the extremum, you could take it on the far side of another extremum and incorrectly classify the point.

The Extremum Test

A more rigorous method to classify a stationary point is called the extremum test, or 2nd Derivative Test. As we mentioned before, the sign of the first derivative must change for a stationary point to be a true extremum. Now, the second derivative of the function tells us the rate of change of the first derivative. It therefore follows that if the second derivative is positive at the stationary point, then the gradient is increasing. The fact that it is a stationary point in the first place means that this can only be a minimum. Conversely, if the second derivative is negative at that point, then it is a maximum.

Now, if the second derivative is 0, we have a problem. It could be a point of inflexion, or it could still be an extremum. Examples of each of these cases are below - all have a second derivative equal to 0 at the stationary point in question:

• $y=x^{3}$ has a point of inflexion at $x=0$
• $y=x^{4}$ has a minimum at $x=0$
• $y=-x^{4}$ has a maximum at $x=0$

However, this is not an insoluble problem. What we must do is continue to differentiate until we get, at the $(n+1)$th derivative, a non-zero result at the stationary point:

$f'(x)=0\ ,\ f''(x)=0\ ,\ \ldots \ ,\ f^{(n)}(x)=0\ ,\ f^{(n+1)}(x)\neq 0$

If $n$ is odd, then the stationary point is a true extremum. If the $(n+1)$th derivative is positive, it is a minimum; if the $(n+1)$th derivative is negative, it is a maximum. If $n$ is even, then the stationary point is a point of inflexion.

As an example, let us consider the function

$f(x)=-x^{4}$

We now differentiate until we get a non-zero result at the stationary point at $x=0$ (assume we have already found this point as usual):

$f'(x)=-4x^{3}$

$f''(x)=-12x^{2}$

$f'''(x)=-24x$

$f^{(4)}(x)=-24$

Therefore, $n+1$ is 4, so $n$ is 3. This is odd, and the fourth derivative is negative, so we have a maximum. Note that none of the methods given can tell you if this is a global extremum or just a local one. To do this, you would have to set the function equal to the height of the extremum and look for other roots.

Critical Points

Critical points are the points where a function's derivative is 0 or not defined. Suppose we are interested in finding the maximum or minimum on given closed interval of a function that is continuous on that interval. The extreme values of the function on that interval will be at one or more of the critical points and/or at one or both of the endpoints. We can prove this by contradiction. Suppose that the function $f(x)$ has maximum at a point $x=c$ in the interval $(a,b)$ where the derivative of the function is defined and not $0$ . If the derivative is positive, then $x$ values slightly greater than $c$ will cause the function to increase. Since $c$ is not an endpoint, at least some of these values are in $[a,b]$ . But this contradicts the assumption that $f(c)$ is the maximum of $f(x)$ for $x$ in $[a,b]$ . Similarly, if the derivative is negative, then $x$ values slightly less than $c$ will cause the function to increase. Since $c$ is not an endpoint, at least some of these values are in $[a,b]$ . This contradicts the assumption that $f(c)$ is the maximum of $f(x)$ for $x$ in $[a,b]$ . A similar argument could be made for the minimum.

Example 1

Consider the function $f(x)=x$ on the interval $[-1,1]$ . The unrestricted function $f(x)=x$ has no maximum or minimum. On the interval $[-1,1]$ , however, it is obvious that the minimum will be $-1$ , which occurs at $=-1$ and the maximum will be $1$, which occurs at $x=1$ . Since there are no critical points ($f'(x)$ exists and equals $1$ everywhere), the extreme values must occur at the endpoints.

Example 2

Find the maximum and minimum of the function $f(x)=x^{3}-2x^{2}-5x+6$ on the interval $[-3,3]$ .
First start by finding the roots of the function derivative:

$f'(x)=3x^{2}-4x-5=0$

$x={\frac {2\pm {\sqrt {19}}}{3}}$

Now evaluate the function at all critical points and endpoints to find the extreme values.

$f(-3)=-24$

$f({\frac {2-{\sqrt {19}}}{3}})={\frac {56+38{\sqrt {19}}}{27}}\approx 8.2088$

$f({\frac {2+{\sqrt {19}}}{3}})={\frac {56-38{\sqrt {19}}}{27}}\approx -4.0607$

$f(3)=0$

From this we can see that the minimum on the interval is -24 when $x=-3$ and the maximum on the interval is ${\frac {56+38{\sqrt {19}}}{27}}$ when $x={\frac {2-{\sqrt {19}}}{3}}$

Saddle Point

A saddle point (in red) on the graph of z=x²−y² (hyperbolic paraboloid)

Saddle point between two hills (the intersection of the figure-eight $z$-contour)

In mathematics, a saddle point or minimax point is a point on the surface of the graph of a function where the slopes (derivatives) in orthogonal directions are all zero (a critical point), but which is not a local extremum of the function. An example of a saddle point is when there is a critical point with a relative minimum along one axial direction (between peaks) and at a relative maximum along the crossing axis. However, a saddle point need not be in this form. For example, the function $f(x,y)=x^{2}+y^{3}$ has a critical point at $(0,0)$ that is a saddle point since it is neither a relative maximum nor relative minimum, but it does not have a relative maximum or relative minimum in the $y$-direction.

The name derives from the fact that the prototypical example in two dimensions is a surface that curves up in one direction, and curves down in a different direction, resembling a riding saddle or a mountain pass between two peaks forming a landform saddle. In terms of contour lines, a saddle point in two dimensions gives rise to a contour graph or trace in which the contour corresponding to the saddle point's value appears to intersect itself.

Saddle point on the countour plot is the point where level curves cross

Mathematical discussion

A simple criterion for checking if a given stationary point of a real-valued function F(x,y) of two real variables is a saddle point is to compute the function's Hessian matrix at that point: if the Hessian is indefinite, then that point is a saddle point. For example, the Hessian matrix of the function $z=x^{2}-y^{2}$ at the stationary point $(x,y,z)=(0,0,0)$ is the matrix

${\begin{bmatrix}2&0\\0&-2\\\end{bmatrix}}$

which is indefinite. Therefore, this point is a saddle point. This criterion gives only a sufficient condition. For example, the point $(0,0,0)$ is a saddle point for the function $z=x^{4}-y^{4},$ but the Hessian matrix of this function at the origin is the null matrix, which is not indefinite.

In the most general terms, a saddle point for a smooth function (whose graph is a curve, surface or hypersurface) is a stationary point such that the curve/surface/etc. in the neighborhood of that point is not entirely on any side of the tangent space at that point.

The plot of y = x³ with a saddle point at 0

In a domain of one dimension, a saddle point is a point which is both a stationary point and a point of inflection. Since it is a point of inflection, it is not a local extremum.

Saddle surface

Hyperbolic paraboloid

A model of an elliptic hyperboloid of one sheet

A monkey saddle

A saddle surface is a smooth surface containing one or more saddle points.

Classical examples of two-dimensional saddle surfaces in the Euclidean space are second order surfaces, the hyperbolic paraboloid $z=x^{2}-y^{2}$ (which is often referred to as "the saddle surface" or "the standard saddle surface") and the hyperboloid of one sheet. The Pringles potato chip or crisp is an everyday example of a hyperbolic paraboloid shape.

Saddle surfaces have negative Gaussian curvature which distinguish them from convex/elliptical surfaces which have positive Gaussian curvature. A classical third-order saddle surface is the monkey saddle.

Examples

In a two-player zero sum game defined on a continuous space, the equilibrium point is a saddle point.

For a second-order linear autonomous system, a critical point is a saddle point if the characteristic equation has one positive and one negative real eigenvalue.

In optimization subject to equality constraints, the first-order conditions describe a saddle point of the Lagrangian.

Other uses

In dynamical systems, if the dynamic is given by a differentiable map f then a point is hyperbolic if and only if the differential of ƒ ⁿ (where n is the period of the point) has no eigenvalue on the (complex) unit circle when computed at the point. Then a saddle point is a hyperbolic periodic point whose stable and unstable manifolds have a dimension that is not zero.

A saddle point of a matrix is an element which is both the largest element in its column and the smallest element in its row.

Extrema for functions of two variables

Suppose that f(x, y) is a differentiable real function of two variables whose second partial derivatives exist and are continuous. The Hessian matrix H of f is the 2 × 2 matrix of partial derivatives of f:

$$H(x,y)={\begin{bmatrix}f_{xx}(x,y)&f_{xy}(x,y)\\f_{yx}(x,y)&f_{yy}(x,y)\end{bmatrix}}.$$

Define D(x, y) to be the determinant

$$D(x,y)=\det(H(x,y))=f_{xx}(x,y)f_{yy}(x,y)-\left(f_{xy}(x,y)\right)^{2}.$$

of H. Finally, suppose that (a, b) is a critical point of f (that is, f_x(a, b) = f_y(a, b) = 0). Then the second partial derivative test asserts the following:

1. If D(a, b) > 0 and f_xx(a, b) > 0 then (a, b) is a local minimum of f.
2. If D(a, b) > 0 and f_xx(a, b) < 0 then (a, b) is a local maximum of f.
3. If D(a, b) < 0 then (a, b) is a saddle point of f.
4. If D(a, b) = 0 then the second derivative test is inconclusive, and the point (a, b) could be any of a minimum, maximum or saddle point.

Sometimes other equivalent versions of the test are used. Note that in cases 1 and 2, the requirement that f_xx f_yy − f_xy² is positive at (x, y) implies that f_xx and f_yy have the same sign there. Therefore the second condition, that f_xx be greater (or less) than zero, could equivalently be that f_yy or tr(H) = f_xx + f_yy be greater (or less) than zero at that point.

Functions of many variables

For a function f of three or more variables, there is a generalization of the rule above. In this context, instead of examining the determinant of the Hessian matrix, one must look at the eigenvalues of the Hessian matrix at the critical point. The following test can be applied at any critical point a for which the Hessian matrix is invertible:

1. If the Hessian is positive definite (equivalently, has all eigenvalues positive) at a, then f attains a local minimum at a.
2. If the Hessian is negative definite (equivalently, has all eigenvalues negative) at a, then f attains a local maximum at a.
3. If the Hessian has both positive and negative eigenvalues then a is a saddle point for f (and in fact this is true even if a is degenerate).

In those cases not listed above, the test is inconclusive.

For functions of three or more variables, the determinant of the Hessian does not provide enough information to classify the critical point, because the number of jointly sufficient second-order conditions is equal to the number of variables, and the sign condition on the determinant of the Hessian is only one of the conditions. Note that in the one-variable case, the Hessian condition simply gives the usual second derivative test.

In the two variable case, $D(a,b)$ and $f_{xx}(a,b)$ are the principal minors of the Hessian. The first two conditions listed above on the signs of these minors are the conditions for the positive or negative definiteness of the Hessian. For the general case of an arbitrary number n of variables, there are n sign conditions on the n principal minors of the Hessian matrix that together are equivalent to positive or negative definiteness of the Hessian (Sylvester's criterion): for a local minimum, all the principal minors need to be positive, while for a local maximum, the minors with an odd number of rows and columns need to be negative and the minors with an even number of rows and columns need to be positive.

Resources

• Local Extrema, Critical Points, & Saddle Points of Multivariable Functions - Calculus 3- Video by he Organic Chemistry Tutor 2019

• Absolute Maximum and Minimum Values of Multivariable Functions - Calculus 3 Video by The Organic Chemistry Tutor 2019

Licensing

Content obtained and/or adapted from:

• Extrema and Points of Inflection, Wikibooks: Calculus under a CC BY-SA license
• Second partial derivative test, Wikipedia under a CC BY-SA license
• Saddle point, Wikipedia under a CC BY-SA license