Linear Approximations and Differentials

Tangent line at (a, f(a))

In mathematics, a linear approximation is an approximation of a general function using a linear function (more precisely, an affine function). They are widely used in the method of finite differences to produce first order methods for solving or approximating solutions to equations.

Linear Approximation

Given a twice continuously differentiable function $f$ of one real variable, Taylor's theorem for the case $n=1$ states that

$f(x)=f(a)+f'(a)(x-a)+R_{2}\$

where $R_{2}$ is the remainder term. The linear approximation is obtained by dropping the remainder:

$f(x)\approx f(a)+f'(a)(x-a)$.

This is a good approximation when $x$ is close enough to $a$; since a curve, when closely observed, will begin to resemble a straight line. Therefore, the expression on the right-hand side is just the equation for the tangent line to the graph of $f$ at $(a,f(a))$. For this reason, this process is also called the tangent line approximation.

If $f$ is concave down in the interval between $x$ and $a$, the approximation will be an overestimate (since the derivative is decreasing in that interval). If $f$ is concave up, the approximation will be an underestimate.

Linear approximations for vector functions of a vector variable are obtained in the same way, with the derivative at a point replaced by the Jacobian matrix. For example, given a differentiable function $f(x,y)$ with real values, one can approximate $f(x,y)$ for $(x,y)$ close to $(a,b)$ by the formula

$f\left(x,y\right)\approx f\left(a,b\right)+{\frac {\partial f}{\partial x}}\left(a,b\right)\left(x-a\right)+{\frac {\partial f}{\partial y}}\left(a,b\right)\left(y-b\right).$

The right-hand side is the equation of the plane tangent to the graph of $z=f(x,y)$ at $(a,b).$

In the more general case of Banach spaces, one has

$f(x)\approx f(a)+Df(a)(x-a)$

where $Df(a)$ is the Fréchet derivative of $f$ at $a$.

Applications

Optics

Gaussian optics is a technique in geometrical optics that describes the behavior of light rays in optical systems by using the paraxial approximation, in which only rays which make small angles with the optical axis of the system are considered. In this approximation, trigonometric functions can be expressed as linear functions of the angles. Gaussian optics applies to systems in which all the optical surfaces are either flat or are portions of a sphere. In this case, simple explicit formulae can be given for parameters of an imaging system such as focal distance, magnification and brightness, in terms of the geometrical shapes and material properties of the constituent elements.

Period of oscillation

The period of swing of a simple gravity pendulum depends on its length, the local strength of gravity, and to a small extent on the maximum angle that the pendulum swings away from vertical, θ₀, called the amplitude. It is independent of the mass of the bob. The true period T of a simple pendulum, the time taken for a complete cycle of an ideal simple gravity pendulum, can be written in several different forms, one example being the infinite series:

$T=2\pi {\sqrt {L \over g}}\left(1+{\frac {1}{16}}\theta _{0}^{2}+{\frac {11}{3072}}\theta _{0}^{4}+\cdots \right)$

where L is the length of the pendulum and g is the local acceleration of gravity.

However, if one takes the linear approximation (i.e. if the amplitude is limited to small swings,) the period is:

$T\approx 2\pi {\sqrt {\frac {L}{g}}}\qquad \qquad \qquad \theta _{0}\ll 1\qquad (1)\,$

In the linear approximation, the period of swing is approximately the same for different size swings: that is, the period is independent of amplitude. This property, called isochronism, is the reason pendulums are so useful for timekeeping. Successive swings of the pendulum, even if changing in amplitude, take the same amount of time.

Electrical resistivity

The electrical resistivity of most materials changes with temperature. If the temperature T does not vary too much, a linear approximation is typically used:

$\rho (T)=\rho _{0}[1+\alpha (T-T_{0})]$

where $\alpha$ is called the temperature coefficient of resistivity, $T_{0}$ is a fixed reference temperature (usually room temperature), and $\rho _{0}$ is the resistivity at temperature $T_{0}$. The parameter $\alpha$ is an empirical parameter fitted from measurement data. Because the linear approximation is only an approximation, $\alpha$ is different for different reference temperatures. For this reason it is usual to specify the temperature that $\alpha$ was measured at with a suffix, such as $\alpha _{15}$, and the relationship only holds in a range of temperatures around the reference. When the temperature varies over a large temperature range, the linear approximation is inadequate and a more detailed analysis and understanding should be used.

Differentials

The term differential is used in calculus to refer to aninfinitesimal]](infinitely small) change in some varying quantity. For example, if x is a variable, then a change in the value of x is often denoted Δx (pronounced delta x). The differential dx represents an infinitely small change in the variable x. The idea of an infinitely small or infinitely slow change is, intuitively, extremely useful, and there are a number of ways to make the notion mathematically precise.

Using calculus, it is possible to relate the infinitely small changes of various variables to each other mathematically using derivatives. If y is a function of x, then the differential dy of y is related to dx by the formula

$dy={\frac {dy}{dx}}\,dx,$

where ${\frac {dy}{dx}}\,$denotes the derivative of y with respect to x. This formula summarizes the intuitive idea that the derivative of y with respect to x is the limit of the ratio of differences Δy/Δx as Δx becomes infinitesimal.

There are several approaches for making the notion of differentials mathematically precise.

1. Differentials as linear maps. This approach underlies the definition of the derivative and the exterior derivative in differential geometry.
2. Differentials as nilpotent elements of commutative rings. This approach is popular in algebraic geometry.
3. Differentials in smooth models of set theory. This approach is known as synthetic differential geometry or smooth infinitesimal analysis and is closely related to the algebraic geometric approach, except that ideas from topos theory are used to hide the mechanisms by which nilpotent infinitesimals are introduced.
4. Differentials as infinitesimals in hyperreal number systems, which are extensions of the real numbers that contain invertible infinitesimals and infinitely large numbers. This is the approach of nonstandard analysis pioneered by Abraham Robinson.

These approaches are very different from each other, but they have in common the idea of being quantitative, i.e., saying not just that a differential is infinitely small, but how small it is.

See also: Implicit Differentiation

Resources

• Differential (infinitesimal), Wikipedia
• Linearization and Differentials PowerPoint file created by Dr. Sara Shirinkam, UTSA.
• Linearization and Differentials Worksheet 1
• Linearization and Differentials Worksheet 2
• Cal1 Linearization Part1 video lecture by Instructor Beatty, UTSA.
• Cal1 Differentials Part1 video lecture by Instructor Beatty, UTSA.
• Cal1 Differentials Part2 video lecture by Instructor Beatty, UTSA.
• Cal1 Differentials Part3 video lecture by Instructor Beatty, UTSA.

Licensing

Content obtained and/or adapted from:

• Differential (infinitesimal), Wikipedia under a CC BY-SA license
• Linear approximation, Wikipedia under a CC BY-SA license