Key solution concepts
- Assume we want to compute the derivatives of f(x) at
x=a, as shown in the figure.
- Conceptually, in the finite difference method we expand f(x)
in a Taylor series around the point x=a, which gives
f(x) = f(a) + f '(a) (x-a) + f ''(a) (x-a)2/ 2! +
f '''(a) (x-a)3/ 3! + . . .
(1)
- Equation (1) is a general expression which, with infinite terms,
gives us the mathematically exact value of f(x) for any
x within the domain of convergence of the series.
- In the finite difference method, we use a truncated form of (1)
to approximate the values of f(x) at specific points in the
neighborhood of x=a. For example, at the points
x=a-h and x=a+h we get from (1)
f(a-h) = f(a) - f '(a) h + f ''(a) h2/ 2! -
f '''(a) h3 /3! + - . . .
(2)
f(a+h) = f(a) + f '(a) h + f ''(a) h2/ 2! +
f '''(a) h3/ 3! + . . .
(3)
Here we have used the fact that x-a =-h in (2)
and x-a =+h in (3).
Approximations for 1st derivative
- Equations (2) and (3) give us a few different choices for
approximating the 1st derivative of f at x=a.
One choice would be to use either (2) or (3) independently
and ignore terms of order h2 and higher. This
gives the following choices of approximation for f '(a) :
f '(a) = [ f(a) - f(a-h) ] / h
(4)
f '(a) = [ f(a+h) - f(a) ] / h
(5)
- Equation (4) is a backward difference approximation for
f '(a), and (5) is a forward difference approximation.
Both are of first order accuracy, which is conventionally denoted by
O(h).
- Another choice for approximating the 1st derivative is to use
(2) and (3) simultaneously. In this case we ignore terms of order
h3 and higher, and then we eliminate the term
that involves f ''(a) by subtracting (2) from (3). This
gives
f '(a) = [ f(a+h) - f(a-h) ] / (2 h)
(6)
This is a central difference approximation, since it uses one
point on either side of x=a. It is of 2nd order accuracy,
i.e., O(h2), which is superior to
that of (4) and (5).
Approximations for 2nd derivative
- Using (2) and (3) simultaneously it is also possible to construct
an approximation for f ''(a). To do this, we ignore terms of
order h4 and higher, and then eliminate the term
that that involves f '(a) by adding (2) and (3). This yields
f ''(a) = [ f(a+h) - 2f(a) + f(a-h) ] / h2
(7)
- The fortuituous cancellation of the h3 term
that also results from this operation makes (7) a 2nd order accurate,
O(h2), approximation for the 2nd derivative.
How do we know the order of accuracy?
- Each of the above approximations is derived by neglecting
higher order terms in the infinite series (2) and (3). We can derive
what is known as the truncation error for each approximation
by retaining the first neglected term, and carrying it through
the derivations.
- For example, in deriving the central difference approximation
(6), if we retain the h3 terms before subtracting
(2) from (3) we get
f(a+h) - f(a-h) = 2 f '(a) h + 2 f '''(a) h3/ 6
==> f '(a) = [ f(a+h) - f(a-h) ] / (2h) -
f '''(a) h2/ 6
In other words, the truncation error is
f '''(a) h2/ 6.
Therefore, the order of accuracy of the approximation is 2 (since the
power of h in the truncation error is 2).
- We can similarly derive truncation errors for each of the above
approximations and show that:
f '(a) = [ f(a) - f(a-h) ] / h +
f ''(a) h / 2
f '(a) = [ f(a+h) - f(a) ] / h -
f ''(a) h / 2
f ''(a) = [ f(a+h) - 2f(a) + f(a-h) ] / h2 -
f (iv)(a) h2 / 12
In each case, the power of h in the truncation error tells us
the order of accuracy of the approximation.