For a differentiable function \(f:\mathbb{R} \rightarrow \mathbb{R}\), the derivative is defined as
Let’s consider the forward finite difference approximation to the first derivative as
where \(h\) is often called a “perturbation”, i.e., a “small” change to the variable \(x\) (small when compared to the magnitude of \(x\)). By the Taylor’s theorem, we can write
for some \(\xi \in [x,x+h]\). Rearranging the above we get
Therefore, the truncation error of the finite difference approximation is bounded by \(M\,h/2\), where \(M\) is a bound on \(\vert f''(\xi) \vert\) for \(\xi\) near \(x\).
Using a similar approach, we can summarize the following finite difference approximations:
In addition to the computation of \(f(x)\), this method requires one function evaluation for a given perturbation, and has truncation order \(O(h) \).
\[f'(x) = \frac{f(x+h)-f(x)}{h}\]In addition to the computation of \(f(x)\), this method requires one function evaluation for a given perturbation, and has truncation order \(O(h) \).
\[f'(x) = \frac{f(x)-f(x-h)}{h}\]This method requires two function evaluations for a given perturbation (\(f(x+h)\) and \(f(x-h)\) ), and has truncation order \(O(h^2) \).
\[f'(x) = \frac{f(x+h)-f(x-h)}{2h}\]Reference text: “Scientific Computing: an introductory survey” by Michael Heath