It is a method to approximate by numerical calculation when it is troublesome to calculate or implement the derivative of the formula you want to find by the steepest descent method or the Levenberg-Marquardt method. It seems that the following three types are often used, and it seems that you should decide by considering the balance between the amount of calculation and the accuracy.
The simplest approximation and the fastest calculation. The accuracy is reasonable. Expressed in a mathematical formula
f'(x_0) \approx \frac{f(x_0+h)-f(x_0)}{h}
However, $ h $ is small enough. When implemented in C language,
#include <float.h>
extern double f( double x );
double df( double x )
{
const double h = FLT_EPSILON;
double y1 = f( x + h );
double y2 = f( x );
return ( y1 - y2 ) / h;
}
It seems that $ h $ can be calculated positively or negatively, but the result will change.
FLT_EPSILON is the value defined in float.h.
It is a small value and is defined as the value of the difference between 1 and the "minimum number greater than 1" called mechanical epsilon.
FLT_EPSILON itself is a mechanical epsilon for float, but if the value of x is small, rounding error is likely to occur, so I am using this for the time being, not for double.
There is also a definition for double, and DBL_EPSILON is defined in the same float.h.
If it is a two-point approximation, even the point $ x_0 $ where the derivative value becomes zero is unlikely to become zero, so it is a method to improve it a little and use the points before and after the point $ x_0 $. The amount of calculation is slightly higher than the two-point approximation, and the accuracy is improved.
Expressed in a mathematical formula
f'(x_0) \approx \frac{f(x_0+h)-f(x_0-h)}{2h}
When implemented in C language,
#include <float.h>
extern double f( double x );
double df( double x )
{
const double h = FLT_EPSILON;
double y1 = f( x + h );
double y2 = f( x - h );
return ( y1 - y2 ) / ( 2 * h );
}
Although it is called a 3-point approximation, only 2 points appear in the calculation.
Considering the mean value theorem, it seems more likely that this is closer to the true value of x.
The result is the same regardless of whether $ h $ is positive or negative.
f'(x_0) \approx \frac{f(x_0-2h)-8f(x_0-h)+8f(x_0+h)-f(x_0+2h)}{12h}
It became an expression that I could not understand at once. It seems to be derived by making full use of Taylor expansion and Lagrange interpolation polynomial.
When implemented in C language,
#include <float.h>
extern double f( double x );
double df( double x )
{
const double h = FLT_EPSILON;
double y1 = f( x + h );
double y2 = f( x - h );
double y3 = f( x + 2 * h );
double y4 = f( x - 2 * h );
return ( y4 - 8 * y2 + 8 * y1 - y3 ) / ( 12 * h );
}
I don't know what it is anymore, so I'm grateful to use it, just remembering that it can be calculated this way.
If it is a function made up of multiple variables, it will be partially differentiated, but the same thing is done. If you want to partially differentiate $ y $ of $ f (x, y, z) $ with a two-point approximation
f_y(x_0,y_0,z_0) \approx \frac{f(x_0,y_0+h,z_0)-f(x_0,y_0,z_0)}{h}
You will implement this.
Recommended Posts