Ordinary differential equations (ODEs) are among the most important mathe-matical tools used in producing models in the physical sciences, biosciences, chemical sciences, engineering and many more fields. This has motivated re-searchers to provide efficient numerical methods for solving such equations. Most of these types of differential models are stiff, and suitable numerical methods have to be used to simulate the solutions. This paper starts with a sur-vey on the basic properties of stiff differential equations. Thereafter, we present the explicit one-step algorithm proposed by Fatunla to solve stiff systems of first-order scalar ODEs. As an illustrative example, we consider the Robertson problem (RP) which is known to be stiff. The results obtained with the explicit Fatunla method (EFM) are compared with those computed by the solver RADAU which is based on implicit Runge-Kutta methods. Our results are in good agreement with the latter ones.
Numerical solutions for ordinary differential equations (ODEs) are very important in scientific computation, as they are widely used to model real world problems [
The phenomenon of stiffness was first recognized by Curtiss and Hirschfelder [
Stiff problems are too important to ignore, and are too expensive to overpower. They are too important to ignore because they occur in many physically important situations. They are too expensive to overpower because of their size and the inherent difficulty they present to classical methods. Even if one can bear the expense, classical methods of solution require so many steps that round off errors may invalidate the solution [
Stiff problem entails rapidly decaying transient solution, which arises naturally in wide variety of applications including the study of spring and damping systems, the analysis of control system and problems in the chemical kinetics [
We have to emphasize that while the intuitive meaning of the term stiff is clear to all specialists, much controversy is going on about its mathematical definition. The main purpose of this paper is to outline some of the important characteristics of stiff problems and to present the explicit one-step algorithm suggested by Fatunla [
The Explicit Fatunla’s method (EFM) has been used by Frapiccini et al. [
The rest of this paper is organized as follows. In Section 2, we concentrate on some basic properties of stiff differential equations. Section 3 contains a brief introduction to the EFM, which has been shown to be available for solving stiff ODEs [
As stated above, stiff differential equations appeared in the early 1950s as a result of some pioneering work by Curtiss and Hirschfelder [
Stiffness is a subtle, difficult, and important concept in the numerical solution of ODEs. It depends on the differential equation, the initial conditions, and the numerical methods [
The differential equations we consider in this work are of the form
d y / d x = f ( x , y ( x ) ) (1)
where x is the independent variable which often plays the role of time (t), and y ( x ) is an unknown function that is being sought. The given function f ( x , y ( x ) ) of two variables defines the differential equation. This equation is called a first-order differential equation because it contains a first-order derivative. Sometimes, for convenience, we will omit the x argument in y ( x ) .
Let us indicate that the general solution of the first-order Equation (1) normally depends on an arbitrary integration constant. To single out a particular solution, we need to specify an additional condition. Usually such a condition is taken to be of the form
y ( x 0 ) = y 0 . (2)
The differential Equation (1) and the initial value condition (2) together form an initial value problem (IVP):
d y / d x = f ( x , y ( x ) ) , y ( x 0 ) = y 0 . (3)
We have to emphasize that many physical applications lead to higher-order systems of ODEs, but there is a simple reformulation that can convert them into equivalent first order systems [
Examining the stability question for the general problem (3) is too complicated. Instead, we examine the stability of numerical methods for the model problem
d y / d x = λ y ( x ) , y ( x 0 ) = y 0 (4)
whose analytical solution is
y ( x ) = y 0 exp ( λ ( x − x 0 ) ) (5)
where λ is a constant. The equation
d y / d x = λ y ( x ) (6)
is called the “Dahlquist test equation” [
d y d x ≈ f ( x 0 , y 0 ) + ∂ f ∂ x | ( x = x 0 , y = y 0 ) ( x − x 0 ) + ∂ f ∂ y | ( x = x 0 , y = y 0 ) ( y ( x ) − y 0 ) (7)
= λ ( y ( x ) − y 0 ) + g ( x ) (8)
with λ = ∂ f / ∂ x | ( x = x 0 , y = y 0 ) and g ( x ) = f ( x 0 , y 0 ) + ∂ f / ∂ x | ( x = x 0 , y = y 0 ) ( x − x 0 ) .
This is a valid approximation if | x − x 0 | is sufficiently small, i.e., x ∈ [ x 0 − h , x 0 + h ] where h is a small positive real number. Introducing V ( x ) = y ( x ) − y 0 , we obtain
d V / d x ≈ λ V ( x ) + g ( x ) . (9)
The inhomogeneous term g ( x ) will drop out of all derivations concerning numerical stability, because we are concerned with differences of solutions of the equation. Dropping g ( x ) in Equation (9), we obtain the model Equation (6).
If we are dealing with a set of m equations, i.e.
d y / d x = f ( x , y ) , (10)
where y ( x ) = ( y 1 ( x ) , ⋯ , y m ( x ) ) T is a vector of m components, and f ( x , y ) = ( f 1 ( x , y 1 , ⋯ , y m ) , ⋯ , f m ( x , y 1 , ⋯ , y m ) ) T is a vector-valued function of m + 1 variables, the partial derivative ∂ f / ∂ y intervening in Equation (7) becomes a matrix called the Jacobian of f . Thus the model equation becomes
d y / d x = J y (11)
where J is the Jacobian of f and can be regarded as a constant matrix. For a linear system of differential equations, d y / d x = A ( x ) y , determined by the matrix A ( x ) , the Jacobian J is precisely the matrix A ( x ) . The differential system (11) reduces to a set of m equations like (6), i.e.
d Z i d x = λ i Z i , 1 ≤ i ≤ m (12)
with λ 1 , ⋯ , λ m the eigenvalues of J and Z i , 1 ≤ i ≤ m , the components of the vector Z defined by
Z = R y ( x ) (13)
where R = ( R 1 , ⋯ , R m ) is a matrix of right eigenvectors of the Jacobian, which means that J R i = λ i R i , i = 1 , 2 , ⋯ , m .
We now study the behavior of the solution computed by the explicit Euler method (EEM) [
y n + 1 = y n + λ h y n . (14)
The ratio of the computed solutions at x n + 1 and x n is given by
y n + 1 y n = 1 + λ h . (15)
We compare this with the ratio of the true solutions at the same points, which is
y ( x n + 1 ) y ( x n ) = exp ( λ x n + 1 ) / exp ( λ x n ) = exp ( λ h ) . (16)
It is clear that when λ is real, 1 + λ h is a reasonable approximation to exp ( λ h ) except if λ h < − 2 . For large negative λ h , exp ( λ h ) is much smaller than 1, while 1 + λ h is (in magnitude) greater than 1. This means that the numerical solution is growing while the true solution is decaying. We say that the numerical solution produced by the EEM for λ h < − 2 is unstable.
Another way of looking at stability is to look at the propagation of local errors. For the EEM we have [
e n + 1 = ( 1 + h ∂ f ∂ y ) e n = ( 1 + h λ ) e n , (17)
where e n = y ( x n ) − y n . If ∂ f / ∂ y = λ , we say that the errors are growing if | 1 + λ h | > 1 . In other words, the errors grow if λ h < − 2 and, for such step sizes, the method is called unstable.
To sum up, the requirement | y n | → 0 as n → ∞ , which, for λ < 0 , mirrors the asymptotic behavior of the analytical solution (5), implies that the inequality
0 < h < − 2 λ (18)
must be satisfied. If λ ≪ 0 , then the step size of integration is severely restricted by Equation (18) even though the true solution y 0 exp ( λ ( x − x 0 ) ) becomes negligible very quickly.
Before finishing this subsection, let us define some stability concepts for one-step numerical methods. For this purpose, we consider the model problem (4) where λ is a constant (possibly complex) and x 0 = 0 . If we assume that y ( 0 ) = η , the exact solution of this problem is
y ( x ) = η exp ( λ x ) . (19)
It is clear that lim x → ∞ y ( x ) = 0 if and only if Re ( λ ) < 0 . We have therefore to look for the conditions that have to be imposed on the above mentioned numerical methods in order that the numerical solution y n = y ( n h ) → 0 as n → ∞ where h is the step size of integration. By applying any one-step numerical method to the model problem in question, we obtain [
y n + 1 = R ( z ) y n (20)
where z = λ h and R ( z ) is the so-called stability function. In order that y n tends to zero as n → ∞ , we must impose Re ( λ h ) < 1 thereby implying some constraints on the step size h. The set
S α = { z = λ h ∈ ℂ ; | π − arg ( z ) | < α } . (21)
This definition calls for a numerical method to have a stability region which is unbounded but which does not include the whole complex left hand half-plane. One of the reasons why this is such a useful definition is that many problems have eigenvalues of the Jacobian that lie in a sector S α .
L-stable methods are the most stable ones [
We now briefly discuss stiffness and the difficulties involved in solving stiff equations. As discussed earlier, a system of ODEs of the form d y / d x = f ( x , y ) may be approximated by d y / d x = J y over a small interval of the independent variable x. If J is diagonalizable, this new system may be transformed into
d Z / d x = D Z , (22)
where D is a diagonal matrix with eigenvalues λ i of J (possibly complex) on its diagonal, and Z is related to y by Equation (13).
Many differential equation systems of practical importance in scientific modeling exhibit a distressing behavior when solved by classical numerical methods. This behavior is distressing because these systems, classified as stiff, are characterized by very high instability when approximated by standard numerical methods.
An intuitive idea of what stiff equations are is that they are problems with some smooth and some transient solutions, where all solutions reach the smooth one after a short time (after the transient phase has finished) [
Since stiffness is closely related to the behavior of perturbations to a given solution, it is important to study the effect of a small perturbation ε Y ( x ) to a solution y ( x ) . The parameter ε is small, in the sense that we are interested only in asymptotic behavior of the perturbed solution as this quantity approaches zero. If y ( x ) is replaced by y ( x ) + ε Y ( x ) in the differential equation and the solution expanded in a series in power of ε , with ε 2 and higher powers replaced by zero, we obtain the system
y ′ ( x ) + ε Y ′ ( x ) = f ( x , y ( x ) ) + ε J Y ( x ) (23)
where J ( x ) is the Jacobian of f , y ′ ( x ) = d y / d x and Y ′ ( x ) = d Y ′ / d x . Subtracting Equation (10) from Equation (23), we obtain the equation
Y ′ = J ( x ) Y ′ ( x ) (24)
which controls the behavior of the perturbation. The Jacobian matrix J ( x ) has a crucial role in the understanding of stiff problems. In fact, its spectrum is sometimes used to characterize stiffness [
y ′ ( x ) = f ( x , y ) , y ( x 0 ) = y 0 (25)
is considered to be stiff oscillatory over the interval S if for every x ∈ S the eigenvalues { λ j = u j + i v j , j = 1 , 2 , ⋯ , m } of the Jacobian J = ( ∂ f / ∂ y ) possess the following properties:
u j < 0 , j = 1 , 2 , ⋯ , m , (26)
max 1 ≤ j ≤ m | u j | ≫ min 1 ≤ j ≤ m | u j | (27)
or if the stiffness ratio S satisfies
S = max j , k | u j u k | ≫ 1 (28)
and | u j | ≪ | v j | for at least one pair of j in 1 ≤ j ≤ m .
If an explicit method like the Euler’s one is used to solve the differential system y ′ ( x ) = f ( x , y ( x ) ) when J has a real eigenvalue λ ≪ 0 , the step size of integration is controlled by a transient solution (which quickly becomes negligible), whereas outside the transient phase we wish the step size to be controlled by accuracy alone. This suggests a rather more pragmatic definition of stiffness, where the definition is not based on an analysis of the actual differential equation to be solved but is instead based on the relative performance of implicit and explicit integration methods. Perhaps the best and the oldest definition of this type is due to Curtiss and Hirschfelder [
The same situation can occur for complex eigenvalues with negative real parts. The problem is that the EEM is not A-stable or even A ( α ) -stable for any α < π / 2 . The same is true for all explicit methods like Euler’s one: no such explicit method can be A ( α ) -stable. We are therefore forced to use implicit methods, like the backward Euler method, to solve stiff systems. These methods require more work per step than explicit methods, however, since a system of nonlinear algebraic equations must be solved at each step. For further details of stiff problems, the reader is referred to Shampine and Gear [
Before finishing this section, let us introduce a definition of stiffness which involves a certain norm that depends on the differential system to be solved. If J ( x , y ) , the Jacobian matrix of the system, is diagonalizable, this definition is as follows. A system of ODEs is stiff if its stiff indicator, defined by
σ ( J ( x , y ) ) = m [ J ( x , y ) ] + M [ J ( x , y ) ] 2 (29)
is “large” and negative. Here, m [ J ] and M [ J ] are the greatest lower bounds (glb) and the least upper bound (lub) logarithmic Lipschitz constants, respectively [
The stiffness indicator is easily computed. Let λ [ A ] denote the eigenvalues of a matrix A . Then, for the Euclidian norm, it holds that
M 2 [ A ] = max λ [ H e ( A ) ] ; m 2 [ A ] = min λ [ H e ( A ) ] (30)
where H e ( A ) = ( A + A T ) / 2 denotes the hermitian part of the matrix A . Other norms can also be used, provided that m [ J ] and M [ J ] are computed using well-known expressions.
The idea behind the EFM is to take into account the stiffness of the differential system by introducing the stiffness parameters in interpolating functions that approximate the solution. This allows one to deal with differential systems where the Jacobian matrix displays eigenvalues (possibly complex with large negative real part) that differ by many orders of magnitude. That explains why Fatunla’s method has the capacity to solve stiff equations.
In his paper [
y ′ = f ( x , y ( x ) ) , y ( 0 ) = y 0 , (31)
with y ( x ) ∈ ℝ m in the finite interval S = [ 0 , x f ] ∈ ℝ , where x f = N h for some positive integer N > 0 . It is assumed that y ( x ) is sufficiently differentiable. Throughout this section, we use the same vector notation as in reference [
According to the EFM, the theoretical solution y ( x ) is, on every subinterval [ x n , x n + h ] , approximated by the interpolating function
F ˜ ( x ) = ( I − exp ( Ω 1 x ) ) a + ( I − exp ( − Ω 2 x ) ) b + c , (32)
a , b and c being m-tuples with real entries, I is the identity matrix, whilst Ω 1 and Ω 2 are diagonal matrices, usually called the stiffness matrices; or
F ˜ ˜ ( x ) = ( I − exp ( Ω 1 x ) ) a + ( I − exp ( − Ω 1 * x ) ) a * + b , (33)
where a and b are m-tuples with complex entries, and (*) denotes complex conjugate. The choice of interpolation is determined by Equation (39).
By demanding that the interpolating function (32) coincides with the theoretical solution at the endpoints of the interval [ x n , x n + 1 ] , the following recursion formula is readily obtainable [
y n + 1 = y n + R f n + S f n ( 1 ) (34)
where we use the notations f n = f ( x n , y n ) , f n ( 1 ) = d f ( x , y ) / d x | x = x n . R and S represent diagonal matrices defined by
R = Ω 1 Φ − Ω 1 Ξ , S = Φ + Ξ (35)
where Φ and Ξ are diagonal matrices with non zero entries in the main diagonal given by
Φ j = exp ( Ω j 1 h ) − 1 Ω j 1 ( Ω j 1 + Ω j 2 ) , Ξ j = exp ( − Ω j 2 h ) − 1 Ω j 2 ( Ω j 1 + Ω j 2 ) (36)
The components of the stiffness matrices are given by [
Ω j 1 = 1 2 ( − D j + D j 2 + 4 E j ) , Ω j 2 = Ω j 1 + D j , (37)
where D j and E j , ( j = 1 , ⋯ , m ) are expressed in terms of the components of the derivatives f n ( k ) , k = 0 , 1 , 2 , 3 :
D j = f j n ( 0 ) f j n ( 3 ) − f j n ( 1 ) f j n ( 2 ) f j n ( 1 ) f j n ( 1 ) − f j n ( 0 ) f j n ( 2 ) , E j = f j n ( 1 ) f j n ( 3 ) − f j n ( 2 ) f j n ( 2 ) f j n ( 1 ) f j n ( 1 ) − f j n ( 0 ) f j n ( 2 ) (38)
provided that the denominator in Equation (38) is not zero.
The complex interpolation formula (see Equation (33)) is adopted if in Equation (37), the following relationship holds:
( D j ) 2 < − 4 E j . (39)
The components of the stiffness matrices, which, in this case, are Ω 1 and Ω 1 * , can be written as
Ω j 1 = λ j + i u j , Ω j 2 = Ω j 1 * = λ j − i u j , (40)
for some real numbers λ j and u j . By imposing the same restrictions on (33) as (32), we still obtain the interpolation formula (34) but now with components R j and S j of R and S , respectively, given by [
R j ( λ j , u j ) = − e λ j h [ ( λ j 2 − u j 2 ) sin ( h u j ) − 2 λ j u j cos ( h u j ) ] − 2 λ j u j u j ( λ j 2 + u j 2 ) , (41)
S j ( λ j , u j ) = e λ j h [ λ j sin ( h u j ) − u j cos ( h u j ) ] + u j u j ( λ j 2 + u j 2 ) . (42)
Fatunla [
T j n + 1 = h 5 5 ! [ f j n ( 4 ) + ( Ω j 2 3 − Ω j 2 2 Ω j 1 + Ω j 2 Ω j 1 2 − Ω j 1 3 ) f j n ( 1 ) − Ω j 1 Ω j 2 ( Ω j 1 2 − Ω j 1 Ω j 2 + Ω j 2 2 ) f n ( 0 ) ] + O ( h 6 ) (43)
for the real interpolation formula and by [
T j n + 1 = h 5 5 [ f j n ( 4 ) + ( λ j 2 + u j 2 ) ( 3 λ j 2 − u j 2 ) f j n + 4 λ j ( u j 2 λ j 2 ) f j n ( 1 ) ] + O ( h 6 ) (44)
for the complex interpolation formula.
As stated earlier, the RP consists of a system of three non-linear ODEs, i.e.
{ y ˙ 1 = − k 1 y 1 + k 3 y 2 y 3 y ˙ 2 = k 1 y 1 − k 2 y 2 2 − k 3 y 2 y 3 , y ˙ 3 = k 2 y 2 2 (45)
modeling the kinetics of three species, with y ( 0 ) = y 0 = ( 1 0 0 ) T and parameters k 1 = 0.04 , k 2 = 3 × 10 7 and k 3 = 10 4 . y 1 , y 2 and y 3 denote the concentrations of the three species. The RP is well-known to be stiff [
J ( y ( t ) ) = [ − k 1 k 3 y 3 k 3 y 2 k 1 − 2 k 2 y 2 − k 1 y 3 − k 3 y 2 0 2 k 2 y 2 0 ] . (46)
Its Hermitian part is
H e ( J ( y ( t ) ) ) = [ − k 1 1 2 ( k 1 + k 3 y 3 ) 1 2 k 3 y 2 1 2 ( k 1 + k 3 y 3 ) − 2 k 2 y 2 − k 3 y 3 1 2 ( 2 k 2 − k 3 ) y 2 1 2 k 3 y 2 1 2 ( 2 k 2 − k 3 ) y 2 0 ] (47)
This one has certainly three real eigenvalues which are solution of the cubic equation
a λ 3 + b λ 2 + c λ + d = 0 (48)
where a = 1 , b = 2 k 2 y 2 + k 3 y 3 + k 1 , c = − [ ( 1 4 k 1 − 2 k 2 y 2 − 1 2 k 3 y 3 ) k 1 + ( − k 2 k 3 + k 2 2 + 1 2 k 3 2 ) y 2 2 + 1 2 k 3 2 y 3 2 ] and d = − ( − 1 2 k 1 k 3 + 1 2 k 3 2 y 3 + k 1 k 2 + 1 2 k 3 2 y 2 ) k 2 y 2 2 .
Using the Cardan method for cubic equations in one variable, we obtain
λ 1 = μ 0 + μ 0 ¯ − b 3 a , λ 2 = j μ 0 + j μ 0 ¯ − b 3 a , λ 3 = j 2 μ 0 + j 2 μ 0 ¯ − b 3 a (49)
with μ 0 3 = ( − q + i Δ / 27 ) / 2 , j = exp ( 2 i π / 3 ) = − 1 / 2 + i 3 / 2 and Δ = − ( 3 p 3 + 27 q 2 ) > 0 , where p and q are defined by p = ( 3 a c − b 2 ) / 3 a 3 , q = ( 2 b 3 − 9 a b c + 27 a 2 d ) / 27 . We can therefore study the stiffness of the RP by use of the stiffness indicator (see Equation (29))
σ 2 [ J ( t ) ] = ( m 2 [ J ( t ) ] + M 2 [ J ( t ) ] ) / 2 (50)
where M 2 [ J ( t ) ] and m 2 [ J ( t ) ] are, respectively, the largest and smallest eigenvalues of H e ( J ( t ) ) .
The components of the solution obtained by means of the EFM is shown in
We want to emphasize that implementation of the EFM to solve the RP requires the calculation of the function f and its four first derivatives, i.e. f ( 1 ) , f ( 2 ) , f ( 3 ) and f ( 4 ) at each value t n . The function f is here given by
f ( y ( t ) ) = ( − k 1 y 1 + k 3 y 2 y 3 k 1 y 1 − k 2 y 2 2 − k 3 y 2 y 3 k 2 y 2 2 ) . (51)
We have used maple 18 software to establish analytical expression of f ( 1 ) , f ( 2 ) , f ( 3 ) and f ( 4 ) .
As expected, we remark, from the plot of the stiffness indicator, that the RP is very stiff. We also find that our results agree noticeably with those computed by the RADAU solver except for the solution component y 2 for small values of the time. We think that the observed discrepancy between our results and the
RADAU ones for t at around 10−4 atomic units (a.u.) is due to the fact that during the implementation of the EFM, we have chosen, for t ≤ 0.1 a.u., a very small truncation error (10−12 in magnitude) to control the integration step size, which means that our results are probably more accurate. In the case of t > 0.1 a.u., the time step has been adapted according to the condition | T n + 1 | ≤ 10 − 9 . It is because we have used two distinct conditions on the truncation error for the control of the integration step size that we have a discontinuity at t = 0.1 a.u. in the graph of the step size.
The object of this paper has been to present some basic characteristics of stiff differential equations, as well as to introduce the EFM which has been shown to be available for solving stiff problems. We have shown that the stiffness indicator of the Jacobian matrix J gives a sufficient information to estimate the computational costs of explicit schemes like the Euler’s one. Numerical results obtained solving the Robertson problem using the EFM on the one hand and the solver RADAU on the other hand, confirm the fact that the explicit method in question can be a good candidate to solve stiff ODEs.
We have to emphasize that modern codes for solving ODEs automatically vary the step size, estimate the local error, and provide facilities to compute the solution at intermediate points via interpolation [
The authors declare no conflicts of interest regarding the publication of this paper.
Nyengeri, H., Ndenzako, E. and Nizigiyimana, R. (2019) The Explicit Fatunla’s Method for First-Order Stiff Systems of Scalar Ordinary Differential Equations: Application to Robertson Problem. Open Access Library Journal, 6: e5291. https://doi.org/10.4236/oalib.1105291