Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N): II. Paradigm Application to a Bernoulli Model Comprising Uncertain Parameters ()
1. Introduction
This work presents the application of the recently developed [1] “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N)” to a simplified Bernoulli model [2] comprising a nonlinear model response, uncertain model parameters, uncertain model domain boundaries and uncertain model boundary conditions. As is well known [2], the Bernoulli nonlinear differential equation admits exact solutions. The demonstration model selected for illustrating the application of the 5th-CASAM-N is a special case of the Bernoulli equation with quadratic nonlinearity, which is also called [2] the logistic differential equation and is used in economics.
This work is structured as follows: the illustrative Bernoulli model, including the paradigm nonlinear model response, is detailed in Section 2, which also presents the application of the 1st-CASAM-N illustrating the exact and efficient computation of the complete set of first-order response sensitivities with respect to the model parameters. These computations also include the illustrative computation of 1st-order sensitivities with respect to imprecisely known domain boundaries.
Section 3 illustrates the application of the 2nd-CASAM-N to obtain the complete set of 2nd-order sensitivities. It is shown that the complexity of the 2nd-Order Adjoint Sensitivity System used for computing efficiently and exactly the 2nd-order sensitivities is determined by the complexity of the 1st-order sensitivity used as the respective response and, hence, as the respective starting point. Thus, the 2nd-LASS that corresponds to 1st-order sensitivities that involve solely the original state function comprises only as many equations as the corresponding 1st-LASS. On the other hand, if the 1st-order sensitivity under consideration involves both the original state function(s) and the 1st-level adjoint sensitivity function(s), then the 2nd-LASS could comprise twice as many equations as the 1st-LASS. The symmetries inherent to the mixed 2nd-order sensitivities make it possible to choose a priori, based on the expressions of the 1st-order sensitivities, the order of priority and the most advantageous path for computing the 2nd-order sensitivities. The primary consideration when computing 2nd-order sensitivities is the priority order indicated by the magnitudes of the relative 1st-order sensitivities: the 2nd-order sensitivities stemming from the largest 1st-order relative sensitivity should be computed first. Once the priorities for computing the 2nd-order sensitivities have been established, it is important to examine the expressions of the 1st-order sensitivities in order to establish the least expensive (computationally) path for computing the mixed 2nd-order sensitivities.
Section 4 illustrates the application of the 3rd-CASAM-N to obtain representative 3rd-order sensitivities. It is shown that 2nd-order sensitivities that depend solely on the original function will require the solution of a 3rd-Level Adjoint Sensitivity System (3rd-LASS) of the same size as the original system or, equivalently, the 1st-LASS. The 2nd-order sensitivities that depend solely on the original state function and the 1st-level adjoint sensitivity system will give rise to a 3rd-LASS of the same size as the corresponding 2nd-LASS. Finally, 2nd-order sensitivities that depend on all of the components of the 2nd-level adjoint function(s) will need the solution of a 3rd-LASS that will have twice the dimensions of the corresponding 2nd-LASS.
Section 5 discusses the important aspects of applying the 4th-CASAM-N to compute the 4th-order sensitivities and the 5th-CASAM-N to compute the fifth-order sensitivities. Concluding remarks are presented in Section 6.
2. 1st-CASAM-N: Computation of First-Order Response Sensitivities
The illustrative model considered in this work comprises a simplified second-order Bernoulli equation subject to an imprecisely known boundary/initial condition
at the imprecisely known location
, having the following standard form:
(1)
. (2)
The model’s response, denoted as
, is considered to be a nonlinear functional of the state function and parameters and is defined as follows:
. (3)
The scalar parameters q,
,
,
, and r, which appear in Equations (1)-(3), are considered to be imprecisely known, subject to uncertainties. These parameters are representative of the type of parameters that can appear in the mathematical model of a physical, as follows: 1) the parameter q typifies uncertain model parameters which appear in the equations underlying the model; 2) the parameter
typifies uncertain boundary or initial conditions; 3) the parameters
and
typify uncertain boundaries of the domain of definition of the independent variable(s); 4) the parameter r typifies uncertain quantities which may appear solely in the definition of the model’s response.
For notational convenience, these imprecisely known model parameters are considered to be components of a vector of parameters
defined as follows:
, (4)
where the subscript
denotes the “total number of model parameters.” The dagger superscript “
” will be used in this work to denote “transposition.” The information customarily available about the model parameters comprises their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are usually determined from evaluation processes external to the physical system under consideration. Occasionally, only lower and upper bounds may be known for some model parameters. The nominal parameter values will be denoted as
; the superscript “0” will be used throughout this work to denote “nominal values.”
The solution of Equations (1) and (2) is obtained by separation of variables and subsequent integration to obtain the following expression:
. (5)
Inserting the result obtained in Equation (5) into Equation (3) yields the following expression for the model response:
. (6)
The parameters q,
and
occur in the expression of the state function
, which is the solution of Equations (1)-(3), but the parameter
occurs only in the expression of the response, thus illustrating the fact that model parameters may be introduced in the model solely through the definition of the model’s response. Although both the model and its response are nonlinear functions of the state variables, the model has been chosen to be sufficiently simple to admit readily differentiable functions of the model parameters, so that the algebraic manipulations would not distract from following the application of the principles underlying the 5th-CASAM-N to obtain the various sensitivities (up to fifth-order) while enabling the analytical verification of the thus expressions obtained.
The nominal (or mean) parameter vales
will differ from their true, but unknown, values by quantities denoted as
, where
. Since the forward state function
is related to the model and boundary parameters
through Equations and , it follows that the variations
in the model and boundary parameters will cause corresponding variations
around the nominal solution
in the forward state functions. In turn, the variations
and
will induce variations in the model’s response.
The 1st-order sensitivities of a model response
are obtained by determining the 1st-order Gateaux- (G-) variation
of the response, which is given, by definition, by the following expression:
(7)
where
(8)
. (9)
The direct-effect term can be computed once the nominal values
are available. The notation
will be used in this work to indicate that the quantity enclosed within the bracket is to be evaluated at the nominal values of the respective parameters and state functions. On the other hand, the indirect-effect term can be quantified only after having determined the variations
in terms of the variations
. The first-order relationship between the vectors
and
is determined by solving the following 1st-Level Variational Sensitivity System (1st-LVSS) obtained by applying the definition of the G-differential to Equations (1) and (2), which yields the following equations:
(10)
. (11)
The alternative to solving repeatedly the 1st-LVSS to obtain the 1st-level variational function
, which depends on the various parameter variations, is to express the indirect-effect term defined in Equation (9) in terms of the solution of the 1st-Level Adjoint Sensitivity System (1st-LASS), which is constructed by applying the principles of the 5th-CASAM-N, as follows:
1) Consider that the functions
and
are elements of a Hilbert space denoted as
which is endowed with an inner product of two vectors
and
denoted as
and defined as follows:
. (12)
2) Using the definition of provided in Equation (12), construct the inner product of Equation (10) with a yet undefined function
to obtain the following relation:
. (13)
3) Integrate by parts the left-side of Equation (13) to obtain the following relation:
(14)
4) Use in Equation (14) the boundary condition given in Equation (11) to obtain the following relation:
(15)
5) Require the left-side of Equation (15) to represent the indirect-effect term defined in Equation (9) and eliminate the unknown value
in Equation (15) by requiring the function
to be the solution of the following 1st-Level Adjoint Sensitivity System (1st-LASS):
(16)
. (17)
6) Use Equations (15)-(17) together with Equation (13) in Equation (9) to obtain the following alternative expression for the indirect-effect term:
(18)
7) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (18) and (8), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (7), yields the following expressions for the first-order response sensitivities with respect to the model parameters:
, (19)
, (20)
, (21)
, (22)
. (23)
The expressions of the sensitivities provided in Equations (19)-(23) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 1st-LASS to obtain the 1st-level adjoint sensitivity function
. The 1st-LASS is linear in
and is independent of parameter variations, so it only needs to be solved once. The expression of
obtained by solving the 1st-LASS by the standard integrating-factor method is as follows:
. (24)
Inserting the result obtained in Equation (24) into in Equations (19)-(21) and evaluating the respective expressions yields the following closed-form results for the respective sensitivities:
, (25)
, (26)
. (27)
The closed-form expressions provided in Equations (22), (23), (25)-(27) have been derived for verification purposes, as they can be compared directly with the respective results which would be obtained by differentiating the closed-form expression of model response provided in Equation (6). In practice, however, the model’ equations and the 1st-LASS must be solved numerically. Consequently, the sensitivities expressed by Equations (19)-(23) must be evaluated numerically; closed-from expressions for these sensitivities are not available in practice.
3. 2nd-CASAM-N: Computation of Second-Order Response Sensitivities
Since there are five 1st-order sensitivities, it follows that there will be twenty five 2nd-order sensitivities, of which 15 will be distinct. The 2nd-order sensitivities could be computed directly by differentiating the expression of the G-differential of the response provided in Equation (7), to obtain the expression of the 2nd-order G-differential,
, which would require the computation of the 2nd-order differential
. The second-order G-differential
would need to be determined by solving the G-differential of the 1st-LVSS, which would involve 2nd-order differential equations, which would depend on first- and second-order parameter variations. Furthermore, this set of 2nd-order differential equations would depend on the solution of the 1st-LVSS and would need to be solved at least 25 times, to account for all combinations of 1st- and 2nd-order variations in the parameters and state function
.
Alternatively, the 2nd-order sensitivities can be defined as the “1st-order sensitivities of the 1st-order sensitivities.” This definition stems from the inductive definition of the 2nd-order total G-differential of correspondingly differentiable function, which is also defined inductively as “the total 1st-order differential of the 1st-order total differential.” As a general principle, the 2nd-order sensitivities should be computed in a priority order that should follow the ranking of the 1st-order sensitivities: the 2nd-order sensitivities that correspond to the largest relative 1st-order sensitivity should be computed first, the 2nd-order sensitivities that correspond to the second largest relative 1st-order sensitivity should be computed next, and so on. Based on a user-selected a-apriori cut-off criterion, 2nd-order sensitivities that stem from very small relative 1st-order sensitivities might be neglected without actually computing them.
Another criterion for prioritizing the computation of the 2nd-order sensitivities may be based on the difficulty involved in computing them. Examining Equations (19)-(23), it becomes apparent that the expressions of
and
involve only the state function
. Therefore, the 2nd-level adjoint sensitivity functions which will be used to compute the sensitivities stemming from
and
will comprise a single component, having the general form
,
. Furthermore, the procedure for computing these 2nd-level sensitivities will be the same as the procedure followed for computing the 1st-order sensitivities, as will be shown in subsections 3.1.1 and 3.1.2, respectively.
On the other hand, the expressions of
,
and
involve the adjoint sensitivity function
, which means that the 2nd-level adjoint sensitivity functions that will be needed for computing these sensitivities will comprise two-components, having the general form
,
. Thus, the computation of the 2nd-order sensitivities stemming from
,
and/or
will require at least twice as many computations as are required for the computation of the 2nd-order sensitivities stemming from
and/or
. This is because solving a 2nd-Level Adjoint Sensitivity System to compute a two-component 2nd-level adjoint sensitivity function of the form
will be at least twice as expensive computationally as solving a 2nd-Level Adjoint Sensitivity System that involves a one-component 2nd-level adjoint sensitivity function of the form
. The application of the principles underlying the 5th-CASAM-N to compute the 2nd-order sensitivities stemming from
,
and
will be illustrated in the subsections 3.2.1, 3.2.2 and 3.2.3, respectively.
3.1. Second-Order Sensitivities Stemming from 1st-Order Sensitivities Involving Just the Original State Function
Examining Equations (19)-(23), it becomes apparent that the expressions of
and
involve only the state function
. Therefore, the 2nd-level adjoint sensitivity functions which will be used to compute the sensitivities stemming from
and
will comprise a single component, having the general form
,
. Furthermore, the procedure for computing these 2nd-level sensitivities will be the same as the procedure followed for computing the 1st-order sensitivities, as will be shown in subsections 3.1 and 3.2, respectively.
3.1.1. Second-Order Sensitivities Stemming from
The 2nd-order sensitivities which arise from
are obtained from the G-differential
of
, which is obtained by applying the definition of the G-differential to the expression provided in Equation (23). This yields the following expression:
(28)
where:
, (29)
. (30)
The direct-effect term has been evaluated at this stage since the function
is already available. The indirect-effect term, however, can be evaluated only after having determined the variational function
, which is the solution of the 1st-LVSS. Solving the 1st-LVSS, which depends on parameter variations, can be avoided by expressing the indirect-effect term defined in Equation (30) in terms of the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), which is constructed by applying the same principles as outlined in Section 3.1, as follows:
1) Consider that the functions
and
are elements the Hilbert space denoted as
which is endowed with the inner defined in Equation (12). Using the definition of provided in Equation (12), construct the inner product of Equation (10) with a yet undefined function
to obtain the following relation:
(31)
2) Integrate by parts the left-side of Equation (31) to obtain the following relation:
(32)
3) Use in Equation (32) the boundary condition given in Equation (11) to obtain the following relation:
(33)
4) Require the left-side of Equation (33) to represent the indirect-effect term defined in Equation (30) and eliminate the unknown value
in Equation (33) by requiring the function
to be the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):
(34)
. (35)
5) Use Equations (33)-(35) together with Equation (31) in Equation (30) to obtain the following alternative expression for the indirect-effect term:
.(36)
6) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (36) and (29), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (28), yields the following expressions for the 2nd-order response sensitivities which stem from the first-order sensitivity
:
, (37)
, (38)
, (39)
, (40)
. (41)
The expressions of the sensitivities provided in Equations (37)-(40) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 2nd-LASS using the standard integrating-factor method to obtain the following expression for the 2nd-level adjoint sensitivity function
:
. (42)
Inserting the result obtained in Equation (42) into in Equations (37)-(39) and evaluating the respective expressions yields the following closed-form results for the respective sensitivities:
, (43)
, (44)
. (45)
3.1.2. Second-Order Sensitivities Stemming from
In preparation for determining the 2nd-order sensitivities that correspond to
, the expression provided in Equation (22) is written in the following integral form:
. (46)
The 2nd-order sensitivities which arise from
are obtained by applying the definition of the G-differential to Equation (22), which yields the following expression:
(47)
where:
, (48)
. (49)
The direct-effect term can be evaluated at this stage since the function
is already available. The indirect-effect term, however, can be evaluated only after having determined the variational function
, which is the solution of the 1st-LVSS. Solving the 1st-LVSS, which depends on parameter variations, can be avoided by expressing the indirect-effect term defined in Equation (49) in terms of the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), which is constructed by applying the same principles as outlined in Section 3.1, as follows:
1) Using the definition of provided in Equation (12), construct the inner product of Equation (10) with a yet undefined function
to obtain the following relation:
(50)
2) Integrate by parts the left-side of Equation (50) to obtain the following relation:
(51)
3) Use in Equation (51) the boundary condition given in Equation (11) to obtain the following relation:
(52)
4) Require the left-side of Equation (52) to represent the indirect-effect term defined in Equation (49) and eliminate the unknown value
in Equation (52) by requiring the function
to be the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):
(53)
. (54)
5) Use Equations (52)-(54) together with Equation (50) in Equation (49) to obtain the following alternative expression for the indirect-effect term:
. (55)
6) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (55) and (48), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (47), yields the following expressions for the 2nd-order response sensitivities which stem from the first-order sensitivity
:
, (56)
, (57)
, (58)
, (59)
. (60)
The expressions of the sensitivities provided in Equations (37)-(40) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 2nd-LASS using the standard integrating-factor method to obtain the following expression for the 2nd-level adjoint sensitivity function
:
.(61)
Inserting the result obtained in Equation (61) into in Equations (56)-(58) and evaluating the respective expressions yields the following closed-form results for the respective sensitivities:
, (62)
, (63)
. (64)
Evidently, the identity of the expressions obtained in Equations (60) and (40) confirms the correct determination of the mixed 2nd-order sensitivity
.
3.2. Second-Order Sensitivities Stemming from 1st-Order Sensitivities Involving the 1st-Level Adjoint Sensitivity Function
The expressions of
,
and
involve the adjoint sensitivity function
, which means that the 2nd-level adjoint sensitivity functions that will be needed for computing these sensitivities will comprise two-components, having the general form
,
. Thus, the computation of the 2nd-order sensitivities stemming from
,
and/or
will require at least twice as many computations as are required for the computation of the 2nd-order sensitivities stemming from
and/or
. This is because solving a 2nd-Level Adjoint Sensitivity System to compute a two-component 2nd-level adjoint sensitivity function of the form
.
3.2.1. Second-Order Sensitivities Stemming from
In preparation for determining the 2nd-order sensitivities that correspond to
, the expression provided in Equation (20) is written in the following integral form:
. (65)
By definition, the G-differential
of
is obtained as follows:
(66)
where:
(67)
. (68)
The direct-effect term has been evaluated at this stage since the function
is already available. The indirect-effect term, however, can be evaluated only after having determined the variational function
, which is the solution of the 2nd-Level Variational Sensitivity System (2nd-LVSS) obtained by G-differentiating the 1st-LASS. The G-differentiation of the 1st-LASS represented by Equations (16) and (17) yields the following system:
(69)
(70)
Equation (69) also involves the variational function
, which is the solution of the 1st-LVSS comprising Equations (10) and (11). Therefore, Equations (69) and (70) are to be concatenated with the 1st-LVSS to obtain the following 2nd-Level Variational Sensitivity System (2nd-LVSS) which is satisfied by the 2nd-level variational function
:
(71)
(72)
where
(73)
(74)
. (75)
The need for solving repeatedly the 2nd-LVSS to obtain the 2nd-level variational function
for every parameter variations is circumvented by expressing the indirect-effect term defined in Equation (68) in terms of the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), which is constructed specifically for the indirect-effect term defined in Equation (68), by applying the principles of the 5th-CASAM-N, as follows:
1) Consider that the function
is an element of a Hilbert space denoted as
. This Hilbert space is considered to be endowed with an inner product of two vectors
and
defined as follows:
(76)
2) Using the definition of provided in Equation (76), construct the inner product of Equation (71) with a yet undefined function
to obtain the following relation:
(77)
which in component form reads as follows:
(78)
3) Integrate by parts the left-side of Equation (78) to obtain the following relation:
(79)
4) Use in Equation (79) the boundary condition given in Equations (11) and (70) to obtain the following relation:
(80)
5) Require the right-side of Equation (80) to represent the indirect-effect term defined in Equation (68) and eliminate the unknown values of the components of
in Equation (80) by requiring the function
to be the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):
(81)
(82)
(83)
. (84)
6) Use Equations (80)-(84) together with Equation (78) in Equation (68) to obtain the following alternative expression for the indirect-effect term:
(85)
7) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (85) and (67), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (66), yields the following expressions for the first-order response sensitivities with respect to the model parameters:
, (86)
, (87)
, (88)
, (89)
. (90)
The expressions of the sensitivities provided in Equations (86)-(90) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 2nd-LASS to obtain the 2nd-level adjoint sensitivity function
. The 2nd-LASS is linear in
and is independent of parameter variations, so it only needs to be solved once. Furthermore, the 2nd-LASS is an upper-triangular system which can be solved in a decoupled manner, by first obtaining the expression of the function
and subsequently obtaining the expression of the function
. Solving the 2nd-LASS comprising Equations (81)-(84) yields the following expressions:
, (91)
. (92)
Inserting into Equations (86)-(90) the expressions obtained in Equations (91) and (92) yields the following closed form expressions:
, (93)
, (94)
, (95)
, (96)
. (97)
Since the expression in Equation (90) must be identical to the expression provided in Equation (38), i.e.,
, (98)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
and
.
Similarly, since the expression in Equation (89) must be identical to the expression provided in Equation (57), i.e.,
, (99)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
and
.
3.2.2. Second-Order Sensitivities Stemming from
In preparation for determining the 2nd-order sensitivities that correspond to
, the expression provided in Equation (21) is written in the following integral form:
. (100)
By definition, the G-differential
of
is obtained as follows:
(101)
where:
(102)
. (103)
The direct-effect term has been evaluated at this stage since the functions
and
are already available. The indirect-effect term, however, can be evaluated only after having determined the variational vector
, which is the solution of the 2nd-LVSS obtained in Equations (71)-(75). The need for solving repeatedly the 2nd-LVSS to obtain the 2nd-level variational function
for every parameter variations is circumvented by expressing the indirect-effect term defined in Equation (103) in terms of the solution of a 2nd-LASS, which is constructed specifically for this indirect-effect term, by applying the principles of the 5th-CASAM-N, as follows:
1) Using the definition of provided in Equation (76), construct in the Hilbert
the inner product of Equation (71) with a yet undefined function
to obtain the following relation:
(104)
which in component form reads as follows:
(105)
2) Integrate by parts the left-side of Equation (105) to obtain the following relation:
(106)
3) Use in Equation (106) the boundary condition given in Equations (11) and (70) to obtain the following relation:
(107)
4) Require the right-side of Equation (107) to represent the indirect-effect term defined in Equation (103) and eliminate the unknown values of the components of
in Equation (107) by requiring the function
to be the solution of the following 2nd-LASS:
(108)
(109)
(110)
. (111)
5) Use Equations (107)-(111) together with Equation (105) in Equation (103) to obtain the following alternative expression for the indirect-effect term:
(112)
6) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (112) and (102), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (101), yields the following expressions for the first-order response sensitivities with respect to the model parameters:
, (113)
, (114)
, (115)
, (116)
. (117)
The expressions of the sensitivities provided in Equations (113)-(117) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 2nd-LASS defined by Equations (108)-(111) to obtain the 2nd-level adjoint sensitivity function
. This 2nd-LASS is linear in
and is independent of parameter variations, so it only needs to be solved once. Furthermore, this 2nd-LASS is an upper-triangular system which can be solved in a decoupled manner, by first obtaining the expression of the function
and subsequently obtaining the expression of the function
. Solving thus the 2nd-LASS comprising Equations (108)-(111) yields the following expressions:
,(118)
. (119)
Inserting into Equations (113)-(117) the expressions obtained in Equations (118) and (119) yields the following closed form expressions:
, (120)
, (121)
, (122)
, (123)
. (124)
Since the expression in Equation (117) must be identical to the expression provided in Equation (39), i.e.,
, (125)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
and
.
Similarly, since the expression in Equation (116) must be identical to the expression provided in Equation (58), i.e.,
, (126)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
and
.
Furthermore, since the expression in Equation (114) must be identical to the expression provided in Equation (88), i.e.,
(127)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
and
.
3.2.3. Second-Order Sensitivities Stemming from
The 2nd-order sensitivities stemming from the 1st-order sensitivity
are obtained, by definition, from by determining the G-differential,
, of the expression provided in Equation (19) for
, which yields:
(128)
where:
, (129)
. (130)
The direct-effect term can be evaluated at this stage since the values of the functions
and
are already available. The indirect-effect term, however, can be evaluated only after having determined the vector-valued variational function
, which is the solution of the 2nd-LVSS obtained in Equations (71)-(75). The need for solving repeatedly the 2nd-LVSS to obtain the 2nd-level variational function
for every parameter variations is circumvented by expressing the indirect-effect term defined in Equation (130) in terms of the solution of a 2nd-LASS, which is constructed specifically for this indirect-effect term, by applying the principles of the 5th-CASAM-N, as follows:
1) Using the definition of provided in Equation (76), construct in the Hilbert
the inner product of Equation (71) with a yet undefined function
to obtain the following relation:
(131)
which in component form reads as follows:
(132)
2) Integrate by parts the left-side of Equation (132) to obtain the following relation:
(133)
3) Use in Equation (133) the boundary condition given in Equations (11) and (70) to obtain the following relation:
(134)
4) Require the right-side of Equation (134) to represent the indirect-effect term defined in Equation (130) and eliminate the unknown values of the components of
in Equation (134) by requiring the function
to be the solution of the following 2nd-Level Adjoint Sensitivity System (2nd-LASS):
(135)
(136)
(137)
. (138)
5) Use Equations (134)-(138) together with Equation (132) in Equation (130) to obtain the following alternative expression for the indirect-effect term:
(139)
6) Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (139) and (129), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (7), yields the following expressions for the first-order response sensitivities with respect to the model parameters:
, (140)
, (141)
, (142)
, (143)
. (144)
The expressions of the sensitivities provided in Equations (140)-(144) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. The expressions of the sensitivities stemming from the indirect-effect term can be evaluated after solving the 2nd-LASS to obtain the 2nd-level adjoint sensitivity function
, which yields:
, (145)
. (146)
Inserting the results obtained in Equations (145) and (146) into in Equations (140)-(144) and evaluating the respective expressions yields the following closed-form results for the respective sensitivities:
, (147)
, (148)
, (149)
, (150)
. (151)
Since the expression in Equation (144) must be identical to the expression provided in Equation (37), i.e.,
, (152)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
and
.
Similarly, since the expression in Equation (143) must be identical to the expression provided in Equation (56), i.e.,
, (153)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
and
. Furthermore, since the expression in Equation (142) must be identical to the expression provided in Equation (113), i.e.,
(154)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
,
,
and
.
Finally, since the expression in Equation (141) must be identical to the expression provided in Equation (86), i.e.,
(155)
it follows that the above identity provides a stringent test in practice for verifying the accuracy of the numerical computation of the functions
,
,
,
,
.
3.3. Remarks on the Computation of Second-Order Sensitivities
Each of the 1st-order sensitivities give rise to as many 2nd-order sensitivities as there are model parameters:
denotes the “total number of model parameters.” Each of the 1st-order sensitivity is considered to be a model response for constructing a 2nd-LASS which is independent of parameter variations and therefore needs to be solved just once in order to obtain all (i.e., TP) of the 2nd-order sensitivities that stem from the specific 1st-order sensitivity considered as a model response. The 2nd-LASS may comprise just as many equations as the 1st-LASS, in which case the computational effort required for solving the 2nd-LASS is comparable to that for solving the 1st-LASS. This was the case for determining the 2nd-order sensitivities stemming from the 1st-order sensitivities
and
. On the other hand, the 2nd-LASS could comprise twice as many equations as the 1st-LASS, as was the case for determining the 2nd-order sensitivities stemming from the 1st-order sensitivities
,
and
. In such cases, solving the 2nd-LASS would be twice as expensive computationally as solving the 1st-LASS, and requires the prior availability of the state function
, which is obtained by solving the original system of nonlinear equations that underly the model under consideration, and the prior availability of the 1st-level adjoint sensitivity function
, which is obtained by solving the 1st-LASS. The primary consideration when computing 2nd-order sensitivities is the priority order indicated by the magnitudes of the relative 1st-order sensitivities: the 2nd-order sensitivities stemming from the largest 1st-order relative sensitivity should be computed first. Once the priorities for computing the 2nd-order sensitivities have been established, it is important to examine the expressions of the 1st-order sensitivities in order to establish the least expensive (computationally) path for computing the mixed 2nd-order sensitivities. For example, it is more advantageous computationally to compute most advantageous it is more advantageous to compute those stemming from the 1st-order sensitivities as starting points
and
. For example, it is computationally more advantageous to compute
by using Equation (38), which is obtained by using
as the starting point, rather than using Equation (90), which is obtained by using
as the starting point.
4. 3rd-CASAM-N: Computation of Third-Order Response Sensitivities
Each of the 2nd-order sensitivities would give rise to five 3rd-order sensitivities, for a total of 125 third-order sensitivities. The 3rd-order sensitivities could be computed directly by differentiating the expression of the 2nd-order G-differential,
, of the response, to obtain the 3rd-order G-differential which would require the computation of the G-differentials
,
. In this case, the G-differentials
,
would need to be determined by solving nth-LVSS, the differentials
,
, which would involve 3rd-order differential equations, which would depend on 1st-, 2nd- and 3rd-order parameter variations. Furthermore, this set of 3rd-order differential equations would need to be solved at least 125 times, to account for all combinations of 1st- and 2nd-order variations in the parameters and state function
. Alternatively, the 3rd-order sensitivities can be defined as the “1st-order sensitivities of the 2nd-order sensitivities,” which enables the 3rd-order sensitivities to be computed by using 3rd-level adjoint sensitivity functions determined as will be illustrated in the remainder of this Section.
4.1. Third-Order Sensitivities Stemming from 2nd-Order Sensitivities Involving One-Component of the State Functions
Examining the expressions of the 2nd-order sensitivities reveals that the sensitivities
and
depend solely on the original function
. Therefore, the 3rd-order sensitivities stemming from these 2nd-order sensitivities will involve a one-component 3rd-level adjoint sensitivity function, as will be illustrated on this Section by determining the 3rd-order sensitivities arising from
. The expression of
is provided by Equation (40), which is identical to Equation (60). The 3rd-order sensitivities stemming from
are obtained from by G-differentiating Equation (60), which by definition yields the following expression:
(156)
where:
, (157)
. (158)
The direct-effect term defined in Equation (157) has been evaluated at this stage by using the already available value expression of
from Equation (5). On the other hand, the indirect-effect term defined in Equation (158) can be evaluated only after having determined the variational function
, which is the solution of the 1st-LVSS provided in Equations (10) and (11), which is computationally expensive to solve in practice for systems comprising many parameter variations. The alternative to solving repeatedly the 1st-LVSS to obtain the 1st-level variational function
, which depends on the various parameter variations, is to express the indirect-effect term defined in Equation (158) in terms of the solution of a 3rd-Level Adjoint Sensitivity System (3rd-LASS), which is constructed by following the same steps as outlined in subsection 3.1.1. Thus, the definition of provided in Equation (12) is used to construct the inner product of Equation (10) with a yet undefined function
to obtain the following relation:
. (159)
Integrating by parts the left-side of Equation (159) yields the following relation:
(160)
Using in Equation (160) the boundary condition given in Equation (11) yields the following relation:
(161)
The left-side of Equation (161) is required to represent the indirect-effect term defined in Equation (158) and the unknown value
in Equation (160) is eliminated by requiring the function
to be the solution of the following 3rd-Level Adjoint Sensitivity System (3rd-LASS):
(162)
. (163)
Using Equations (162), (163), (161), and (159) in Equation (158) yields the following alternative expression for the indirect-effect term:
(164)
Adding the expressions for the indirect-effect and direct-effect terms obtained in Equations (164) and (157), respectively, and identifying the expressions that multiply the respective parameter variations, as indicated in Equation (156), yields the following expressions for the third-order response sensitivities which stem from
:
, (165)
, (166)
, (167)
, (168)
. (169)
The expressions of the sensitivities provided in Equations (165)-(168) are to be evaluated at the nominal values of the respective parameters and state functions but the respective indication
has been omitted, for simplicity. These expressions can be evaluated after solving the 3rd-LASS to obtain the 3rd-level adjoint sensitivity function
.
Solving 3rd-LASS by the standard integrating-factor method yields the following expression for the 3rd-level adjoint sensitivity function
:
. (170)
Inserting the result obtained in Equation (170) into in Equations (165)-(167) and evaluating the respective expressions yields the following closed-form results for the respective sensitivities:
, (171)
, (172)
. (173)
4.2. Third-Order Sensitivities Stemming from 2nd-Order Sensitivities Involving Two Components of the State Functions
The 2nd-order sensitivities
and
depend only on the functions
and
. Similarly, the 2nd-order sensitivities
and
depend only on the functions
and
. Furthermore, the 2nd-order sensitivities
and
depend only on the functions
and
. Finally, the 2nd-order sensitivities
and
depend only on the functions
and
. Consequently, the 3rd-order sensitivities that stem from these 2nd-order sensitivities can be expressed in terms of a 3rd-level adjoint sensitivity function which will comprise only two components, as will be illustrated by determining the 3rd-order sensitivities that stem from
. These 3rd-order sensitivities are obtained from the G-differential of Equation (117), which has, by definition, the following expression:
(174)
where:
(175)
(176)
The direct-effect term defined in Equation (175) has been evaluated at this stage by using the already available values of the 2nd-level adjoint sensitivity function
, cf. Equation (119), and
, cf. Equation (5). On the other hand, the indirect-effect term defined in Equation (176) can be evaluated only after having determined the variational function
, which is the solution of the G-differentiated system of Equations (110) and (111), which has the following form:
(177)
(178)
Equation (177) also involves the variational function
, which is the solution of the 1st-LVSS comprising Equations (10) and (11). Therefore, Equations (177) and (178