Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N): I. Mathematical Framework

This work presents the mathematical framework of the “Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5th-CASAM-N),” which generalizes and extends all of the previous works performed to date on this subject. The 5th-CASAM-N enables the exact and efficient computation of all sensitivities, up to and including fifth-order, of model responses to uncertain model parameters and uncertain boundaries of the system’s domain of definition, thus enabling, inter alia, the quantification of uncertainties stemming from manufacturing tolerances. The 5th-CASAM-N provides a fundamental step towards overcoming the curse of dimensionality in sensitivity and uncertainty analysis.


Introduction
This work presents the mathematical framework of the "Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems" abbreviated as "5 th -CASAM-N." The 5 th -CASAM-N generalizes the previous mathematical works on this topic, which stems from the framework of the first-order adjoint sensitivity analysis methodology for generic nonlinear systems established in [1]. Numerous specific applications using adjoint functions for the deterministic computation of first-order sensitivities of scalar-valued model responses to model parameters have been published over the years, as discussed and referenced in the book by Cacuci [2]. The history regarding the deterministic computation of second-order sensitivities of model responses to model parameters reveals that although many particular applications which used specifically-computed 2 nd -order response sensitivities have been published over the years, the generic mathematical framework of the 2 nd -order adjoint sensitivity analysis methodology was presented in [3] for linear systems and in [4] nonlinear systems. The mathematical frameworks and notable applications of the 2 nd -order adjoint sensitivity analysis methodology for both linear and nonlinear systems are presented and referenced in the book by Cacuci [5].
The largest application to date of the 2 nd -order adjoint sensitivity analysis methodology for linear systems to date has been presented in [6]- [11] for the polyethylene-reflected plutonium (acronym: PERP) OECD/NEA reactor physics benchmark [12]. The numerical model [6]- [11] of the PERP benchmark com- These results were contrary to the previously held belief that 2 nd -order relative sensitivities are smaller than 1 st -order relative sensitivities for neutron transport models such as the PERP benchmark's model.
The finding that many 2 nd -order sensitivities were significantly larger than the 1 st -order ones has motivated the subsequent computation of the 3 rd -order sensitivities of the leakage response with respect to the PERP benchmark's total cross sections. The largest 3 rd -order sensitivities were computed in [13] [14] [15] by applying the 3 rd -order adjoint sensitivity analysis methodology. It has been found that the number of 3 rd -order mixed relative sensitivities that have large values (and are therefore important) is significantly greater than the number of important 2 nd -and 1 st -order sensitivities. For example, the first-order relative sensitivity of the benchmark's leakage response with respect to the total microscopic cross section of hydrogen in the lowest energy group, denoted as S σ σ σ the 4 th -Order Comprehensive Adjoint Sensitivity Analysis Methodology for Linear Systems (4 th -CASAM-L), which was applied in [17] [18] [19] [20] to the PERP benchmark for computing exactly and efficiently the most important 4 th -order sensitivities of the benchmark's total leakage response with respect to the benchmark's 180 microscopic total cross sections, which include 180 4 th -order unmixed sensitivities and 360 4 th -order mixed sensitivities corresponding to the largest 3 rd -order ones. The numerical results obtained in [17] [18] [19] [20] indicated, in particular, that the largest overall 4 th -order relative sensitivity was the 4 th -order relative sensitivity of the benchmark's leakage response with respect to the total microscopic cross section of hydrogen in the lowest energy group, namely ( ) ( ) 2.720 , 10 , , The results obtained in [6]- [19] have indicated that higher-order sensitivities cannot be simply ignored out of hand but must be computed and their impact (e.g., on uncertainty analysis) must be evaluated in the context of the application under consideration.
The results obtained in [6]- [20] have also motivated the development of the general mathematical framework for computing exactly and efficiently arbitrarily-high-order sensitivities of model responses to model parameters. Since only linear systems admit bona-fide adjoint operators (in contradistinction to nonlinear operators, which do not admit adjoint operators), Cacuci has developed [21] "The n th -Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems" (n th -CASAM-L), which enables the exact and efficient computation of sensitivities, of any order, of model responses to model parameters, including imprecisely known domain boundaries, thus enabling the quantification of uncertainties stemming, among other factors, from manufacturing tolerances. The n th -CASAM-L overcomes the "curse of dimensionality" [22] in sensitivity and uncertainty analysis, as detailed in [23].
For nonlinear systems, the first general methodology which also enabled the exact and efficient computation of model response sensitivities to uncertain domains of definition of the model's independent variables (in addition to response sensitivities to model parameters) were the works [24] [25] [26] [27] [28] on the "1 st -Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems." The "Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5 th -CASAM-N)" to be presented in this work generalizes and extends the mathematical framework presented in [24] [25] [26] in order to enable the computation of 5 th -order sensitivities. This work is structured as follows: Section 2 presents the mathematical framework of the 5 th -CASAM-N, which builds on the lower-order adjoints sensitivity analysis methodologies [24] [25] [26] [27] [28]. The significance of the potential applica-† superscript.
The computational model of a physical system comprises equations that relate the system's state variables to system's independent variables and parameters, which are considered to be afflicted by uncertainties. ( ) Ω α is also considered to be imprecisely known since it may depend on both geometrical parameters and material properties. For example, the "extrapolated boundary" in models based on diffusion theory depends both on the imprecisely known physical dimensions of the problem's domain and also on the medium's properties (atomic number densities, microscopic transport cross sections, etc.).
The model of a nonlinear physical system comprises coupled equations which can be represented in operator form as follows: The quantities which appear in Equation (1) are defined as follows: 1)  is a TD-dimensional column vector of dependent variables (also called "state functions"), where "TD" denotes "total number of de- is a TD-dimensional column vector which represents inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters α ; 4) since the right-side of Equation (1) may contain "generalized functions/functionals" (e.g., Dirac-distributions and derivatives thereof), the equalities in this work are considered to hold in the distributional ("weak") sense.
When differential operators appear in Equation (1), their domains of definition must be specified by providing boundary and/or initial conditions. Mathematically, these boundaries and/or initial conditions can be represented in operator form as follows: where the column vector 0 has TD components, all of which are zero. The components ( ) ; , , ; The model response considered in this work is a nonlinear functional of the model's state functions and parameters which can be generically represented as follows: where ( ); S     u x α is suitably differentiable nonlinear function of ( ) u x and of α . Noteworthy, the components of α also include parameters that may occur just in the definition of the response under consideration, in addition to the parameters that appear in Equations (1) and (2). Since the system domain's boundary, ( ) ∂Ω α , is considered to be subject to uncertainties (e.g., stemming from manufacturing uncertainties), the model response  u x α will also be affected by the uncertainties that affect the endpoints

The First-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (1 st -CASAM-N)
The model and boundary parameters α are considered to be uncertain quantities, having unknown true values. The nominal (or mean) parameter vales 0 α are considered to be known, and these will differ from the true values by quantities denoted as ( ) . Since the forward state functions ( ) u x are related to the model and boundary parameters α through Equations (1) and (2), it follows that the variations δα in the model and boundary parameters will cause corresponding variations δ R e h , of the response will exist and will be linear in 2) ( ) R e satisfies the following condition: In Equation (7), the symbol F denotes the underlying field of scalars. Numerical methods (e.g., Newton's method and variants thereof) for solving Equations (1) and (2) also require the existence of the first-order G-derivatives of original model equations. Therefore, the conditions provided in Equations (6) and (7) are henceforth considered to be satisfied by the model responses and also by the operators underlying the physical system modeled by Equations (1) and (2). When the response ( ) R e satisfies the conditions provided in Equations (6) and (7), the 1 st -order G-differential .
In Equation (8), the "direct-effect" term ; ; , The direct-effect term can be computed once the nominal values ( ) u α are available. The notation { } 0 α will be used in this work to indicate that the quantity enclosed within the bracket is to be evaluated at the respective nominal parameter and state functions values.
On the other hand, the quantity in Equation (8) comprises only variations in the state functions and is therefore called the "indirect-effect term," having the following expression: The "indirect-effect" term induces variations in the response through the var- the definition of the G-differential to Equations (1) and (2), which yields the following equations: In Equations (14) and (15), the superscript "(1)" indicates "1 st -Level" and the various quantities which appear in these equations are defined as follows: ; ; ; The system comprising Equations (14) and (15)   .
Evidently, Equations (14), (15), (20) and (21) indicate that ( ) ( ) ( ) ( ) j v x , 1 1, , j TP =  . As has been originally shown by Cacuci [1], the need for computing the vec- is eliminated by expressing the indirect-effect term defined in Equation (12) in terms of the solutions of the "1 st -Level Adjoint Sensitivity System" (1 st -LASS), the construction of which requires the introduction of adjoint operators. This is accomplished by introducing a (real) Hilbert space denoted as ( ) 1 x Ω H , endowed with an inner product of two vectors ( ) u u and defined as follows: where the dot indicates the scalar product ( ) ( ) ( ) ( ) Using the inner product defined in Equation (22), construct the inner product of Equation (14) with a vector ( ) ( ) 1 a x to obtain the following relation: Using the definition of the adjoint operator in , the left-side of Equation (23) is transformed as follows: ; , where ( ) ( )

;
A u α is the operator adjoint to ( ) ( ) α , and where ; ; ; The first term on the right-side of Equation (24) is required to represent the indirect-effect term defined in Equation (12) by imposing the following relationship: The domain of ( ) ( ) 1 ; A u α is determined by selecting appropriate adjoint boundary and/or initial conditions, which will be denoted in operator form as: The above boundary conditions for ( ) ( )  (15) and (26)  ; ; , which will contain boundary terms involving only is linear in δα , it can be expressed in the following form: The results obtained in Equations (24) and (25) are now replaced in Equation (12) to obtain the following expression of the indirect-effect term as a function ; , Replacing in Equation (8) the result obtained in Equation (27) together with the expression for the direct-effect term provided in Equation (9) yields the following expression for the first G-differential of the response ; , where, for each 1  u x α with respect to the model parameters 1 j α and has the following expression: As indicated by Equation (29), each of the 1 st -order sensitivities ∈ a x H , using quadrature formulas to evaluate the various inner products involving ( ) ( ) ∈ a x H is obtained by solving numerically Equations (25) and (26), which is the only large-scale computation needed for obtaining all of the first-order sensitivities. Equations (26) and (25) , is called the 1 st -level adjoint function. It is very important to note that the 1 st -LASS is independent of parameter variation , and therefore needs to be solved only once, regardless of the number of model parameters under con- , solving it requires less computational effort than solving the original Equation (1),

The Second-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (2 nd -CASAM-N)
The 2 nd -CASAM-N relies on the same fundamental concepts as introduced in [4], but in addition to the capabilities described in [4], the 2 nd -CASAM-N also enables the computation of response sensitivities with respect to imprecisely known domain boundaries, thus including all possible types of uncertain parameters. Fundamentally, the 2 nd -order sensitivities are defined as the "1 st -order sensitivities of the 1 st -order sensitivities." This definition stems from the inductive definition of the 2 nd -order total G-differential of correspondingly differentiable function, which is also defined inductively as "the total 1 st -order differential of the 1 st -order total differential" of a function. The 1 st -order sensitivities  u x a x α are assumed to satisfy the conditions stated in Equations (6) and (7), for each 1 , which ensures the existence of the 2 nd -order sensitivities. The G-variation  u x a x α has the following expression: .
Also in Equation (30), the indirect-effect term The functions ( ) ( ) δ a x are obtained by solving the following 2 nd -Level Variational Sensitivity System (2 nd -LVSS): The argument "2" which appears in the list of arguments of the vector To distinguish block-vectors from block matrices, two capital bold letter have been used (and will henceforth be used) to denote block matrices, as in the case of "the second-level variational matrix" is indicated by the superscript "(2)". Subsequently in this work, levels higher than second will also be indicated by a corresponding superscript attached to the appropriate block-vectors and/or block-matrices. The argument " 2 2 × ", which appears in the list of arguments of ( ) The other quantities which appear in Equations (33) and (34)    ; ; .
The structure of the second component of the source-term Taking into account the expressions in Equations (43) and (44) while recalling the expressions in Equations (17) and (18) indicates the actual form of 2 nd -LVSS to be solved (if one would wish to solve it) would be as follows: Thus, there would be 2 TP "variational vectors" ( ) ( ) x to be computed. The need to avoid such impractical, extremely expensive, computations provides the fundamental motivation underlying the development of the adjoint sensitivity analysis methodologies for computing sensitivities (of all orders) of model responses with respect to the model's parameters. Thus, since "variational sensitivity systems" will never need to be actually solved if the 5 th -CASAM-N methodology developed and presented in this work is utilized, the dependence on the indices 1 j , 2 j of the variation sensitivity systems will be suppressed in this work, in order to simplify the mathematical notation. On the other hand, since the solutions of the adjoint sensitivity systems of various levels will actually be computed in practice, the dependence on the indices will be displayed explicitly. The need for solving the 2 nd -LVSS is circumvented by deriving an alternative expression for the indirect-effect term defined in Equation (32), in which the function ( ) ( ) x is replaced by a 2 nd -level adjoint function which is independent of variations in the model parameter and state functions, and is the solution of a 2 nd -Level Adjoint Sensitivity System (2 nd -LASS) which is constructed by using the 2 nd -LVSS as starting point and following the same principles as outlined in Section 2.1. The 2 nd -LASS is constructed in a Hilbert space, denoted as ; , ; Following the same principles as outlined in Section 2.1, the inner product defined in Equation (47) where:   TD , thus comprising a total of ( ) 2 2 2 TD × components (or elements) and is obtained from the following relation: 2; , 2 2; 2; ; 2; , where the quantity ; ; ; ; terms containing unknown values of ( ) ( ) Using the 2 nd -LASS to obtain the alternative expression for the indirect-effect term in terms of ( ) ( and the expression for the direct-effect term provided in Equation (31) yields the following expression for the total differential defined by Equation (30): ; ; ; ; .
If the 2 nd -LASS is solved TP-times, the 2 nd -order mixed sensitivities will be computed twice, in two different ways, in terms of two distinct 2 nd -level adjoint functions. Consequently, the symmetry property ; ;

The Third-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (3 rd -CASAM-N)
The 3 rd -order sensitivities will be computed by considering them to be the "sensitivities of a 2 nd -order sensitivity." Thus, each of the 2 nd -order sensitivities will be considered to be a "model response" which is assumed to satisfy the conditions stated in Equations (6) and (7) for each 1 2 , 1, , j j TP =  , so that the 1 st -order total G-differential of , , ,  0 0 0 0 0 and where: ;  2;  ; ; The right-side of the 3 rd -LVSS actually depends on the indices large-scale systems comprising many parameters. Since the 3 rd -LVSS is never actually solved but is only used to construct the corresponding adjoint sensitivity system, the actual dependence of the 3 rd -LVSS on the indices 1 2 3 , , 1, , j j j TP =  has been suppressed.
The 3 rd -CASAM-N circumvents the need for solving the 3 rd -LVSS by deriving an alternative expression for the indirect-effect term defined in Equation (56) , denoted as four TD-dimensional vector-components of the form where each of these four components is a TD-dimensional column vector. The inner product of two vectors x Ω H will be denoted as The steps for constructing the 3 rd -LASS are conceptually similar to those described in Sections 2.1 and 2.2 and are detailed in [27]. The final expressions for the 3 rd -order sensitivities are as follows: ; , ; ; denotes residual boundary terms which may have not vanished automatically, and where the 3 rd -level adjoint function 3  3  3  3  3  3 4; 1; , 2; , 3; , 4; H is the solution of the following 3 rd -LASS: where: ; ; The boundary conditions to be satisfied by each of the 3 rd -level adjoint func- .

The Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (4 th -CASAM-N)
Assuming that the 3 rd -order sensitivities ; ; satisfy the conditions stated in Equations (6) and (7) is given by the following expression: .
In component form, the total differential expressed by Equation (94) can be written in the following form, for each 1 where the quantity and has the following expression: ; ; , , In Equation (96), the quantity ; ; The quantities which appear in Equations (97) and (98)   ; ; ; ;

The Fifth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (5 th -CASAM-N)
Each of the 4 th -order sensitivities ( ) In Equation (106) where: The direct-effect term  ; where: The quantities which appear in Equations (113)-(119) are evaluated at the nominal values of the parameters and respective state functions, but the notation { } 0 α , which indicates this evaluation, has been omitted, in order to simplify the notation.
The quantities ( ) ( ) ( ) 2 ; , , , , ; j j j j j V x, which is unrealistic for large-scale systems comprising many parameters. Since the 5 th -LVSS is never actually solved but is only used to construct the corresponding adjoint sensitivity system, the actual dependence of the 5 th -LVSS on the indices 1 2 3 4 5 , , , , 1, , j j j j j TP =  has been suppressed.
The 5 th -CASAM-N methodology, which will presented in the remainder of this Section, circumvents the need for solving the 5 th -LVSS by deriving an alternative expression for the indirect-effect term defined in Equation (108), in which the function ( ) ( ) 5 4 2 ; V x is replaced by a 5 th -level adjoint function which is independent of parameter variations. This 5 th -level adjoint function will be the so-lution of a 5 th -Level Adjoint Sensitivity System (5 th -LASS) which will be constructed by applying the same principles as those used for constructing the The inner product defined in Equation (121) The inner product on the left-side of Equation (122) is further transformed by using the definition of the adjoint operator to obtain the following relation: ; , where:  2 ; x ∈ Ω A x H satisfies adjoint boundary/initial conditions denoted as follows: The 5 th -level adjoint boundary/initial conditions represented by Equation (125) are determined by requiring that: 1) they must be independent of unknown values of ( ) ( ) ; ; must cause all terms containing unknown values of ( ) ( ) 5 4 2 ; V x to vanish.
Implementing the boundary/initial conditions represented by Equations (112) and (125) into Equation (123) will transform the later relation into the following form:  ; .
The definition of the 5 th -level adjoint function 1, , j j =  . Each of these distinct 5 th -level adjoint functions will correspond to a specific ( ) 1 2 3 4 , , , j j j j -dependent indirect-effect term.
The left-side of Equation (127) will be identical to the right-side of Equation (108) by requiring that the following relation be satisfied by the 5 th -level adjoint functions In component form, the total differential expressed by Equation (136)  ; . ; × AM U α is block-diagonal; therefore, solving the 5 th -LASS is equivalent to solving five times the 1 st -LASS, with five different source terms. The 5 th -LASS was designated as the "fifth-level" rather than "fifth-order" adjoint sensitivity system since the 5 th -LASS does not involve any explicit 2 nd -order, 3 rd -order, 4 th -order and/or 5 th -order G-derivatives of the operators underlying the original system but involves the inversion of the operators similar to those that needed to be inverted for solving the 1 st -LASS. The 5 th -CASAM-N provides a fundamental step towards overcoming the curse of dimensionality in sensitivity and uncertainty analysis. A paradigm illustrative application to a Bernoulli model with uncertain parameters and boundaries will be presented in an accompanying work [28].

Conflicts of Interest
The author declares no conflicts of interest regarding the publication of this paper.