Numerical Methods in Financial and Actuarial Applications : A Stochastic Maximum Principle Approach

While numerical approaches to solve financial and actuarial stochastic optimization problems are usually based on dynamic programming, we explore an approach through a stochastic maximum principle formulation followed by the use of least squares regression to determine the optimal control policy. We show that this methodology can be applied to a number of realistic financial and actuarial problems of increasing complexity to highlight potential strengths and applications of this approach. We cast a direct connection between this approach and the stochastic duality approach to stochastic optimization. In particular, we discuss the potential improvements which can derive from this reformulation in terms of numerical precision and in order to provide bounds to control the simulation errors. The critical numerical issue is shown to be the numerical computation of conditional expectations which is performed applying the approach of Longstaff and Schwartz [1].


Introduction
A portfolio simulation approach to the valuation of optimal portfolio policies appears to be the only viable approach in many financial and actuarial applications.
While usually numerical methods are formulated within a dynamic programming approach to the optimization problem, we explore the possibility to state a numerical scheme that goes through the solution of the forward-backward stochastic differential equation (FBSDE, hereafter) which arises from the maximum principle formulation of the optimal control problem.We exemplify situations where the maximum principle approach to portfolio optimization jointly with the simulation approach to the backward stochastic differential equation (BSDE, hereafter) provides an efficient alternative to standard simulation methods based on direct Montecarlo simulation and produces a reliable estimate of the optimal allocation strategies.
Our computation approach relies on the least-squares Monte Carlo approach for estimating the conditional expectations made popular in financial mathematics by of Longstaff and Schwartz [1] and it was first applied to BSDEs and analyzed in this setting by Bouchard and Touzi [2], Gobet et al. [3] and Lemor et al. [4].
More recently, Bender and Steiner [5] focus on the least-squares Monte Carlo scheme for approximating the solution of a BSDE by using basis functions which form a system of martingales.
The paper is organized as follows.Section 2 introduces the stochastic maximum principle and the economic implications of the first-order adjoint process.
Section 3 illustrates some known results on the FBSDEs.In Section 4 we analyze the simulation approach to the BSDEs.Section 5 is devoted to the discussion of some financial and actuarial applications.

The Stochastic Maximum Principle
We consider a finite time horizon [ ] 0,T , and a complete probability space ( ) Let us consider the time evolution of the system described by a state variable ( ) x ⋅ whose dynamics follow the stochastic differential equation: where The function ( ) ν ⋅ is the control variable representing the policy of the decision-maker at any time and it is required to be progressively measurable since the controller knows what happened up to the moment, but not able to foretell what is going to happen later on, due to the uncertainty of the system.
The objective is to minimize a given cost functional defined as: , over the set of the admissible controls: Of course, for any ( ) ( ) , by standard assumptions on b, σ, ν that can be found, e.g., in Yong and Zhou ([6], Theorem 6.17, p. 50), the above state Equation (1) admits a unique solution ( ) ( ) ( ) and Zhou [6], Definition 6.15, pp.48-49) and Then, the problem is to find an optimal control ( ) ( ) are called optimal state trajectory and optimal pair, respectively.
Recalling that, for any given function φ, we may also treat maximization problems with the same steps.
The Hamiltonian function associated to our problem is given by: ( while the generalized Hamiltonian function is given by the following FBSDE: The extended Hamiltonian system corresponding to the pair given by the following FBSDE: where: is the -function.
In the extended Hamiltonian system above, the second and the third relations are clearly BSDEs and called adjoint equation of the first-order and adjoint equation of the second-order respectively, while the last relation is the so-called maximum condition.
The state variable joint with the related first-order adjoint equation are called Hamiltonian system.
is called an optimal 6-tuple (resp.admissible) if is an optimal pair, (resp. is an admissible pair), ⋅ is an adapted solution of the first-order adjoint equation and ( ) ( ) ( ) ⋅ is an adapted solution of the second-order adjoint equation.
The following statement is the well-known (Pontryagin) stochastic maximum principle, which gives a set of first-order necessary condition for 6-tuple pairs.Theorem 2.3 (stochastic maximum principle).Let Hypothesis 2.2 hold and be an optimal pair.Then the optimal 6-tuple solves the above extended stochastic Hamiltonian system.
is concave -a.s., and the maximum condition is verified, i.e.
is an optimal pair.

Economic Implications of the First-Order Adjoint Process
The first-order order adjoint process plays a very important role in economic theory that we aim to introduce.
Assume that the framework of the previous section hold, but for simplicity all the variables are scalar-valued.For any initial data ( ) [ ] , 0, t x T ∈ × , consider the controlled system: , and the functional: is the unique solution to the above state equation.
We have the following definition.
Definition 2.6.The following: ( , ×  and tx V is also continuous, then: , , -a.s., (3b) At any time t suppose ( ) Intuitively, the above relation shows that the value of ( ) y t measures the rate of variation of the value function induced by an infinitesimal increase of the level of the state variable.Accordingly, the adjoint variable is called the marginal value of the optimal state.
Suppose the increment of the optimal state value could be purchased in the market.Thus, the adjoint variable represents the maximum price it would be worth paying an additional amount of the state.For this reason, the adjoint variable is also called the shadow price of the optimal state.Moreover, if V  is the convex dual value function of V, then the following relationships apply (see, e.g., Di Giacinto et al. [7]): It is straightforward from the above relations to derive the following interesting relations:

FBSDEs and BSDEs
The conditions of optimality derived from the maximum principle are in general FBSDEs.In the following we review the basic results on the mathematical theory that explores existence uniqueness and solvability of these equations.In particular, we will focus on the interesting examples that will be useful in the applications when the FBSDE is decoupled, i.e. the forward component is independent from the backward one.
While for the forward component the basic properties are known from the theory of general stochastic processes, it is important to review the results about the backward component that is determined by a BSDE.Basic mathematical results on this class of equations are reviewed in the following subsection.


-progressively measurable, the pair ( ) , g ξ is the data, called the terminal condition and the generator (or driver), respectively.Here, ξ is T  -measurable and . A solution to the above BSDE is a pair of processes ( ) , y z adapted to the given filtration.

Solving an ordinary differential equation formulated as a terminal value problem or an initial value problem on the same time interval [ ]
the contrary, when we are looking for a solution to a BSDE that is adapted respect to the given filtration we know only the past information, but are not able to know what is going on to occur in the future.This means that it is not possible simply reverse the time to find a solution as it would remove the adaptiveness.
The theory of BSDEs is an active domain of research due to its connections with stochastic control, mathematical finance, and partial differential equations (see, e.g., the survey elaborated by El Karoui et al. [8]).
BSDEs were pioneered by Bismut [9] [10] [11] within the stochastic optimal control.Existence and uniqueness of adapted solutions were first proved by Pardoux and Peng [12] in a Brownian setting, with integrable terminal condition and generators that fulfill a canonical Lipschitz assumption in the space variables.
These results were extended by Lepeltier and San Martín [13] to continuous drivers with linear growth.Since then, a lot of efforts have been made in order to study the well posedness of these equations.
With regards to the quadratic growth BSDEs, the proof of the existence of solutions are easily obtained under rather general framework.Conversely, the question of the uniqueness results is not trivial and requires stronger assumptions on the solutions.
In the standard framework where the terminal condition is bounded, in Kobylanski [14] obtained for the martingale case the uniqueness of the solution under some Lipschitz conditions.This seminal result has been extended in several directions, among others, by Tevzadze [15], Morlais [16], and Briand-Elie [17].In Tevzadze [15] as well as in Briand-Elie [17] the proof in Kobylanski [14] was simplified in some less general cases.In particular, Tevzadze [15] showed the proof for the existence and uniqueness of a bounded solution in the Lipschitz quadratic, while Briand-Elie [17] incorporated delayed quadratic BSDEs, whose generator depends on the recent past of the component of the solution.Morlais [16] treated quadratic BSDEs driven by a continuous martingale and applied the corresponding results to study some related utility maximization problem.
Concerning the quadratic BSDEs with unbounded terminal value, Briand-Hu [18] first showed the existence of solution, while Barrieu and El Karoui [19] revisited the existence by a direct forward method that does not apply the result of Kobylanski [14].
Barrieu and El Karoui [19] also showed some uniqueness results that require stronger assumptions comparing with the other existing literature.However, their alternative approach has other potential applications as numerical simulations of quadratic BSDEs and risk measures and their dual representations.
The uniqueness result was tackled for the first time by Briand and Hu [20] under some restrictive conditions.In particular, the authors proved the uniqueness among the solutions of quadratic BSDEs with convex generators and unbounded terminal conditions which admit every exponential moments.
Delbaen et al. [21] [22], strengthened the above result proving that uniqueness holds among solutions reducing the required order of exponential moments.

Simulation Approach to BSDE
The simulation approach to BSDE applying Longstaff and Schwartz [1] has been investigated the first time by Bouchard and Touzi [2].In order to consider the numerical solution of a BSDE consider for simmplicity the simple case where the backward component is a pure martingale, i.e. the drift component is zero.Then the determination of ( ) y ⋅ and ( ) z ⋅ can be performed observing at time T, and can be obtained simulating forward ( ) Given the random variable − can be performed computing a conditional expectation using the simulation approach of Longstaff and Schwartz [1] relying on a linear regression and moving backward.The exact choice of the polynomial family is relevant only for the numerical efficiency.For example, using Hermite polynomials we obtain: where the standard error of the residual ( ) Note that the approximation of ( ) ( ) ( ) , 0, .
We will show that in many cases, in particular in the applications that we will discuss later, the approximation to the pair: provides an approximate solution of the BSDE that can be used to recover the optimal control strategies using the optimality conditions.

Applications
In the following applications we will always consider a standard Black & Scholes market, i.e. a risk free bond ( ) 0 S ⋅ , whose dynamics are: where 0 r ≥ is the risk free rate, and a traded risky asset ( ) where 0 σ ≥ is the volatility, : 0 r µ σλ =+ > is the drift, and 0 λ > is the instantaneous risk premium, i.e. the Sharpe ratio.

Classical Merton Problem
In this subsection we illustrate the application of the stochastic maximum principle to the solution of the well-known Merton's portfolio allocation and consumption problem (see, e.g., Merton [23]) when the utility has a constant relative risk-aversion as in Merton [24].
The state equation describing the dynamics of wealth is given by: The preferences of the agent are given by the following utility function: while the bequest function is: The objective functional we aim to optimize is given by: belongs to the following set of admissible strategies: The problem we aim to solve is to find an optimal control ( ) ( ) The stochastic Hamiltonian system associated to the problem is: ( while the second-order adjoint equation is: It can be easily found that: ( Once we find an adapted solution candidate for the optimal control can be deduced by (4).To this regard, suppose  is an adapted solution and take the following guess function for ( ) We would like to determine the equation that ( ) ⋅  should satisfy.To this end, we differentiate the above using Itô formula: Comparing the above relation with the first-adjoint equation that appears in the previous stochastic Hamiltonian system, we have: ( The above (6b)-(6c) is a Bernoulli equation whose solution is: where ( ) and subject to the condition 0 β > , i.e.
( ) With the position of ( ) ⋅  stated in (7) and the condition on ρ stated in (8), the pair of processes: is the only square integrable adapted solution to the first-order adjoint equation.
The necessary condition required from the stochastic maximum principle theorem leads to the following policy: and state trajectory (simply obtained substituting the expression of the above control policy to the stochastic Hamiltonian system): ( ) ( ) ( ) (10) as candidates to be an optimal pair.Finally, a simple check shows that sufficient conditions of optimality are satisfied.We conclude that ( ) ( ) ( ) pointed out to (10) and ( 9) is an optimal pair of the classical Merton optimization problem.Therefore, the following result holds.
Proposition 5.2.There exists a unique optimal investment and consumption strategy given by: where ( ) ⋅  must satisfies the following Bernoulli ordinary differential equation:

Merton Model with a Retirement Endowment
In a standard Black & Scholes market, the dynamics of wealth are given by: where , , r λ σ are the usual market parameters.
In addition, the agent is endowed with a nontradable investment I in a pension fund whose risky component is perfectly correlated with the market index.The dynamics of the investement are given by: in order to avoid arbitrage opportunities.
The results can be tested since it is possible to evaluate the optimal proportion.
As a matter of fact, due to the market completness it is possible to estimate the exact value of the optimal proportion ( ) and the relative error is of order 2.20%.

Income Drawdown Option with Minimum Guarantee
In this subsection we consider a second interesting application of the same approach that provides a numerical way to extend the analysis carried out in Di Giacinto et al. [7] which is briefly described below.The numerical extension we explore is necessary in order to assess the impact of the running cost as defined in the model.
We consider a defined contribution pension plan for a single representative participant during the decumulation phase, i.e. after retirement.Depending on the law and the rules of the scheme in many countries the retiree is allowed: 1) to defer annuitization at some time after retirement (this option is named, e.g., in UK "income drawdown option", in US "phased withdrawals"); 2) to withdraw periodic income from the fund; 3) to invest the rest of the fund in the period between retirement and annuitization.
Thus, the pensioner has three degrees of freedom: 1) to decide when to annuitize (if ever); 2) to decide how much of the fund to withdraw at any time between retirement and ultimate annuitization (if any); 3) to decide what investment strategy to adopt in investing the fund at her disposal.
The first choice (i.e. the optimal time of annuitization) can be tackled by defining an optimal stopping time problem.The last two choices (i.e. the optimal consumption and investment selection) represent a classical investment and consumption problem, which can be solved using, e.g., stochastic optimal control techniques.
The contribution by Di Giacinto et al. [7] is the solution in closed form of the optimal control problem with constraints regarding the third choice (i.e. the investment choice) using a quadratic loss function and by applying the dynamic programming approach.
In a standard Black & Scholes market, the state equation is given by: where y 0 is determined following the same procedure applied in (11).
Note that the above Hamiltonian system is linear, thus the Riesz duality can be applied to switch the forward and the backward components.While for 0 κ = the Hamiltonian system is represented by a decoupled FBSDE, so the previous numerical approach can be applied, for 0 κ ≠ the FBSDE is coupled but still linear and therefore numerically solvable by applying a different numerical approach, e.g., the four step scheme illustrated by Ma et al. [25].
In the following we illustrate the simulation paths of the optimal investment strategy for 0 κ = and some realistic parametrization.The parameter chosen are:

Conclusions
In the above analysis, we showed how to use the stochastic maximum principle to reformulate conventional portfolio consumption models in order to explore their solution through the numerical solution of a linear stochastic hamiltonian system.We showed in a number of applications the flexibility of this methodology.
In the next future we aim to explore the performance of this method in two important extensions: a multidimensional state variable problems and in presence of optimal stopping control strategies.This last extension is particularly relevant for actuarial applications.

Hypothesis 2 . 4 .
The maps , , b f σ are locally Lipschitz in ν ∈  and their derivatives in n x ∈  are continuous in ( ) sufficient condition of optimality).Let Hypotheses 2.2 and 2.4 hold.Suppose that  is the investment strategy representing the proportion of wealth invested in the risky asset, → +∞ is the consumption stategy, and , , r λ σ are the market parameters of the Black-Scholes model.
→ +∞ is the process representing the fund wealth of the pensioner, x 0 is the fund wealth at the retirement date 0 t = , ( ) π ⋅ is the dollar amount invested in the risky asset, b 0 is the consumption rate of the pensioner, and , , r λ σ are the usual market parameters. d the retiree age, b 0 is computed by applying the projected Italian males life table and NP stands for performed number of paths.The final target 97.13F = and the final guarantee 27.75 S = are subjective and determined by the pensioner's risk aversion.