Optimal Control of a Vaccinating Game toward Increasing Overall Coverage

In this paper, we study an asymmetric game that characterizes the intentions of players to adopt a vaccine. The game describes a decision-making process of two players differentiated by income level and perceived treatment cost, who consider a vaccination against an infectious disease. The process is a noncooperative game since their vaccination decision has a direct impact on vaccine coverage in the population. We introduce a replicator dynamics (RD) to investigate the players’ optimal strategy selections over time. The dynamics reveal the long-term stability of the unique Nash-Pareto equilibrium strategy of this game, which is an extension of the notion of an evolutionarily stable strategy pair for asymmetric games. This Nash-Pareto pair is dependent on perceived costs to each player type, on perceived loss upon getting infected, and on the probability of getting infected from an infected person. Last but not least, we introduce a payoff parameter that plays the role of cost-incentive towards vaccination. We use an optimal control problem associated with the RD system to show that the Nash-Pareto pair can be controlled to evolve towards vaccination strategies that lead to a higher overall expected vaccine coverage.


Introduction
In case of an epidemic arising from an infectious disease for which an effective vaccine is available, the desired policy is to increase the number of people who will choose this vaccine [1] [2].One of the common ways to analyze this issue is The origin of game theory can be traced back to the works of Von Neumann [14] [15].The works of Nash [16] form the basis of classic 2-player noncooperative games.Since then, numerous works have been advanced in this area.Some of the most recent ones are for instance: [17], studying 2-player zero-sum game with random payoff matrix to investigate the connections of the best payoff of 2 players under joint chance constraints; and [18] who have discussed a 2-person zero-sum game with infinite action space and bounded payoff functions.In symmetric games, the idea of finding a solution for a competitive contest between two players or more is strongly related to the concept of Nash equilibrium (NE).In nonzero asymmetric games, the Nash-Pareto equilibrium concept is required to improve the strategy of one of the players without making the opponent's strategy worse [19].The existence of Pareto equilibrium was discussed in [20], in a game with multicriteria payoffs, neglecting the linear and convexity structure, in general, topological spaces.Pareto-efficient optimization is addressed in [21] to explore the significance of players' cognition in the game.The authors introduced a new definition for Nash game with the observance of players' beliefs.Consequently, optimizing the players' belief can forthright the game to the Pareto-efficient equilibrium.
One way to incentivize players to act in a certain way is to consider the incentive as a parameter in the players' payoffs.Exerting control on this parameter leads us to consider the game from an optimal control problem's perspective.Starting in the 50's, optimal control was the key to studying variational problems.The main contribution in this field was made by (Pontryagin, 1964) when he theorized the maximum principle theorem [22].The optimal control problem has been extensively studied by many authors over the years.For instance, [23] and [24] discussed optimal control problems with necessary conditions.However, [25] and [26] looked at optimal control problems with sufficient conditions.The recent century has witnessed contributions by many scientists in enriching the concepts and literature of theoretic games and control problems.For example, [27] used a straightforward model where certain several suppositions will let integrate evolutionary games into the drug-dosage formulation as an optimal control problem.In [28] built a novel model to optimize cancer chemotherapy by studying it as an evolutionary game between cancer and the oncologist.In our most recent work, we applied optimal control to a replicator dynamics system of a norms game in a population, and we showed that the otherwise evolutionarily stable state of all defectors can be controlled towards a norm-dominated population [29].
In this paper, we study and formulate an asymmetric game as in [1], where the players have different levels of income, different probabilities of infection and differing costs of vaccinating.The suggested control is used as an incentive to encourage the players toward vaccination and away from a stable Nash-Pareto pair of strategies which has a cost depending probabilities of vaccinating for both player types.We find the optimal control values and the optimal strategies with the objective of maximizing the sum of the players who choose to vaccinate numerically.
The structure of this paper is presented as follows: In section 0 we present our formulation of the vaccination game inspired by [1], where we implement a modification on its payoffs.Further, we find the game's Nash equilibria and their stability characteristics after we formulate the game in an associated replicator dynamics form.We show that there exists on Nash-Pareto stable pair of mix strategies for the game.In Section 3 we incorporate the replicator dynamics equations into an optimal control problem where we seek to decrease the perceived costs of vaccinating in both groups.We show that this exogenous control results in gains in overall expected vaccine coverage in the population.

The Formulation of the Vaccination Game Model
We consider a game similar to the one in [1], but where the pathogen is Journal of Applied Mathematics and Physics transmitted only via humans.The game considers two players in different positions where each player has two pure strategies: to vaccinate (V), or not vaccinate (NV).A player { } 1, 2 i ∈ who decides to vaccinate must pay some "cost", denoted by i c , which here reflects an amalgamation of psychological cost, time or monetary cost, and cost of side effects (if developed).The probability of getting infected, denoted by q, will cause a player to incur some "loss", denoted by i L if the infection develops.Thus we consider the following asymmetric bimatrix game: ) ( ) , , where i I is the income of Player i.This game has a bimatrix payoff, s.t.we can write the sub-matrices for each player in the game as follows: If we consider this game in mixed strategies, denoted by ( ) 0,1 x y∈ being the probabilities of Player 1, respectively Player 2, to vaccinate, we write the expected payoff functions for each player as follows: which imply further: ) ,1 ,1

E x x xqL xc xyqL I qL yqL E y y yqL yc yxqL I qL xqL
To solve the game in (1) we use the reaction curves method so we rewrite the expected payoffs in (3) as linear functions of their corresponding variables s.t.
for 1 E we have: To maximize 1 E and 2 E in (4) we need to find their stationary points: The derivative of 1 E depend on the parameter y; therefore, the sign of depends on y to determine the growth in Same analysis can be made for 2 E and we get that ∈ then by follow the same analysis as above, we get: Thus, we have an equilibrium when ≥ ≥ .The Nash equilibrium in mixed strategies can be now calculated from finding all points of x and y so that the equilibria are (Figure 1):

Replicator Dynamics and Nash-Pareto Pairs
Let us think now of our game as a game with two types of players, differentiated by income and vaccination costs, each with two pure strategies: Then we think of the mixed strategy , with 1 the fraction of population of type 1, who chooses pure strategy i.Similarly, Journal of Applied Mathematics and Physics  It is known (see [19]) that the replicator dynamics (RD) associated with our game is: To simplify our study of the replicator dynamics associated with our game, we first agree to denote the pure strategies V, NV by indexes 1, respectively 2, for each player.Then we rescale (as in [19], Chapter 11) our game (1) into a bimatrix game with the same Nash equilibria computed in the previous section, using the following definitions: , then the bimatrix game ( ) It is known that Nash equilibrium points of a rescaled game are the same as those of the original bimatrix game.We show below that our bimatrix game can be rescaled to a γ-zero-sum-game.
Proof.We show first that there exists ( ) , G G as defined in (14), which simplifies the players' matrices as an intermediary step.
Then we show that ( ) is a γ-zero-sum-game using (15).In (14)   ) ( ) where and 0 0 In this way the relations ( 14) can be written as: We now show that the bimatrix game ( ) ( ) , that there exist , , We solve this linear system and we find the following: and : , 0 and : , 0 Using these values in (15) of Definition 0.1, the bimatrix game ( ) Using now ( 16) and ( 14) we can conclude that ( ) where : , 1, 2 and , 1, 2 Hence the original bimatrix game (1) is a (−1)-zero-sum-game: Nash-Pareto is a relaxation of the idea of evolutionarily stable strategy (ESS) for asymmetric games so that we can include mixed strategies.Suppose that our two subpopulations (of player of type 1, respectively of player of type 2) are in a state ( ) 2 2 , p q S S ∈ × where w w w = ∈ + = .This state will not be stable in an evolutionary sense if there exists a neighboring state, say ( ) , x y , such that both types can increase their mean payoff by to ( ) , x y .We then define the following: Definition 2.2 Let ( ) , p q be a state for our population in an asymmetric game with payoff matrices ( ) , G G .Then ( ) , p q is a Nash-Pareto pair for this game if the following two conditions hold: 1) 2) For all states x G y p G y y G x q G x ⋅ > ⋅ ⇒ ⋅ < ⋅ , and The following result is known (see [19]).
Theorem 2.1 If the pair ( ) , x y is in the interior of the set 2 2

S S
× , then ( ) x y is a Nash-Pareto pair of the bimatrix ( ) , G G is a rescaled zero-sum-game.Moreover, the Nash-Pareto point is stable in the replicator dynamics (13).
Using Proposition 1 and Theorem 2.1 above we have that ( ) , x y S S ∈ × is indeed a Nash-Pareto point of our bimatrix game and is stable for the dynamics (13).

Nash-Pareto Equilibria and Evolutionary Stability
0,1 0,1 x y ∈ × be a set of mixed vaccinating strategies for the players in our game so that * 3 x x ≠ and/or * 3 y y ≠ .Then, the reduced replicator dynamics associated with the dynamics (13) and to the asymmetric bimatrix game is given by (see [30]): ( )( Putting all computations together, we have the following explicit RD: It is immediate to see that all three Nash equilibria found in (10) are critical points of the RD (18).
According to the analysis in [30], we can show next that our Nash-Pareto pair ( ) , x y in the reduced RD (18) dynamics is not asymptotically stable, as whenever 2 0 P = and 4 0 P = we automatically get 1 3 0 P P = , and not negative.This is good in the applied context of our game, as we started our analysis with the goal of increasing vaccine coverage in our population, which means we wish to increase the probabilities of vaccinating for both types of players.While the mixed strategy point ( ) , x y is stable, we can next ask what can be done to nudge the population away from this state.In the next section, we use an optimal control problem associated with the reduced RD (18) where the control parameters are the vaccination costs 1 2 , c c .

Optimal Control of Vaccination Away from the Nash-Pareto State
We formulate the problem of controlling cost in the system (18) as an optimal control problem that is considered by Yosida in [31].Moreover, we are going to follow the same formulation as in [29] to build up a classical optimal control problem where we want to minimize the following functional problem: , L x t u t is the Lagrangian function defined as And the set ( . Problem Ω is formulated subject to the following conditions: , satisfiy an O.E.D system . .0, 0 , , where . Journal of Applied Mathematics and Physics In our case, we define the control will reduce the cost of vaccination in our game (1) and in our RD system (18) as follows: ( ) ( ) and : . 1 1 Thus, we study the following problem (for details see [29]): the system 18 and 1, 2 , . . , The proof of existence and uniqueness of the optimal solution ( ) , , x y u for the problem * Ω can be found in [32].We solve the problem * Ω numerically by using the steepest descent method to determine the first optimality conditions to * Ω (for more details see [33] and [29]).
In all our simulations below, we consider . Also, we fix below the values of 1 2 0.5 L L = = to be the same. In

{ }
1, 2 i ∈ and we find that the variation in costs has a big impact onto the probability of getting vaccinated in each group, as well as on the expected coverage in the population, assuming both groups of players (of type 1 and of type 2) are equally sized.
We see that after 6 T = time periods, we get that for initial values of costs: ( )   , thus the optimal control applied in this case, with equal population fractions, results in higher expected coverage for the vaccine.In this case, the application of the control over a number of 6 time periods (years) resulted in a net growth of expected vaccine coverage of approximately 0.12% (see Figure 3).with equal size groups.
The application of the control over a number of 6 time periods (years) resulted in a net growth of expected vaccine coverage of approximately 0.12%.

Minority-Majority Perspectives
The comparison of coverage levels can, in fact, be conducted for any pair of ( ) , c c in our analysis.The most desired optimally controlled states are those where the control leads to better coverage than the Nash-Pareto case.Moreover, our analysis can be easily expanded to two more cases: 1) players of type 1 are a majority (recall these are players with income level 1 I ; since 1 2 I I < , then we have a population where the lower income group is a majority); 2) players of type 1 are a minority.
We present the difference these proportions make on the discussion of our results from the perspective of the overall vaccine coverage.First, let us consider that 1 0.7 =  and 2 0.3 =  . In this case, we see (Figure 4) that the best coverage after applying the control results in a level of 83.47% for 1 0.3, c = 2 0.1 c = which is higher than the corresponding Nash-Pareto coverage level for which has a level of 75.38%.In this case, the application of the control over the same number of years resulted in a net growth of expected vaccine coverage of approximately 0.09%, and this is directly due to the population makeup and groups sizes (check Figure 4).Last but not least, we which has a level of 75.38%.In this case, the application of the control over the same number of years resulted in a net growth of expected vaccine coverage of approximately 0.03% (see Figure 5).

Conclusions
In this paper we presented a 2-player asymmetric bimatrix game with two pure strategies, vaccinating or non-vaccinating, against a potentially infectious disease.
We transformed and associated with our game a replicator dynamics system whose only mix strategy state is a Nash-Pareto stable pair.This means that over time, this mixed strategy of vaccination is likely to endure.From a public health perspective, the time-stability of the mixed Nash-Pareto pair is not desirable, as it is hoped that vaccine coverage can be increased in a population.
We showed that by introducing an exogenous control to decrease the perceived costs of vaccinating to all individuals in the population, an increase in the expected vaccine coverage can be achieved.We also showed that the net growth in the expected coverage depends on the population mix of players of type 1 and type 2, which is to be expected, as their vaccinating strategies are different.
Our game can be easily generalized in at least two directions: first, we can Journal of Applied Mathematics and Physics Second, we can introduce differing mechanisms for cost control.Our work here presents the basics of how such a multiplayer vaccination game can be analyzed and controlled, with the scope of gaining net increases in overall expected vaccine coverage levels.

Figure 1 .,
Figure 1.The blue curve shows the maximization of 1 E w.r.t the coordinates of ( ) * * 1 1 , x y .The orange curve shows the maximization of 2 E w.r.t the coordinates of ( ) * * 2 2 , x y .The point ( ) * * 3 3 , x y is the intersection of the two colored curves.

Figure 2 ( 6 T
upper panel), we present the Nash-Pareto pairs ( ) = , as functions of the initial group costs 1 c and 2 c , which were evolved, correspondingly, from the initial values of ( )

.
Though group 2's probability of vaccinating has slightly decreased, group 1's has increased.This is an advantage when we look at these values from the point of view of the expected coverage in the population, which is roughly estimated as (without considering time lags between vaccination and vaccine uptake):

Figure 2 .
Figure 2. In Figure 2 (upper panel), we present the Nash-Pareto pairs ( ) * * 3 3 , x x as functions of the perceived costs 1 2 , c c , while in Figure 2 (lower panel) we present the optimally controlled strategies ( ) * 3x T and