Mixed Strategy Nash Equilibria in Signaling Games

Signaling games are characterized by asymmetric information where the more informed player has a choice about what information to provide to its opponent. In this paper, decision trees are used to derive Nash equilibrium strategies for signaling games. We address the situation where neither player has any pure strategies at Nash equilibrium, i.e. a purely mixed strategy equilibrium. Additionally, we demonstrate that this approach can be used to determine whether certain strategies are part of a Nash equilibrium containing dominated strategies. Analyzing signaling games using a decision-theoretic approach allows the analyst to avoid testing individual strategies for equilibrium conditions and ensures a perfect Bayesian solution.


Introduction
Signaling games are a class of games with incomplete information.We use the tools of decision theory to provide a process for analyzing signaling games and divide the solution process into two phases: 1) strategy selection-determining the strategies for each player that are chosen at equilibrium; and 2) equilibrium calculation-finding the percentage of each time the players should choose the selected strategies.We address this second task by solving for a purely mixed strategy Nash equilibrium.Additionally, we suggest how the approach used to solve for a purely mixed Nash equilibrium may assist in the first task, that of identifying the strategies that form an equilibrium solution.

Background
A basic signaling game has two players.Player 1 (the "Sender") has private information about her type, while Player 2 (the "Receiver") does not know the type of Player 1.However, Player 2 knows the population distribution of types of Player 1. Player 1 sends messages that Player 2 receives.Player 2's actions depend on his beliefs about Player 1's type.In a classic example of a signaling game [1], education is a signal that can be obtained by workers of both high and low skill-level.Milgrom and Roberts [2] subsequently define a limit pricing game where an incumbent firm may temporarily charge lower prices to signal that the market is unpro-fitable.Descriptions of additional applications of signaling games have been provided by Kreps and Sobel [3] and Riley [4].
Previous research has been devoted to finding efficient Nash equilibrium solution algorithms for extensive form games. Von Stengel [5] summarizes much of the research on equilibrium calculation in extensive-form games as part of a text on algorithmic game theory [6].While much of this research has not exclusively focused on signaling games, some of the algorithms can be used to calculate Nash equilibria in signaling games.For example, the Gambit software package developed by McKelvey et al. [7] can be used to solve signaling games.However, the developers state on the Gambit website that it is "...quite easy to write down games which will take Gambit an unacceptably long amount time to (solve)."This has been our experience in using this software to calculate Nash equilibria for signaling games.Thus, we feel there is a need to develop algorithms with heuristic approaches for determining Nash equilibria in signaling games, and this is the focus of our paper.
We suggest a method that combines modeling techniques from the fields of decision analysis and game theory.Previous research has combined game theory, decision analysis, and/or statistical risk analysis to address situations where the decision maker has an adaptive adversary, such as those in counterterrorism decisions.We mention some examples here.Rios Insua et al. [8] introduce a Bayesian approach to adversarial risk analysis where two or more opponents make decisions with uncertain outcomes.Decision trees and influence diagrams are used to simultaneously model the decisionmaking of each player.Parnell et al. [9] use decision tree and influence diagram models to illustrate a defenderattacker-defender problem where the United States selects actions both prior to and after a bioterrorism attack.Paté-Cornell and Guikema [10] use an influence diagram for the attacker's decision to provide input to an influence diagram model for the defender in a counterterrorism measures selection problem.
The three articles cited above are similar to our method in that they use decision-theoretic models to examine a strategic problem from the perspective of each opponent.Our research differs in that we seek a Nash equilibrium solution (as opposed to a utility maximizing solution) and limit our analysis to problems that can be framed as a signaling game.Our primary objective is to enhance the methodology available for solving signaling games.We are optimistic that achievement of this objective will allow signaling game methodology to be applied to important decision problems that involve strategic interaction.We primarily address situations where a purely mixed Nash equilibrium exists in signaling games.Toward the end of the paper, we examine how our method can be extended to games with other types of equilibria.
Most game theory textbooks classify the equilibria for signaling games into three categories-separating on the message, pooling on the message, and semi-separating equilibria where some types of players select mixed strategies [11, pp. 326-328].The typical analytical approach to solve for these equilibria is to assume that the equilibrium exists and then test whether either player has an incentive to deviate from the strategy or not.This process forces the game theorist to test each possible strategy pair for each of the three types of possible equilibria.In other words, the strategy selection task is by-passed and the equilibrium calculation task is simplified by testing only equilibria for which the mathematical conditions can be easily identified.In games with more than two types of players sending messages and/or more than two possible messages this approach can be tedious, time consuming and prone to errors.

Decision-Theoretic Approach
We suggest that a decision theoretic approach may provide a standardized process for analyzing signaling games that does not involve testing individual strategies for equilibrium qualities.Our approach can simplify both the strategy selection and equilibrium calculation tasks required to solve signaling games.This approach to solving signaling games uses the concept of Nash equilibrium.Thus, in the mixed-strategy equilibrium, each player acts in a way that makes other players indifferent between choosing among different actions.
Thus conceptually, our approach is not that different from the usual PBE (Perfect Bayesian Equilibrium) approach to solving signaling games [12].However, the actual solution process for PBE involves testing each possible strategy for "beliefs that are consistent with strategies, which are optimal given the beliefs" [11, p. 326].This circularity in the solution process precludes the possibility of using backward induction as a way of solving signaling games.However, in our approach we do use backward induction.
This paper uses a decision tree approach to model the two-player, n-type symmetric signaling game.The use of decision trees for representing problems of strategic interaction was introduced by van Binsbergen and Marx [13].In their model, a decision tree is constructed for each player.A player's own choices are modeled with decision nodes and the opponent's strategies are shown as chance variables.
Cobb and Basuchoudhary [14] modified the decision tree approach by modeling the choices of both players in each tree with chance nodes and using the probabilities in the trees to represent the strategies.This allowed use of the decision tree approach to solve the two-type signaling game; however, this prior research only discussed the determination of pooling, separating, and semi-separating equilibria.This paper extends the previous research by both addressing signaling games with more than two types of Senders and by finding a purely mixed Nash equilibrium (the equilibrium calculation task), which is one where neither player has any pure strategies.
We employ a decision tree representation of the signaling game to determine Nash equilibrium strategies for several reasons.First, the equations capturing the Nash equilibrium strategies are actually calculated while solving the decision tree, as opposed to being abstractly determined by analyzing the game tree.This is particularly advantageous in the signaling game because Bayes' rule is used routinely at the time the decision tree is constructed.Second, the Nash equilibrium conditions are intuitively apparent through inspection of the decision tree.Additionally, the decision tree representation is more easily expanded as the number of Sender types increases than a corresponding game tree representation.

Outline
After first introducing signaling games and notation used in the paper, we solve for Nash equilibrium strategies in a two-type signaling game.The purpose of this section is to illustrate how decision trees are constructed for signaling games.Next, we give general derivations of Nash equilibrium strategies in the n-type signaling game where both players select purely mixed strategies at Nash equilibrium.In general, for a strategy vector chosen by the Sender to be part of a purely mixed Nash equilibrium, the Receiver observing each possible message must be indifferent between all of its subsequent actions after assigning any dominated strategies for the Sender a value of zero.Similarly, for a strategy vector chosen by the Receiver to be part of a purely mixed Nash equilibrium, each type of Sender must be indifferent between transmitting each of its possible messages after assigning any dominated strategies for the Receiver a value of zero.
Later in the paper, we will discuss how to apply decision trees to signaling games where the players do not necessarily select purely mixed strategies at Nash equilibrium, i.e. we use decision trees to address the strategy selection task.In the final section, we discuss our results and describe the development of a comprehensive approach to determining all types of equilibria in signaling games as a direction for future research.

Preliminaries
This section outlines notation and definitions that will be used throughout the paper.
In the n -type signaling game, the Sender has n possible types, 1 and can choose one of n possible messages, 1 , from a discrete strategy set.The Receiver responds with one of n possible actions, 1 , that it chooses from its discrete strategy set once it observes the Sender's message.Descriptions and definitions of the parameters in the -type signaling game are shown below.P a m .We use a decision tree approach to identify Nash equilibrium strategies in signaling games.We first consider the case where neither player has any dominated strategies at Nash equilibrium and address the equilibrium calculation task.In this situation, the payoffs are structured so that there is an equilibrium where each type of Sender plays a mixture of all of its possible messages, so each Sender strategy satisfies 0 1 at Nash equilibrium.Correspondingly, the payoffs are also structured so that the Receiver plays a mixture of all its possible strategies after observing each message, which requires each Receiver strategy to satisfy 0 a jk By inspecting the decision tree models, we are able to intuitively observe the conditions that must exist for a solution to be a Nash equilibrium.The decision tree provides for easy calculation of the expected values for each player, which are ultimately used to derive the mathematical conditions necessary for a Nash equilibrium in purely mixed strategies.
Receiver's strategy after observing message , Payoff to the Receiver observing message and taking action

Signaling Game Example
This section analyzes a two-type signaling game.For now, we assume nature's selection of each Sender type is equally likely, i.e.Once the Receiver observes the signal, it decides whether or not to move Up or Down and this threat represents its strategy.If it observes the Left message, its r and Receiver are shown in T Up strategy is denoted by 11 a .If it observes Right, its To find its Nash equilibrium strategies, the Receiver rolls back the Sender's decision tree one level as shown in the left panel of Figure 2. Since all nodes in the tree are chance nodes, the roll-back procedure involves calculating expected values.For example, the 1 Sender observing 1 calculates its expected value as 11 12 at the chance node representing the Receiver action at the top of its tree in Figure 1.This expected value is placed in the rolled back decision tree in Figure 2 as the payoff on the branch representing the  The model shows that nature chooses the Sender's type.After learning its type, the Sender chooses its message.Whereas the "Type" node is a chance node, the "Signal" nodes are random strategy nodes as defined by Cobb and Basuchoudhary [14], as the probabilities at these nodes represent Sender strategies.These nodes are shaded to distinguish them from typical chance nodes.After its strategy is chosen, the Sender's payoff in the game is determined by the action chosen by the Receiver.Up  

   
( ) . The conditional probabilities for the Sender given the observed message in this model are rmined 's type dete using Bayes' rule as . To find its Nash equilibrium strategies, the Sender ack the Receiver's decision tree one level as shown rolls b in the right panel of In this section, we discuss the gene game.Decision trees similar to those used in the 2 n  case will be useful in demonstrating the conditions are required to establish a Nash equilibrium."Message" branch for the Sender.This expansion shows that the Send hooses from among possible messages.Once it selects a message, it which of the ossible actions has bee taken.An expansion of the decision tree for the other Sender types would appear similarly.
Rolling back the section of the decision tree in Figure 3  To solve for the mixed trategy Nash equilibrium strategies for the Receiver, we construct the 2 2  n n where n n n   matrix where I is the n n  identity matrix.To satisfy a Nash equilibrium, we need NWa = .
Thus, we want P  a 1 .Let and define . We solve in the equation Thus, the Nash equilibrium strategies for the Receiver are determined as provided these entries are all non-negative.When the Receiver plays the strategies in , the Sender cannot unilaterally change its strategy to earn a higher expected payoff.
The prior discussion describes how the decision tree uilibrium calculation in the case of a purely mixed strategy equilibrium.To aid in strategy selection, the vector can be examined to determine whether there are any dominated strategies at equilibrium.This is indicated when contains entries that are not strictly between 0 not invertible.In the -type signa e, if a r of an assig n s i pris gies in a Nash equilibrium.Since the Receiver can assume that the Sender will never play dominated strategies, it can a ccordingly and may find do ti strategies f iver.A well-defined algorithm for finding Nash equilibria where each player chooses some pure strategies and mixes over the remaining strategies is beyond the scope of paper and requires future research.An example of such a solution in the context of a specific example will be provided later  The marginal probabilities for the message observed by the Receiver are calculated as The conditional probabilities for Sender type given the message observed by the Receiver are calculated as where probability the Sender is given m ssage al containing onl diagonal, with the diagon y 1 j q .We QD  r m strategies, we have that .However, in cal equilibrium will not act culating the Nash ually need the j q 's and hence we will not need .The m will be used to calculate the expected value in the game for the Receiver.is n n  is analogous to the m and contains the Receiver's payoffs.It atrix from previous analysis of the Sender's tree, but is defined in a different manner.It can be viewed as a diagonal block matrix, .
r EV .At Nash equilibrium, the Receiver observing the message must be indifferent between each of its actions, so that The matrix N will be   , where Since Q  is invertible (as long as no 0 j q  ), this implies that NVD  m 0 , and hence we can ignore Q and directly solve for m .
We currently have 2 n n  equations describing our where b is defined as in the previous section.Thus, the Nash equilibrium strategies for the Sender are determined as 1 .
1 2 i j i , and

then will not
In the -type signaling game, if a Receiver observing an ominated re of the Receiver's payoffs, or perhaps because t Receiver can assum pe of Sende ominated strategies that will never be played at Nash equilibrium.One su case mentioned earlier occurs when the Receiver has multiple actions that lead to exactly the same (this cause to be invertible).All but one of suc strategies ca considered dominated and assigned zero probabilities.As stated in the previous ditional di furt ods for using decision trees to solve for such equilibria will be the subject of ongoing research.
The same conclusions to those in Section 3 for the type signaling game about the solutions G will have two identical be invertible.ns and n ch payoffs h y message has any d strategies, these should be assigned as zero in any Nash equilibrium solution.This may occur simply because of the structu he e that at least one ty r has d s G n be section, some ad scussion will be provided later in the paper regarding the use of decision trees to identify such equilibria, and her study of meth two- a and  m outlined above can be made for the -ty lin he solutions either compute a purely mixed N i ndicat Nash equili

Examples
ind Section 3 with n pe signa g game.T ash equilibr um or i e that a brium containing at least some pure strategies exists.
In this section, the process from Section 4 is used to f Nash equilibri strategies in the signaling game from um 2 n  .

rm
. ting the values from T gives Up to this point, we have two equations for four un d in the matrix N .
knowns.The sum of the jk a should be one (when j is fixed This is ensured by the matrix P .Thus, we have hat 11 12 1 . Combine Equations ( 4) and ( 5) together.The matrices NW and P are both 2 4   , and hence the matrix The Nash equilibrium stra gies are   T 0.5 0.5 0.3 0.7 .  a

Determining the Sender's Nash Equilibrium Strategies
This section illustrates the determination of the Sender's so The matrix V contains the payoffs ijk v and is also block diagonal, Finally, with invertible, an easily be found.

W if nee
The Nash equilibrium strategies are G m c ith this, the i q 's can be easily determined as can r ded.


 T 1 3 2 3 2 3 1 3 .EV EV  S , is row can be removed from th atrix ince 12 0 a  the second column of NW can be removed.The matrix is lthough it has selected 11 0.926 a a  and 13 0.074 a  at equilibrium).Since 11 m , the Receiver can never gain a higher payoff by assigning any positive probability , so this was set to 12 0 a  12 r a he othe prior to determ eq no purely mixed (the strategy selection task).The process for interpreting the results of Section 4 for an arbitrary signaling game where a purely mixed Nash equilibrium does not exist, then adapting the matrices representing the Nash equilibrium conditions is not well-defined and has only been demonstrated by example.Creating such an algorithm requires further investigation, but the decision tree approach appears to hold promise for developing a heuristic approach to determining a Nash equilibrium in signaling games.

Conclusions
The main contribution of this paper is to introduce a standard approach to constructing and solving the equa-ining t uilibrium strategies.This example demonstrates that the decision trees are still useful for identifying Nash equilibria that are t   tions required to determine a mixed strategy Nash equilibrium in a signaling game.Some existing algorithms in the literature can solve two-type signaling games very fast.The problem is that once the game becomes even slightly more complex, finding equilibrium becomes increasingly difficult.This is acknowledged, for instance, by the developers of the Gambit software package [7].
In some sense, the purely mixed strategy is the most general solution for the n-type signaling game.Naturally, many signaling games have payoffs structured so that certain strategies for either player are dominated.In these cases, the decision tree can still be used to identify the Nash equilibrium conditions and solve for a mixed equilibrium over the remaining strategies.The eventual goal of future research is to provide new heuristic approaches for solving signaling games that can be derived from the solutions to the equations for the mixed strategy Nash equilibrium.This potential was demonstrated in Section 5.3 through an example where the Sender has a dominated strategy.
Decision trees provide a convenient acilitate the calculation of purely mixed Nash equilibria in signaling games.This paper extends decision tree results for the two-type signaling game presented by Cobb and Basuchoudhary [14] by finding general results for a purely mixed Nash equilibrium.Most game theory textbooks limit the possible types of solutions to pooling, separating, and semi-separating equilibria.The decision tree allows an analyst to simply compute a purely mixed Nash equilibrium, as opposed to testing a hypothesized Nash equilibrium by abstractly examining the payoffs or game tree for the problem.
The graphical representation of the decision tree models used to determine the purely mixed strategy Nash equilibria will clearly grow exponentially as the number of Sender types expands in the symmetric signaling game.However, the expected values required to solve such a decision tree are easily defined in matrix form.In a sense, we can dynamically construct nodes of interest in the decision tree, calculate expected values, then find the Nash equilibria using these expected values.This allows the decision tree solution to be used to find the purely mixed Nash equilibrium solution, even when the Sender as many possible types.
takes action k a

Figure 1 .
Figure 1.Decision trees for the general two-type signaling game.

Figure 2 .
Figure 2. Decision trees rolled back one level.

3. 2 .
Sen given by Pro b and B udhary p. 252].der's Nash Equilibrium Strategies shown es not The decision tree for the Receiver in this game is in the right panel of Figure 1.The Receiver do directly learn the Sender's type, but does observe whether it chooses Left or Right.Once it observes this message, it decides whether to move Up or Down.The "Action" nodes are shaded to indicate these are random strategy nodes (as opposed to typical chance nodes).The payoff to the Receiver in the game is then determined by the Sender's actual type.The marginal probabilities for the Sender's message type are calculated as

Figure 2 . 1 . 1 ) 2 ) 1 m 12 m 2 make 4 .
For example, the Receiver observing 1 m and taking action 1 a calculates its expected value as Copyright © 2013 SciRes.node representing the Sender type at the top of its tree in Figure This expected value is placed in the rolled back decision tree in Figure 2 as the pay ff If the Re ve cei r observes 1 m , it must be indiffere tween the strategies 11 a and 12 a .If the Receiver observes 2 m , it must be indifferent between the strategies 21 a and 22 .These conditions are met when the expected values at the end of either branc a h at the two Action nodes in the decision tree in the right panel o Figure 2 are equal, or when  , and 22 m  determined using the process outlined above are only guaranteed to 11 m the Receiver indifferent between any assignment of its strategies if none of its strategies are dominant.Recall the example from the last section where the Sender's payoffs are such that it always plays and Receiver in the decision trees and rolling back the trees, the Nash equilibrium strategies for a semi-separating equilibrium can be determined.The solution for the Receiver observing 1 m is given by Proposition 2 in Cobb and Basuchoudhar 14, p. 252].In this next section, decision trees and the expecte General Signaling Game aling lues obtained from the solution process are used to obtain a pure mixed strategy Nash equilibrium for the n -type signaling game.

Figure 3 Figure 3 .
Figure 3 shows a portion of the decision tree fo Sender in this game.The model is expanded beyond the

4 . 2
Thg This sh eyond the "A tion" node when the Receiver observes message .The detail shows that the Receiver will first learn the m ssage selected from the Sender's possible y dominated strategies, these should be ned zero probabilities prior to finding the rema ing strategie n a that com e the Receiver's djust its strategies a minated strategies of its own.The decision tree methodology can s ll be useful in determining Nash equilibrium or the Rece this in this section.. Analysis of Receiver's Decision Treee Receiver's decision tree in the general n -type signalin game is partially shown in Figure 4. ou We have yet to e fact use th jk a sh diagram ows the detail in the tree b c 2 e n m s.O n ould be one (wh is fixed).T en j he 2 n n  takes this into account: matrix P in the general signaling game.

Figure 4 .
Receiver's decision treeobserves the message, it chooses an action from its n available possibilities.Only after the Receiver selects an action does it finally learn which of th possi Sender types it opposes in the game.In this etc.One ca onal, n view Q as being block diag

2 n
unknowns.Our final n equations come from the the way that e have indexed this amounts to summing every en and getting 1.This can again be atrix m matrix which is the identity to solve for m in the equation We can show an example of the conditions under which will not be invertible.If for two values of say , we have that , G  m b determine its equilibrium strategies, the Sender must adjust the Receiver's decision tree for the conditions and .After adapting the process in Secdded to the vector.The decision tree for the Receiver 1 own in Figure 6 w h the timal strategies and resulting conditional probabilities for b inserted.The Sender's strategies make the Receiver indifferent between any possible assignment a m

Figure 6 .
Figure 6.Decision tree for the Receiver observing m 1 .

1
Sender and the 1 observation.Other expected values are calculated similarly during the roll-back process.

Table 1 . The payoffs to each player in the signaling game.
These conditions are met when the expected values at the end of either branch at the two Signal nodes in the decision tree in the left panel of Figure2are equal, or

m 5.3. Nash Equilibria without Purely Mixed Strategies
 

Table 2 .
T Sender may be one of three types and the Receiver has three possible actions.