Evaluating the Use of Role Playing Simulations in Teaching Negotation Skills to University Students

This paper critically evaluates the use of role-playing simulations in a negotiation course taught to graduate students. The course consisted primarily of a series of simulations involving the alternative dispute resolution (ADR) processes of negotiation, facilitation and mediation. Data were obtained from two sets of questionnaires completed by 41 students before and after the course. A review of previous research reveals that despite the widespread use of role-playing simulations in education, there has been very little empirical evaluation of their effectiveness, especially in conflict resolution and planning. Comparison of the data acquired from the two surveys generated findings regarding student understanding of ADR processes and key issues in conflict resolution; the educational value of simulations; the amenability of types of planning and planning goals to ADR; appropriate learning objectives; the importance of negotiation skills in planning; challenges in conducting effective simulations; the value of simulations in resolving real conflicts; the utility of negotiation theory; and obstacles to applying ADR to planning disputes. More generally, the paper concludes that role-playing simulations are very effective for teaching negotiation skills to students, and preparing them to manage actual conflicts skillfully and to participate effectively in real ADR processes. However, this technique is somewhat less valuable for teaching aspects of planning other than conflict resolution. Surprisingly, prior experience with simulations had no significant influence on the responses to the pre-course survey. Also surprising was the lack of a significant correlation between final exam scores and responses to relevant questions on the post-course survey.


Introduction
The educational use of role-playing simulations has become popular in many diverse fields.However, few, if any areas of instruction have adopted this modality of teaching to the same extent as conflict resolution.So ubiquitous has this technique become in this field that a course on negotiation or mediation that did not employ at least some component of simulation would stand out to participants as unusual and would likely be perceived as lacking credibility.Since the success of negotiation and mediation depends in large part on complex interpersonal dynamics, simulations are extremely valuable to illustrate concepts first introduced at a theoretical level but only fully grasped through "learning by doing" (Schön, 1983).
Despite the widespread use of role-playing simulations in education, there has been very little empirical evaluation of the effectiveness of this style of learning.While most educators accept the unique contribution that simulations make to learning in a broad range of fields, published literature addressing its merits has been predominantly descriptive and anecdotal.
The purpose of this paper is to critically and empirically evaluate the use of role-playing simulations, based on two questionnaires completed by 41 Master's level students enrolled in a short (four weeks) course on negotiation.The course consisted primarily of a series of role playing simulations involving negotiation, facilitation and mediation.One questionnaire was administered just before the course began and the second at the end of the final session.The survey results are used to examine the effectiveness of the simulations from the perspective of the student, in terms of benefits accruing from this teaching device, strengths and weaknesses of various aspects of the simulations, and the appropriateness of simulations to various components of planning education.Three types of statistical analysis are applied to these data: comparing the pre-and post-simulations responses, comparing the responses of students with and without prior experience with role-playing simulations, and testing for correlation between responses to selected questions and scores on the final examination for the course.

Previous Research
Role playing has long been recognized as an effective means of learning in various fields, especially those reliant on interpersonal relations (Krause & Amaral, 1994).Innes and Booher (1999;Meligrana & Andrew, 2003) also point out that simulation is a legitimate tool of analysis.Yet, there has been very little published research on the advantages of using simulations in an educational setting, nor on evaluating their effectiveness (Krause & Amaral, 1994;Petranek et al., 1992).This section summarizes previous research regarding role-playing and simulation games.It begins with a description of the commonly held beliefs regarding the benefits of simulations and a review of the scant empirical research into these beliefs.The discussion then turns to how simulations have reportedly been used to teach planning and negotiation.

Educational Value of Role-Playing Simulations
Contributions that simulations can make to teaching that are frequently cited include easier learning of subject matter (especially complex concepts) (Walford, 1981); better long-term recall of material learned (Innes & Booher 1999;Petranek et al., 1992;Randel et al., 1992, Walford, 1981); greater student interest, motivation and enjoyment (Randel et al., 1992;Walford, 1981); the fostering of a more appropriate teaching environment (Walford, 1981); and the improved development of problem-solving skills (Walford, 1981).Nightingale (1981) believes that the value of simulations and games as teaching tools lies in their emphasis on decision making processes and the myriad of human factors that influence them.They also appropriately capture the concept of the uncertainty of the outcomes of these processes.
However, the literature offers little empirical evidence to support these claims (Walford, 1981).Much of the published evaluation of simulations and games is descriptive and anecdotal, with a few notable exceptions (Randel et al., 1992).Early research (e.g.Cherryholmes, 1966) found no evidence that simulations are more effective than other forms of learning.A decade later, Pierfy (1977) reviewed 22 evaluations of the effectiveness of simulations and games and concluded that simulations are neither better nor worse than other methods of teaching.The exception was that simulations are better at changing attitudes and stimulating participant interest.More recent research (e.g.Bredemeier & Greenblat, 1981;Foster et al., 1980;Hankinson, 1987) suggests that simulations may provide better behavioural, cognitive and affective learning, and subjective understanding of social issues.Randel et al. (1992) compared the effectiveness of 46 cases of simulations or games reported in the social sciences (which would include planning) literature with traditional classroom teaching.They found that in 33 (or 72 percent) of the cases there was no significant difference in the instructional effectiveness of the two methods.In ten cases (22 percent), simulations/games were more effective, while the remaining three cases favoured conventional methods.

Debriefing Simulations
Most of the literature (e.g.Dolin & Susskind, 1992;Ryan 2000;Walford, 1981) maintains that a well-structured debriefing session with the participants is an essential component of an effective role-playing simulation.Debriefing refers to the discussions among the participants about what occurred during the simulation, and what lessons may be gleaned from the experience.Such discussions are usually led by the instructor, using a set of questions to stimulate and guide the dialogue.Krause and Amaral (1994) observe that it is essential to evaluate the effectiveness of any role-playing exercise by carrying out a post-simulations evaluation.This is often done in conjunction with a post-game debriefing, in which most of the learning usually occurs (Krause & Amaral, 1994).Although Krause and Amaral (1994) carried out such a formal evaluation of a simulation debriefing with students, Randel et al. (1992) observe that there has been to date no systematic analysis of the education merits of simulations with and without post-simulations debriefing.Bredemeier and Greenblat (1981) and Szafran and Mandolini (1980) provide empirical evidence to support their dissenting opinion that debriefing does not enhance learning.Innes and Booher (1999) argue that role-playing simulations can be useful to train planners to act in a more cooperative, con-sensus-based manner.It helps planners to think more creatively, and to respond more flexibly to unforeseen and complex circumstances.However, simulations are unlikely to be effective unless carefully integrated into a program of study which includes other, more didactic modes of learning (Innes & Booher, 1999).

Use of Simulations in Teaching Planning
The use of simulations has considerable potential to remedy what Baum (1997) and others identify as a significant weakness of most university planning programs that was identified by Baum (1997) and others (e.g.Friedmann & Kuester, 1994;Sawicki, 1988).This result in many graduates being inadequately prepared to enter the profession, they argue.Furthermore, most programs equate planning "methods" with the more academic activities of analysis and research, rather than the important skills of communicating and interacting effectively with others, and using knowledge to influence others.Although Baum does not call them such, the latter are essentially negotiation skills.Learning them requires hands-on training, which most programs lack, but which the inclusion of simulations as a teaching tool could provide.As Baum (1997: p. 182) writes: "It is impossible to learn to practice planning without practicing: trying out ways of acting, analyzing the results, designing new actions at least as likely to produce desirable results, and so forth."Shepherd and Cosgriff (1998) agree with Baum that academic teaching of planning poorly prepares students for careers as planners.What students lack, they believe, are well-developed problem-solving skills.They endorse "problem-based learning" (which in other fields such as medicine often includes role-playing simulation) to correct this deficiency.
Simulations could also help students learn about the "ethical, intellectual, emotional, and practical predicaments" (Baum, 1997: p. 185) that are fundamental to planning, and give them the opportunity for the necessary post-experience reflection on their actions and behaviours.Use of simulations (especially the post-simulations briefing session) in teaching is consistent with Schön's (1983Schön's ( , 1987) ) seminal writings on the "reflective practitioner".Schön refers to knowledge gained through the process of problem-solving as "procedural knowledge" or "knowingin-action".More conventional methods of teaching emphasize the less important "declarative knowledge": knowing what we know without knowing how to employ that information.Dolin and Susskind (1992) discuss the use of simulations not to teach planning, but to assist disputants in actual planning disputes.Parties that are reluctant to come to the bargaining table may be willing to participate in a simulation of the conflict in which they are embroiled.There are examples of this application of simulation, in which it was successful in convincing the disputants that negotiation had the potential to resolve their dispute by allowing the parties to each satisfy their principal interests.Dolin and Susskind (1992) examine one such case, in which simulation catalyzed disputants in a contentious national energy policy issue to engage in a large-scale consensus-building exercise, which ultimately resolved the conflict.Several of the more popular commercially distributed simulations in the planning field were originally created to educate parties in actual disputants.One example is the "Harborco" game used in this course, which was designed by the MIT-Harvard Public Disputes Program in the early 1980s to assist disputants in a long-standing controversy over the proposed construction of a deepwater port in the Lower Delaware Bay.
The only example the authors found of the evaluation of the use of role-playing simulations to teach planning was Krause and Amaral (1994).They ran and evaluated a harbor management simulation to teach about public participation methods in planning.They also correlated student opinions of the simulation (such as their satisfaction and its effectiveness as a teaching tool) with their exam scores (as did we).However, Krause and Amaral (1994) only looked at student perceptions of the simulation (measured by a few variables only), their level of participation and their performance on the exam.They found that students who actively participated in the simulations performed better on the exam.The degree to which the students engage themselves in the simulation is important because there is evidence that this correlates well with the amount learned (Petranek et al., 1992).Krause and Amaral also found that both the exam scores and a brief post-simulations survey of student satisfaction indicated that the simulation had been successful.

Use of Simulations in Teaching Negotiation
One application of role-playing simulations that has become fairly common is in teaching negotiation.However, there has been very little discussion in the literature about how to teach negotiation effectively.Lewicki (1986) believes that role-playing simulations are effective in teaching students how to be good negotiators because developing these skills requires both academic training and the development of a set of behavioural skills.Simulations provide the instructor with the opportunity to combine teaching of theory with students actually practicing with various behaviours (Lewicki, 1986).However, Schultz (1989) is critical of most negotiation training programs for not basing their instruction on the well-established body of theory on the subject.Simulations allow participants to experiment with various negotiation styles and techniques, and to receive feedback from others in a low-risk environment where there is no actual dispute or any relationships at stake (Lewicki, 1986).A search of the literature for evaluations of negotiation role-playing simulations revealed a single example.Schultz (1989) studied four negotiation training programs (throughout North America), and found that none monitored the extent to which the graduates were effective conflict resolvers.All measured the success of their program solely on the basis of questionnaires completed by students immediately on completion of the course, in which the success of the simulation was measured only by the students' satisfaction with the course.
In summary, despite widespread acceptance of role-playing simulations as a teaching tool in a broad spectrum of fields, there is a clear need for empirical evaluation of their strengths and weaknesses.This is particularly true in the fields of conflict resolution and planning, which the course on which this study is based attempted to fuse.

Method
Five role-playing simulations were conducted in the course, which consisted of seven sessions in four weeks and was taught by the primary author.The simulations were all purchased from Harvard University's Program on Negotiation Clearinghouse.All were known by the instructor to be effective, frequently-run simulations dealing with urban planning issues.Three of the simulations involved negotiation and one each dealt with the negotiation-based processes of facilitation and mediation.These are the three principal processes comprising Alternative Dispute Resolution (ADR), a spectrum of voluntary, negotiation-based processes in which representatives of the parties to a current or potential dispute meet together for collaborative problem solving and consensus building, with the goal of achieving a mutually acceptable resolution.The processes are intended to be less adversarial alternatives to traditional conflict resolution pathways.Negotiation is simply face-to-face consensus building between parties working co-operatively to achieve a mutually acceptable resolution, without the services of a neutral party such as a facilitator or mediator.Neither the facilitator nor the mediator has the authority to impose a settlement on the disputants, but they differ in their degree of involvement in the consensus building process.The facilitator is a process manager whose mandate is limited to procedural issues.Mediators likewise deal with procedural matters; in addition, they shuttle ideas and offers back and forth between the parties, help each party to formulate proposals that are more likely to be acceptable to the others, participate in the generation of creative options, and assist in writing the final agreement.The mediator may provide each party with a confidential and independent assessment of its position early in the mediation process, as well as help it to see the true interests underlying its positions.
The first simulation, called "Bradford Development," was a two-party negotiation about a single issue: a proposed "linkage" payment that a municipal government would charge developers to compensate for impacts of development on infrastructure and housing.The conflict and settlement reached concerned the magnitude of the payment.Students negotiated in pairs (representing the city and the developer), four to a game, with ten different games running simultaneously.The next simulation, "Negotiated Development at Redstone," was also a two party negotiation with four participants to a game.Representatives of a developer and a neighbourhood group negotiated two variables of a proposed condominium development.Each party was allocated a performance score based on the values of the two variables in the agreement."Harborco" was a five issue, scorable negotiation over the proposed construction of a deepwater port.Seven games ran simultaneously, with six parties in each."A Development Dispute at Menehune Bay" was a seven party facilitation (including facilitator), with five different games.It involved three issues associated with the proposed development of a resort hotel.It was partially scorable, in that performance was judged by ranked priorities rather than points that could be summed.The final simulation was a three issue, scorable mediation over a proposed homeless shelter, called "Westville."It involved three parties (including mediator) playing thirteen simultaneous games.
In general, the simulations increased in complexity as the course progressed (as suggested by Lewicki, 1986).The number of parties (or roles) in each simulation ranged from two to seven.Therefore the number of simulations being carried out simultaneously varies inversely with the number of roles in each, to allow each student to participate in each simulation.Multiple rooms were available to allow each simulation to be carried out essentially in isolation.
A set of readings compiled by the instructor was a mandatory component of the course.There were approximately three short readings selected for each session, intended to provide theoretical background on negotiation prior to seeing various principles and lessons applied in practice in the simulations.Therefore, students were advised to have read them prior to each session.Prior to the commencement of the course, students were required to have read Fisher, Ury and Patton's (1991) popular book Getting to Yes: Negotiating Agreement without Giving in.
Data for this study were obtained from two sets of questionnaires completed by all 41 students in the course: the first in the few days prior to the course beginning, and the second at the end of the final session.The questionnaires combined three types of questions: ordinal, ranking, and open-ended.Ordinal questions asked the students to respond to the question by choosing an item from a five point ordinal scale, ranging from excellent to poor, or from strongly agree to strongly disagree, or extremely important to unimportant.
Ranking questions asked respondents to rank-order the importance of five items (e.g.learning objectives), from one for the most important to five for the least important.For most of the first two types of questions, the number of responses was between 39 and 41.For a very few questions, there were 37 or 38 responses.Open-ended questions asked respondents to write in answers (e.g.identify other learning objectives not in the list of five in the ranking question).For all but one of the questions, the number of responses ranged from 10 to 33, and averaging 20.The one exception was the final question on the pre-simulations questionnaire, which invited comments on any aspects of the simulations not already covered.
The only question that differed from these types was the first question of the pre-simulations survey, which asked the students if they had participated in any type of role-playing simulation of a conflict resolution process before.This question was asked for two reasons.The first was to test whether experience with simulations prior to this course influenced respondents' answers to questions on the pre-simulations survey, compared to respondents lacking prior experience.The second was to determine whether the simulations in the course were still of educational value to students with prior experience.Based on prior experience and discussions with students, it was assumed (correctly) that a significant number of students would have prior experience.
The pre-simulations questionnaire contained 26 questions; the post-simulations 33.To permit the researchers to use before-after comparisons to evaluate the educational contributions of the simulations, 25 of the questions were common to both questionnaires, with slight wording changes to match the timing of the questionnaire.One question was unique to the pre-simulations questionnaire; and eight to the post-simulations.
After the questionnaires were completed by the students, the responses to the ordinal and ranking questions were coded and entered into a database.For the ordinal questions, the chosen response was coded from one to five.A value of one was assigned to the first choice, which was also the most positive (e.g."excellent", "strongly agree" or "extremely important").A value of five was assigned to the least positive choice.In other words, a lower value was usually "better."The open-ended responses were transcribed and grouped by question.The Statistical Package for the Social Sciences (SPSS) was used to generate frequency tables, means and standard deviations for the ordinal and ranking data.Statistical analysis was applied to some of the ordinal and ranking data, where visual inspection of the frequency tables and means suggested that there might be significant associations between relevant pairs of data.In these cases, statistical tests were used to detect the following conditions: Whether there was a significant difference between the "before" and "after" responses to the 25 questions common to both surveys.
Whether there was a significant difference in the survey responses and scores on the exam taken at the end of the course of students with prior experience with role-playing simulations and those without.
Whether there was a correlation between exam scores and responses to nine of the ordinal questions on the post-simulations questionnaire.
The before/after comparisons (1 above) employed the Wilcoxon test for the ordinal questions and the t test for the ranked questions.
For the prior experience/no prior experience comparisons (2 above), the data were first sorted according to whether the respondent had previously participated in a role-playing simulation of a conflict resolution process (Question 1 on the pre-simulations survey).Frequency tables of the responses to each subsequent ordinal and ranked question on the pre-simulations survey were produced for each of the two set of respondents, along with means and standard deviations.The Kolmogorov-Smirnov test was then applied to the data for each pre-simulations question, to detect any significant differences between the answers of each set of respondents.
Spearman correlation analysis was applied for comparing the exam scores to the other relevant questions (3 above).
For all tests, the null hypothesis was that there is no significant difference between the two sets of data.This was rejected only if the p value was calculated to be less than a probability level of .05.For the Wilcoxon test, this equated to rejection of the null hypothesis when the z score is in the range of -1.96 to +1.96.Rejection of the null hypothesis allowed us to reasonably conclude that the two sets of data are statistically different.

Findings
This section summarizes the key findings generated from analysis of the responses to the pre-and post-simulations questionnaires.This analysis includes meaningful patterns observed in the data; as well as statistical analysis comparing the responses contained in the two sets of questionnaires, comparing the responses of students with prior experience with negotiation simulations to those without, and testing for significant associations between course exam scores and relevant survey questions.

Understanding of ADR Processes
In both surveys, students were asked to assess their own level of understanding of each of the three conflict resolution processes taught in the course: negotiation, facilitation and mediation.The frequencies of the responses for these three sets of questions, with their means and standard deviations, are shown in Table 1.Also shown are the results of the Wilcoxon test for association between the pre-and post-simulations responses in each pair of questions.The Wilcoxon test employs the raw data on each respondent's answer, rather than the grouped data in the frequency table.In other words, it compares each person's preand post-simulations responses.For all three pairs, the Wilcoxon test indicated a statistically significant difference between the way in which the same set of respondents answered each of these three questions before and after their participation in the roleplaying simulations.Overall, respondents rated their understanding of each of these three conflict resolution processes significantly higher at the end of the course than before.Both pre-and post-simulations, negotiation was the best understood by the students, and facilitation the least.While the means indicate that the change in level of student understanding was slightly greater for negotiation, the Wilcoxon results suggest that facilitation

Overall Educational Value of Simulations
The finding that respondents believed that the simulations significantly improved their understanding of negotiation, facilitation and mediation is consistent with their high appraisals of the overall educational value of the simulations.Although this assessment improved slightly after the course (mean of 2.05; slightly less than "very good"), the students also anticipated that the simulations would be valuable for learning (mean of 2.10).It was apparent that there was no significant difference between the pre-and post-simulations responses.It is noteworthy that this is in contrast to the relatively large changes in students' levels of understanding of all three conflict resolution processes.
Respondents were asked (on the post-simulations survey only) to rank the five role-playing simulations used in the course according to their educational value (1 for the most valuable to 5 for the least).These results are shown in Table 2.The highest and lowest ranked simulations were both negotiations.Although the clearest trend appears to be that students felt that simulations with a greater number of roles or parties were the best, this is likely confounded by the Harborco and Menehune Bay simulations (with six and seven parties, respectively) also being the best designed of the five, which meant they ran more smoothly.Students seemed quite sensitive to how well each simulation functioned, and this was reflected in their level of satisfaction.
The three negotiation simulations had an overall mean of 3.05, compared to the facilitation mean of 2.38 and the mediation mean of 3.38.Therefore, this ranking of facilitation as having the greatest educational value and mediation the least was completely consistent with the Wilcoxon test results for the respondents' level of understanding of the three conflict resolution processes.However, these findings are quite uncertain, given that there was only one simulation for each of facilitation and mediation.Following up on the preceding question, students were asked (in an open-ended question on the post-simulations survey only) to indicate what was particularly valuable about the simulation to which they had assigned the top rank.Comments about the overall highest-ranked "Harborco" simulation included its diverse interests, its emphasis on searching for common interests, its high level of realism, its large number of interests (six parties) and issues, and its provision of opportunities to form coalitions of parties.As one student wrote: "There were multiple interests and it was more realistic/complicated than the others, [which] were too constrained by the roles to be fully useful." Students who picked "Menehune Bay" as the most valuable commented on its multiple parties (seven), the fact that it was the most complex of the simulations, the involvement of a facilitator (the previous three simulations had been negotiations), and its flexibility to invent options in order to meet as many of the parties' interests as possible.The latter is one of the "Getting to Yes" principles.This was the first simulation in which they were not limited to fixed options when assembling packages of agreements, which made it more realistic.According to one student: "It was not based upon points therefore it enabled us to expand the pie and have real reason to explore underlying interests."

Value of Simulations in Teaching Planning Concepts
Within the same theme of the educational value of the simulations, six other questions were asked on the pre-and postsimulations surveys.Of these, two were open-ended and one was a ranking question.Asking the students about the usefulness of the simulations for learning about planning yielded a pre-simulations mean of 2.68 and a post-simulations mean of 2.85 (shown in Table 3).Both were slightly better than the "good" response.Surprisingly, the post-simulations questionnaires were slightly less enthusiastic.Slightly more positive but experiencing a very small change was the question whether the simulations would help the students create better environments as planners.As Table 4 illustrates, this question had pre-and post-simulations means of 2.20 and 2.18, respectively; both slightly less than "somewhat agree."A slightly greater positive change was observed for the question of whether the simulations would/did help the students to better understand different perspectives on urban development (present and future) held by various stakeholders (also shown in Table 4).
This mean changed from 2.10 to 1.87, yet still remained within the "somewhat agree" range.The slight before-after changes of these three questions, although varying in direction, are similar in magnitude to the very slight change already seen in the overall educational value question (whose mean changed from 2.10 to 2.05).The responses regarding the merits of simulations in learning negotiation skills and substantive knowledge in planning were remarkably consistent.

Amenability of Types of Planning and Planning Goals to ADR
Comparing the pre-and post-simulations surveys, there were few differences in how respondents ranked five planning goals according to the extent to which they could be achieved using ADR.Although the only statistically significant change (as detected by the t test) in mean ranking was observed for the goal of "promoting and supporting planning as a profession," its reduced achievability had no effect as it was already ranked fifth on the pre-simulations survey.The only ranking change was a switch between first and second places, with "making policies and plans more equitable" moving to the top rank and "improving the communication of complex planning concepts" dropping to the second rank.The third and fourth most achievable planning goals were "making policies and plans more efficient" and "improving planning education", respectively.Overall, the experience with the simulations did little to change the students' minds about the achievability of planning goals using ADR.
The post-simulations survey asked participants to rank five types of planning according to their amenability to the use of negotiation-based conflict resolution processes.The results may not be very useful as they appear to reflect the types of planning involved in the five simulations themselves.In other words, students ranked land use, environmental and community as the topthree most appropriate types of planning (in decreasing order of amenability), since these were the three types most featured in the simulations, in roughly the same order of emphasis.The mean rankings for these three items were very close together, ranging from 2.42 to 2.61.Social and heritage planning ranked fourth and fifth, with respective means of 3.37 and 4.13.The only anomaly is the low ranking of social planning, since the "Westville" mediation dealt with the opening of a homeless shelter.Heritage planning had a much lower mean than the other types of planning, reflecting its very minor role in the simulations.

Learning Objectives
The students were asked, both before and after the course, to rank five learning objectives for the simulations, according to their importance (before) and the extent to which they were achieved by the simulations (after).Rank one was assigned to the most important/achieved objective and five for the least).
Table 5 shows the pre-and post-simulations means and overall rankings of each objective.There was a considerable re-ordering of the objectives as a result of experience with the simulations."Learning how to choose the most effective conflict resolution approach to a dispute, and "learning how to determine one's own interests and needs in a dispute" switched places between the second and fifth ranks.These were the only two objectives for which the change in mean was determined to be statistically significant by the t test."Learning how to identify common interests and trade-offs" and "learning how to effectively communicate one's own interests and needs to other parties" experienced a less dramatic place-swapping of the first and third ranks."Learning how to better understand the interests and need of other parties" remained at the fourth rank.Clearly, the simulation taught the students the importance of "interests" in negotiation, with identifying common interests being of prime importance.Identifying and communicating one's own interests, and understanding the interests of other parties were of lesser importance.After the course, the selection of the optimal conflict resolution process was seen as much less important than before.Copyright © 2012 SciRes.
In both the pre-and post-simulations surveys, an open-ended question asked respondents to identify any learning objectives not among the five listed in the previous question.Prior to the simulations, the answers tended to be more general, such as learning how to negotiate to achieve one's own objectives, developing confidence in negotiating with others, learning to work more productively in groups, developing greater patience, learning how to put the theory contained in the readings into practice.Learning to concentrate on the issues in dispute rather than the personalities involved, and how to speak more effectively and diplomatically when negotiating were mentioned on both surveys.Respondents on the post-simulations survey focused on more specific skills developed in the simulations, such as how to understand and utilize one's own BATNA (Best Alternative To a Negotiated Agreement), how to build effective coalitions with other parties, how to effectively communicate ideas and interests between parties, how to build consensus by generating creative options that meet various interests, and how to develop trust and strong relationships between parties.
Students were asked to identify the single most important thing they expected to learn, and did learn, from the simulations (in the pre-and post-simulations questionnaires, respectively).As with the previous question (and some overlap with it was inevitable), the initial set of answers tended to be very general.Many emulated the language of "Getting to Yes".Typical comments included: negotiation and communication skills, how to meet one's own needs as well as other's needs, applying negotiation theory to real life conflict situations, how to work with difficult people to solve problems, how to persuade people, how to listen to understand other parties' viewpoints, and how to negotiate without getting emotional.One comment astutely summed up many students' expectations of what they would learn from the simulations, as "how to conduct negotiations in a manner that produces an agreement that is mutually beneficial to all parties."

Assessing What Was Learned
On completion of the course, students tended to identify more specific skills or concepts that they had learned, compare to their responses to the preceding question.Commonly mentioned examples included: the importance of discovering parties' interests behind their stated positions, how to identify and capitalize on common interests, how to trade-off interests for mutual gains, how to determine one's BATNA, how to effectively communicate one's interests to other parties, how to deal with emotional situations, how to creatively invent options without committing to them initially.One student made the interesting observation that "honesty is the most important part of reaching a mutually-beneficial agreement."Several others identified maturity and listening skills as critically important features.
The post-simulations questionnaire also asked students to identify the one most important thing they learned about planning from participating in the simulations.Many of the responses were quite interesting.A commonly mentioned theme was that planning disputes are more complex and difficult to resolve than they had realized, as they often involve many stakeholders with many seemingly-irreconcilable interests.Many involve conflicts over values, several people noted.Several others commented on the importance of understanding other parties' viewpoints.Another frequently mentioned lesson learned was the need to generate creative options that allow trade-offs of interests to be made.
According to one respondent: "Planning is about collaboration amongst all stakeholders".Although the importance of fostering and maintaining good relationships between parties was a recurring theme, one person noted that: "People's egos and prejudices can ruin a negotiation."One student observed that planners can seldom function as truly neutral mediators, picking up on one of the themes of the "Westville" mediation simulation.Finally, one student complained that s/he had not learned anything about planning because the simulations were "too fictitious" and they lacked sufficient background information and the flexibility to allow players to "adapt outcomes".
The post-simulations survey also wanted to know if the simulations helped to dispel any misconceptions previously held by the participants.One corrected misperception that several students mentioned was that most agreements satisfy all of the main interests of the parties, i.e. they're "win-win" (to quote an overused cliché).Another was that achieving a consensus agreement is easy if you follow the "principled negotiation" process outlined in books such as "Getting to Yes".Several students mentioned that they had underestimated the great differences between negotiation, facilitation and mediation.One was disappointed to discover that facilitators and mediators are not always entirely neutral, unbiased and impartial; nor do they always enjoy the respect of the disputants.The simulations dispelled the belief of at least a few students that it would be relatively easy to get everyone to engage in principled negotiation and not act in an adversarial manner.Another had previously believed that negotiation-based processes could easily control emotions, even in volatile disputes.Finally, several students had underestimated the importance of the BATNA concept after having read "Getting to Yes".

Importance of Negotiation Skills in Planning
Both the pre-and post-simulations results were very positive in response to a question asking how important it is for planners to develop effective negotiation skills.In fact, students felt that these skills were even more important after they had completed the course.The mean response changes slightly from 1.61 to 1.51.Although both were in the "very important" range, the post-simulations mean was almost closer to the "extremely important" response.In fact, 23 of the 39 respondents (59 percent) felt that negotiation skills were "extremely important" after they had participated in the five simulations.

Logistics of Conducting Simulations
The post-simulations survey (only) asked respondents to evaluate the quantity provided of three aspects of the simulations: the preparatory information provided prior to each simulation, the time allocated to the simulations, and the time allocated to debriefing the simulations.The two questions related to time were rated on a five-point scale from far too much to far too little.They had mean scores of 2.92 (slightly to the "too much" side of the "just right" response), and 3.00 (exactly equal to "just right"), respectively.The scale for the question about information ranged from very sufficient to very insufficient.Its mean of 2.00 represented the "somewhat sufficient" response.Students were less satisfied with the information they were given in advance the simulations than they were about the timing of the games.Since pre-packaged simulations were used, the instructor had no control over the information provided.Although the instructor did have some control over the timing, he generally adhered to the timing suggested by the game designers (comment about student frustration over info.).

Value of Simulations in Resolving Real Conflicts
The next set of questions common to both surveys asked respondents the extent to which they agreed with four statements: that the simulations would help them to participate effectively in ADR processes in the future, to achieve better outcomes using these processes, to give them confidence using these processes, and to effectively manage conflicts.The first three of these questions (shown in Tables 6 and 7), which are all process-centred, had similar means which experienced very slight changes (no more than .03)between the two surveys.All of these means ranged from 1.66 to 1.80, closer to "somewhat agree" than to "strongly agree".These results indicate that participation in the simulations did very little to change students' perceptions of how this experience would help them in using ADR.The fourth question (also shown in Table 8) was not focussed on ADR processes, per se.Although it too remained within the "somewhat agree" range, its larger shift in its mean response toward stronger agreement (from 2.02 before to 1.77 after) was statistically significant, with the Wilcoxon test indicating a z score of -2.138 and a p value of .033.Prior to the course, students believed that the simulations would be less helpful in teaching them to effectively manage future conflicts than they would be for learning how to use ADR processes.However, their participation in the simulations improved their collective opinion about conflict management to the point that it was on par with that regarding ADR.The next question asked respondents (both pre-and postsimulations) the extent to which they believed that the simulations would be/were realistic.This was related to the preceding set of four questions, and the earlier question about the adequacy of the preparatory information provided.The overall presimulations mean for this question was 2.15, which was slightly on the neutral side of "somewhat agree".This indicates that respondents were less confident about the realism of the simulations than there were about any of the other merits of the simulations probed in these related questions (which all had lower means).However, the mean for this question dropped to 2.51 on the after-simulation questionnaire, in contrast to the very slight changes in the preceding four questions.In other words, after having participated in the simulations respondents felt they were less realistic than they had anticipated them to be prior to the course.The Wilcoxon test indicated that this change was not quite statistically significant (z = -.1826,p = .068).

Value of Negotiation Theory
In both sets of surveys, students were asked about the usefulness of the theory presented in the set of required course readings to their participation in the simulations, and their usefulness in real negotiation situations they will encounter as planners.Both questions showed a small improvement in their preand post-simulations means, from 2.56 to 2.49 and from 2.68 to 2.36, respectively.Although small, each change was sufficient to switch the closest response from "good" to "very good".After completion of the course, students felt that the theory contained in the readings was more useful to both the simulations and to application of ADR to real conflicts than they had before the first simulation.This fits with the fact that several students expressed concern about the large volume of reading prior to the course, but when asked about this following the course reported that they had not found the reading load to be onerous and that it had been useful and interesting.

Understanding of Substantive Issues in Conflict Resolution
Four ranking questions probed students' understanding and opinions about several key substantive issues in conflict resolution, both before and following the course.The first of these asked students to rank the importance of five aspects of "success" in an ADR process (with one being the most important).As shown in Table 9, the only change in rankings pre-and postsimulations was a switch in the third and fourth ranks, between "reducing the time required for a subsequent legal process to resolve the dispute" (third rank after) and "reaching a final agreement" (fourth rank after).This switch was largely the result of a statistically significant decline (as indicated by the t test) in the perceived importance of reaching agreement.This was not surprising, since as people learn more about ADR, they typically place less emphasis on the traditional measure of success of reaching an agreement and recognize that there are other important benefits that may accrue from ADR even if no final settlement is produced, such as resolving some of the issues in dispute, improving inter-party relationships, and reducing the time required to resolve the conflict through a subsequent process.
The second question in this set asked respondents to rank the importance of five desirable characteristics of a facilitator or mediator.No item changed more than one place in rank between the pre-simulations and post-simulations surveys (see Table 9).Three items experienced small changes to their mean responses.Larger changes were seen in "impartiality and neutrality", which improved its ranking from second to first (changing places with "listening and communication skills"); and in "substantive knowledge in the area of the dispute", which dropped from the fourth rank to the fifth (switching with "acceptability to each of the disputing parties").These changes were confirmed by the t test to be statistically significant.Impartiality/neutrality and substantive knowledge were both themes addressed in the simulations.Apparently, students learned the critical importance of the former, and came to understand that facilitators and mediators can be effective without previously being experts in the specific subject area of the dispute.The third question involved ranking five characteristics of a dispute that make it amenable to resolution using ADR.The highest ranking two characteristics experienced very small changes between the two surveys, and did not change positions (see Table 10).They were "a good working relationship between the disputing parties, or the need to develop one" (ranked first); and "multiple issues that the parties value differently, allowing them to make trade-offs" (ranked second).The remaining three items all experienced statistically significant changes within the lowest three rankings."An impasse having been reached in a previous attempt at conflict resolution" moved from fourth to third, "the impending start of a less desirable conflict resolution process (e.g. a trial date)" ascended from fifth to fourth, and "a relatively small number of disputing parties" dropped from third to fifth place.It is interesting that although the simulations changed respondents' opinions about the dispute characteristics they felt were less important, they had little effect on interparty relationships and issues valued differently, which were seen as most important.For the latter two, either students possessed prior knowledge about the amenability of disputes to ADR (which is unlikely), or the simulations did not contribute much to their knowledge in this area.
The fourth and final question in this set asked students to rank the relative importance of five principles of negotiation purported in "Getting to Yes".This question experienced the opposite set of changes to the preceding question.The only statistically significant changes (confirmed by the t test and shown in Table 11) were in the first and second ranked principles, which switched positions."Focus on interests, not positions" moved up to first place (with its mean ranking changing from 2.22 to 1.56), while "separate the people form the problem" dropped to second (due to its mean changing from 2.14 to 2.64).In decreasing order of importance, the remaining three principles were: "invent options for mutual gain", "don't bargain over positions", and "insist on using objective criteria".Since the simulations and debriefings placed fairly heavy emphasis on identifying parties' interests and attempting to capitalize on differing but compatible interests, it is not surprising that this principle was ranked most important in the post-simulations survey.That the least important three principles did not change their rankings is surprising, considering that all five principles were repeatedly discussed in introducing and debriefing the simulations, which might be expected to alter the students' perceptions about their relative importance.

Concerns and Problems Encountered with the Simulations
The pre-simulations survey asked respondents (in an openended format) if they had any concerns about the upcoming simulations.Commonly expressed concerns included that the simulations would not be very realistic, participants would have difficulty acting their roles or would not take the simulations seriously, the determination of course grades would not be fair, the workload would be too onerous, and they themselves would not perform well in the simulations.A few students worried that the simulations would not focus on planning issues, and that the large class size and brevity of the course would not allow sufficient time or instructor attention to make the simulations worthwhile.
The corresponding question on the post-simulations survey asked about any problems encountered in the simulations.Frequently mentioned ones included some of the simulations lacking sufficient background information (including independent standards that negotiators could turn to), and some people not adhering to the "rules of the game".Concerns mentioned prior to the first simulation that came to fruition as problems reported by some included respondents feeling that they had not performed well in the simulations, the simulations not being realistic (due mainly to the artificial constraint of scorable points systems in some of the simulations), and participants having difficulty "getting into their roles".One student expressed that "participants had difficulty playing their assigned role without allowing any personal biases to enter".
Several of the "problems" that were identified on the postsimulations survey are considered by the authors to be commonly encountered in real negotiations, and speak to the realistic nature of the simulations.These include variability in the personalities of the participants (especially in key areas such as assertiveness, charisma and persuasiveness); and behaviours such as stubbornness, inflexibility and even dishonesty by some negotiators.These findings are consistent with the problems encountered by Lewicki (1986) in using simulations to teach negotiation, although we did not observe as great a display of emotion in our simulations as he reported.One respondent wrote: "People would not really listen to each other.[They were] too entrenched in the positions or they would even lie about their bottom line."One reported difficulties brainstorming to "invent options" without criticizing.Another found it challenging communicating with a negotiating partner and acting as a cohesive team.S/he wrote: "I found it difficult having a partner in my party, because it made me reluctant to take the initiative in making offers.When negotiating on my own, I was much more confident in taking initiatives".

Additional Comments about the Simulations
The final question on each survey invited respondents to add any comments about issues not covered in the questionnaire.The single response on the pre-simulations questionnaire was that a full-length (rather than a four-week module) negotiation course should be a mandatory component of the Master's in planning curriculum.Several students expressed the same sentiment at the end of the course.Nearly all of the responses were positive comments about the course, especially regarding its usefulness in training future planning and their high level of enjoyment.A few students confidently looked forward to their first opportunity to engage in a "real life" planning negotiation and apply what they had learned.Others were eager to learn more about specific areas of negotiation, such as preparing for a negotiation and drafting a final agreement.These responses were consistent with the feedback received through the confidential course evaluation forms.Many echoed the sentiment that the course should be full-length, although for some their rationale was that the workload and time commitment was too heavy for a four-week module worth one-third of a course credit.One student wrote that: "Covering negotiation/mediation skills is very important to planning.The module was helpful in developing these skills."Another commented that "[the] simulations were excellent learning tools."

Influence of Prior Experience with Simulations on Student Responses
The pre-course questionnaire asked students whether they had ever participated in any type of role-playing simulation of a conflict resolution process to test whether prior experience had any effect on the responses.Nineteen respondents indicated that they had prior experience, and 22 indicated that they did not.We hypothesized that for at least some of the pre-simulations questions, there would be a difference between the responses of those with prior experience and those without.However, prior experience would be irrelevant to the responses on the postsimulations survey, since by then everyone would have participated in five simulations.Visual examination of the frequency tables and means of the pre-simulations responses suggested a few possible differences.Differences in the mean response between the prior experience and no prior experience respondents for each question ranged for .02 to 1.01.However, the Kolmogorov-Smirnov test was run on the data for each question, and none of the differences were revealed to be statistically significant.This clearly indicates that prior experience with role-playing simulations in conflict resolution did not influence respondents' opinions and knowledge regarding the issues asked about in the pre-simulations survey.This is a surprising finding.However, it is particularly interesting when combined with the findings of significant differences between the pre-and post-simulations surveys discussed earlier.When combined, these findings suggest that the five simulations in this course significantly changed the knowledge, perceptions and opinions of respondents with respect to some areas of inquiry, even though prior experience with simulations had failed to do so.Of course, another factor that may be at play is the fact that some of the respondents' prior experience may have been very limited or some time ago, and therefore presented the respondents with little opportunity to learn.

Testing for Association between Responses and Exam Scores
The final statistical analysis involved testing for an association between each student's score on the course exam and each of nine questions on the post-simulations survey that were relevant to evaluating knowledge acquired in the course.The exam was a take-home exam distributed at the end of the final session, and due 28 hours later.It accounted for 40% of the course grade.The selected questions were those asking respondents their opinions about the level of their understanding of each of the three ADR processes (Questions 1-3 in Table 12); the overall educational value of the simulations (Question 10); the usefulness of the simulations for teaching participants to participate effectively, achieve better outcomes, have more confidence when engaged in real ADR processes (Questions 11-13); to help participants to manage actual conflicts more effectively (Question 13); and the importance of planners developing effective negotiation skills (Question 27).The exam scores were paired with the response data for each of the above questions in turn.A Spearman correlation coefficient was then calculated for each pair, to test for associations between their data.The results are shown in Table 12.Surprisingly, none of these pairings produced a statistically significant association.However, this may be due in large part to the unusually small standard deviation of exam scores.

Conclusion
These findings indicate that role-playing simulations are very effective for teaching negotiation skills (and negotiation-based ADR processes such as facilitation and mediation) to planning students.Participants reported high levels of satisfaction with the simulations, both in terms of their educational value and the students' enjoyment of the experience.The ability of the simulation to change students' knowledge, perceptions and opinions is indicated by the differences in responses between the pre-and post-simulations questionnaires for many of the areas of inquiry.This finding regarding the effectiveness of role-playing simulations in teaching negotiation is consistent with the non-empirical conclusions of Lewicki (1986).However, no previous research was found which specifically addressed the merits of using simulations to teach negotiation skills to planning students, providing no basis of comparison for our findings.Furthermore, none of the evaluation criteria employed in this article were also used in any previous studies on the effectiveness of role-playing simulations in other educational settings.This further precluded placing our findings in the context of previous research.
The simulations taught students the importance of interests in conflict resolution, especially identifying common interests, and identifying and communicating one's own interests.Issues that are valued differently by different parties, and which can therefore be traded off, are helpful.Simulations are quite valuable in preparing students to participate effectively in real ADR processes, both in giving them confidence and in bringing about good outcomes.More generally, they help future planners to manage actual conflicts more skillfully.Participants found the theory contained in the course readings to be useful and relevant in the simulations.Two of the five principles of negotiation presented in "Getting to Yes" were especially appropriate to these simulations: "Focus on interests not positions" and "Separate the people from the problem".
The simulations were able to dispel the misconception that "success" using an ADR process consists only of the achievement of a final settlement.Rather, the students learned that other benefits may accrue even if agreement is not reached, such as resolving some of the issues in dispute, lessening the duration of time required to ultimately settle the dispute, and improving relationships between the parties.The importance of good relationships, or a mutual desire to develop them; and the complete impartiality and neutrality of facilitators and mediators were revealed.Simulations involving a greater number of parties (e.g. six or seven) and a facilitator appear to be particularly beneficial as educational tools.
Simulations were found to be slightly less valuable for teaching aspects of planning other than conflict resolution.In part, this contradicted the findings of Baum (1997), Innes and Booher (1999) and Krause and Amaral (1994).However, we found ADR to be particularly useful for generating more equitable policies and plans, improving communication of complex planning concepts, and understanding the diverse viewpoints of stakeholders.These negotiation-based processes seem most appropriate for land use, environmental and community planning.Participants in the simulations learned that planning disputes are often complex and difficult to resolve, especially if they involve multiple parties and many competing interests.They also discovered that disputants are not always willing or able to engage in productive and principled negotiation.
Some potential weaknesses of using simulations in this setting that were detected include the provision of inadequate background information, lack of realism, difficulty of some participants to portray their roles convincingly, counterproductive (but often realistic) behaviours (e.g.stubbornness, inflexibility and dishonesty) by some participants, and communication problems between members of negotiating teams.
A surprising finding was that prior experience with role-playing simulations of conflict resolution processes did not influence students' opinions and knowledge of ADR processes prior to the course beginning.This makes the ability of the simulations in this course to change students' knowledge levels and opinions about conflict resolution in planning more impressive, since prior experience with simulations had failed to do so.Finally, it was also surprising that there was no significant association between final exam scores and any of the nine postsimulations questions tested, which were relevant to evaluating knowledge acquired in the course.On one hand, the students clearly learned a great deal from the simulations; yet how they answered selected questions at the end of the course did not have any bearing on how they performed on the exam.

Table 1 .
Students' levels of understanding of ADR processes.
dent understanding.It was expected that the greatest improvement would be in facilitation, since it is traditionally the least familiar of the three processes to those with no conflict resolution experience.A fairly good prior understanding of negotiation was anticipated, and in fact it was slightly better than for facilitation and mediation.However, since three of the five simulations involved negotiation, this is likely responsible for this ADR method seeing the largest gains (as measured by means) in student understanding.

Table 2 .
Perceived educational value of the five simulations.

Table 3 .
Usefulness of the simulations for learning about planning.

Table 4 .
Other merits of role-playing simulations in learning planning.

Table 5 .
Ranking of importance/achievement of learning objectives by simulations.
Note: N.B.Rank of 1 represents the most important; 5 is the least important.

Table 6 .
Merits of role-playing simulations in teaching effective use of ADR processes.

Table 7 .
Merits of simulations in teaching effective ADR use and conflict management.

Table 8 .
Ranking the importance of aspects of ADR success.
Note: N.B.Rank of 1 represents the most important; 5 is the least important.

Table 9 .
Ranking the Importance of Characteristics of a Facilitator/Mediator.

Table 10 .
Ranking importance of dispute characteristics for amenability to ADR.

Table 11 .
Ranking the importance of principles of negotiation in "Getting to Yes".

Table 12 .
Testing for correlation between exam scores and 9 post-simulation questions.