Open Journal of Social Sciences
Vol.02 No.08(2014), Article ID:49217,9 pages
10.4236/jss.2014.28013

The Potential of Adaptive Mentorship©: Experts’ Perspectives

Edwin Ralph, Keith Walker

University of Saskatchewan, Saskatoon, Canada

Email: edwin.ralph@usask.ca

Copyright © 2014 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

Received 7 June 2014; revised 23 July 2014; accepted 7 August 2014

ABSTRACT

In recent years, global interest in the processes of mentorship and coaching has expanded across all disciplinary fields. Educational institutions, commercial enterprises, and other organizations have integrated mentorship processes into their educational programs to help prepare/train protégés for entry into a specific professions or occupations and/or to upgrade their related skills/knowledge. Over the past quarter century, in partial response to the popularity of mentoring, the authors have developed a mentoring model called Adaptive Mentorship© (AM). Research conducted by the authors and others has affirmed AM’s value in improving mentoring practice in a variety of disciplines. In the present article, the authors summarize assessments of the model that they solicited during the past five years from 49 multi-disciplinary groups or panels of experts. The experts’ positive statements regarding AM outweighed their cautionary comments by a ratio of 2:1. The strengths that they identified were that AM conceptualized the entire mentorship process in an understandable manner, and that it helped reveal potential interpersonal conflicts as well as practical solutions for them. The caveats identified by the experts were that personnel employing the AM model must apply it sensibly, sensitively, and flexibly―especially in cross-cul- tural contexts.

Keywords:

Adaptive Mentorship, Coaching, Experts, Mentoring, Supervision

1. Introduction

Program administrators in charge of professional education, occupational training, and/or leadership development share a common objective, which is to enhance mentoring practice in their respective fields [1] . Achieving this objective will not only directly bolster the professional development of the graduates and/or employees in their respective pre-service and in-service programs, but it will indirectly benefit society as a whole by helping provide more competent and confident professionals and qualified practitioners in every sector [2] [3] .

However, to accomplish these goals program leaders will be expected to employ mentorship processes that are grounded in research-based evidence and are effective. Such research evidence must be trustworthy, which in turn necessitates that the agents conducting and reporting that research must also be recognized as experts in their respective fields of study [4] -[6] . In this present article, the authors summarize the perspectives of several groups of mentorship experts that recently assessed the effectiveness of the Adaptive Mentorship model. The authors also invite other researchers and educators interested in enhancing their respective mentorship programs to consider whether AM warrants possible application in their settings.

The practical or clinical phase of professional education is an integral part of the pre-service and/or in-service preparation of prospective practitioners in all disciplines; and this experiential-learning component typically occurs within internships, co-operative education placements, field-experiences, or other practicum programs, where neophytes gain hands-on practice in actual work settings [7] . In such field-based learning settings or rotations, mentors (i.e., individuals with more experience, knowledge, and skill in the particular field) assist protégés (i.e., learners with less experience) to develop their related knowledge and skills [8] .

Although it appears that the overall mentorship/coaching/supervision process within organizational life has functioned in a relatively satisfactory manner [9] , a growing body of research suggests that certain limitations and deficiencies persistently arise in the mentorship relationship to inhibit or hamper its effectiveness [10] , such as: inadequate mentorship training; lack of a mentorship framework, miscommunication and misinterpretation between mentors/protégés; interpersonal disagreements/conflicts; attribution or denial of responsibility/ blame; accusation of stubbornness/intransigence; or equivocations invoking such evasions as “irreconcilable differences” or “personality clashes.” One model that addresses these mentoring problems is Adaptive Mentorship©, which the authors have created and researched during the past 25 years; and in this article, they summarize the perspectives of 597 individuals invited to assess the AM model. These educators, researchers, and practitioners were considered by the authors to be experts in the field of mentoring, because of their related background, experience, training, research, and/or interest in mentorship [11] .

2. The Adaptive Mentorship Model

2.1. Description of AM

Adaptive Mentorship is [12] a model that guides mentors/supervisors in adjusting their mentoring responses to appropriately match the task-specific development level of protégés whom they are assisting in the learning/ working situation. The authors depict the AM model in Figure 1.

The outer border of the diagram represents the entire context within which the mentorship process is conducted, much of which is out of the control of the mentor or the protégé. However, they can manage their own behaviour. Thus, mentors can modify their mentorship action, which consist of two dimensions shown in Figure 1: a) their “task” response (i.e., the degree of specific direction given to the protégé regarding the technical, mechanical, or procedural aspect of the latter’s performance of a specific task being learned); and b) their “support” response (i.e., the degree of “human” or psycho/social/emotional expression they provide the protégé learning the skill-set).

By contrast, the factor over which protégés have most control is their task-specific developmental level. It likewise consists of two dimensions: their “competence” level (i.e., their actual technical ability to perform the task in question), and their “confidence” level (i.e., their degree of self-assurance, composure, psychological comfort, and security and/or safety in performing the skill-set).

The heart of the AM model is represented by the shaded arrows linking the D- and A-grids, which portray the mentor’s matching of one of four typical “A” (adaptive) responses with a similarly numbered “D” (developmental) level characterizing the protégé’s performance of the particular skill/competency. Of course, there are many more than four positions within each grid, because there is a host of possible A/D combinations. However, for conceptual/analytical purposes, we highlighted these four combinations simply to reflect types within each quadrant.

2.2. Application of AM

Implementing Adaptive Mentorship consists of three phases [10] :

Figure 1. The Adaptive Mentorship© model. The mentor synchronizes his/her adaptive response indicated in the A-grid to appropriately match the task-specific developmental level of the pro- tégé shown in the D-grid [8] [10] [12] -[14] .

1) Determining the protégé’s development. First, the pair ascertains the existing development level of the protégé to perform a specific skill-set being learned at the time (e.g., classroom management). As illustrated in the “D-grid,” a protégé’s task-specific level of development consists of both his/her competence and his/her confidence levels to consistently organize and manage student learning. The D1 quadrant reflects an individual with “low competence” and “high confidence” to accomplish the task (i.e., he/she does not know exactly how to conduct it, but is confident, willing, and eager to try). D1 often exemplifies novice teachers’ early perception of being able to work with a group of students. A protégé at D2 is low on both competence and confidence; a protégé at D3 shows higher competence and lower confidence; while a protégé at D4 is high on both dimensions for the particular skill-set.

A protégé’s developmental level may be identified by: a) the mentor’s formal and informal observations of the protégé’s actual performance of the skill/task; b) the pairs’ informal conversations about the protégé’s D-level; and/or c) the protégé’s answers to the mentor’s direct questions about his/her progress. These D-levels are: task-specific; changeable over-time; different for different skill-sets; and temporary indicators of a protégé’s stage at a specific point in time [8] [14] .

2) Synchronizing the mentor’s response. Next, the mentor appropriately adjusts his/her mentorship response to match the existing D-level of the protégé regarding the particular competency: A1 matches D1; A2 matches D2, and so on. The mentor’s “A” adaptive-response likewise consists of two dimensions: the degree of support the mentor provides (i.e., the psycho-emotional responses of encouragement, reinforcement, and praise to bolster the protégé as he/she attempts to develop the particular skill-set). Support consists of genuinely positive words and/or actions, and varies along a continuum. The other A-element is task (i.e., the amount of direction the mentor gives the protégé regarding the technical or mechanical performance of the task). This task action also varies along a continuum ranging, for example, from one extreme of direct telling, to demonstrating, to suggesting, to questioning, up to delegating with respect to the protégé’s technique in the skill.

The key principle for mentors in correctly matching their A response with their protégé’s D levels is that: a) the task aspect must be inversely proportional to the extent of the protégé’s competence level for the skill set; and b) the extent of the mentor’s support must be similarly inversely proportional to the novice’s level of confidence for the particular task being practiced.

3) Monitoring the protégé’s development. Subsequently, the mentorship pair continually and mutually monitors the protégé’s changing level of development, which will necessitate that the mentor simultaneously adjusts his/her adaptive response to match, in inverse proportions, the protégé’s changing development level(s). Because a protégé will be at different D levels for different skills, the mentor will likewise need to provide different A responses for these tasks.

3. The Role of Experts

Society has relegated to experts the role and responsibility of providing valid advice, counsel, and direction to facilitate decision-making by individuals and groups; and this role spans all public policy, scientific/medical, judicial/legal, and business/organizational sectors [15] [16] . Consequently, citizens expect experts across this professional/occupational landscape to possess a degree of superior knowledge and competence in their respective fields. They also expect them to perform consistently better than the majority of their peers, and also to produce tangible, measurable, and positive results in their areas of expertise [11] [17] .

This reliance on expertise has been integrated into each society’s structures of civic government, forms of organizational leadership, and institutions of education/training―including the related processes of mentorship, coaching, and teaching [18] . Yet, by contrast, both the research literature and individuals’ personal experiences have shown that experts’ judgments can be wrong [19] [20] . One way society has devised to help counter this possibility of expert error has been to solicit the perspectives of multiple experts, because group opinion may be more accurate than the view of any single individual [11] [21] .

A recognized example of this application of expert groups was the Delphi technique [22] , originally developed in the 1950s by the Rand Corporation to facilitate military decision-making [23] -[25] . The goal of the Delphi exercise, and that of later adaptations of the approach [26] , was for the group of experts to eventually reach a consensus regarding the best solution for the particular problem they were addressing. Group members engaged in a series of collaborative iterations, which were interspersed by the group facilitator’s consecutive presentations of feedback and syntheses from previous iterations [27] [28] . These iterations were to be characterized by independent thought, interactive discussion, controlled debate, and personal and co-operative reflection, and by an attempt to avoid/reduce close-minded biases, angry confrontation, or bullying behavior [29] [30] . Research results regarding the Delphi and other expert panel methods showed mixed results, in that these approaches seemed relatively more effective for simple technical issues, but less so for complicated and ill-defined problems [31] -[33] .

In Table 1, the authors present a synthesis of key findings derived from several research reports regarding the strengths and limitations of using expert groups to improve decision making.

Table 1. Strengths and limitations of expert groups.

Note. These items have been synthesized by the authors from the following researchers: [17] [19] [21] [23] [26] [29] [33] [41] -[45] .

One logical result of examining the positive and negative aspects of utilizing the expert-group technique is that participants will not only want to preserve its strengths, but will also want to eliminate or at least to reduce the weaknesses. As is the case with any procedural model, method, or approach in the social sciences, it is also obvious that expert panels could be misused, under-used, over-used, abused, or non-used, or they could be sensibly and sensitively used―which of course would be more profitable for all participants and stakeholders [14] [34] [35] .

4. Method

During the five-year period between 2009-2010 and 2013-2014, the authors were invited to present the Adaptive Mentorship model at 49 academic and practitioner conferences, workshops, seminars, or meetings in a variety of locations in Canada, Cook Islands, Fiji, New Zealand, Spain, Tonga, the United Kingdom, and the United States. This international dissemination effort was supported in part by grants the authors received from the Social Sciences and Humanities Research Council of Canada and the University of Saskatchewan. The Adaptive Mentorship segments of these meetings varied in length from one to three hours, and typically formed one portion of longer organized events that focused on a variety of broader educational, leadership, professional, or mentorship subjects.

Delegates at all these events had been invited to attend by the various conference/meeting organizers, because of the invitees’ involvement and interest either in some form of mentorship across professional education settings, or in other coaching, supervising, or training processes. With respect to the AM parts of these meetings, the authors, who delivered the AM presentations, alerted all participants at the beginning of each session that: a) the presenters recognized the attendees as experts regarding mentorship, because of the latters’ accumulated mentoring experiences, backgrounds, and involvement; and b) as such, at the end of each AM session, these experts would be invited to offer their assessment of the AM model by anonymously and confidentially writing brief responses to two questions: What were the positive aspects of AM? What were any pitfalls or challenges they may have observed?

Each AM workshop-presentation consisted of a description the model and its implementation, a summary of its research record (strengths and limitations), a brief practice period for attendees to become acquainted with AM, and the invitation to submit their responses to the questions.

The authors later employed the constant comparative technique of the qualitative research approach [36] to collate, analyze, and categorize/re-categorize the 1422 submitted responses. The authors examined/re-examined these data and observed for emerging patterns, themes, or categories [37] . They tabulated the percentages of respondents’ views for each emerging category and reported the values in Table 2.

5. Results

Forty-nine cohorts consisting of approximately 600 mentorship experts, who represented a variety of professional disciplines from several countries, assessed the efficacy of the Adaptive Mentorship model. At the conclusion of an AM workshop that they were invited to attend the experts submitted their written judgments regarding their views of the model’s strengths and weaknesses. An examination of the summary of these results in Table 2 indicates that: a) experts’ positive comments regarding the model outnumbered their cautionary statements by a two-to-one ratio; b) within the data for each of these two broad categories four sub-themes emerged; and c) these findings were generally consistent with trends reported in previous research regarding both the Adaptive Mentorship model [8] [10] and its earlier prototype, Contextual Supervision [14] [34] [38] . The positive and negative categories and their respective sub-categories are highlighted in Table 2 and illustrated below.

5.1. Positive Aspects

Two-thirds of all written responses submitted by the experts identified positive features of AM, as shown in the four sub-categories in the upper portion of Table 2. The authors provide typical comments from respondents that illustrate each of these perspectives.

Provides clarity. The largest category in the entire survey was related to experts’ perceptions that the AM model presented a clear conceptualization of the mentoring process. Typical comments revealing this theme were: “It is excellent, because it put all the elements in perspective;” “I found the diagram was good for visual

Table 2. Summary of feedback-comments provided by experts attending 49 adaptive mentorship© Meetings/Seminars/Workshops Regarding the AM Model, 2009-2014.

Note. From the 49 workshops/seminars conducted in eight countries, 597 participants submitted a total of 1422 evaluatory comments. All participants provided at least one positive comment, while 481 participants provided at least one cautionary comment.

learners, and simple to follow for both mentors and protégés;” “To me it requires partners to open up communication and to be reflective; “AM shows the balance for all four dimensions, that each partner is interdependent;” and “I found it logical and easy to understand and use.”

Offers guidelines. The second greatest number of positive comments described the practical guidance that the AM model offered mentoring partners, particularly mentors. Written remarks that exemplified this feature were: “It shows mentors exactly how to adjust both their direction and support to meet the individual’s needs;” “The model allows for mentors to adapt and differentiate their actions;” “It helped me locate [the protégé]: ‘You’re here [at D2] now, but you’re going up every day;’” and

I saw [protégé] at D3, but earlier both of us located me as at A2. I have been gradually moving to A3 to match up with [protégé], by still being as supportive as before, but trying not to direct him as much.

Promotes development. The third largest positive theme focused on experts’ views regarding the model’s facilitation of personal and professional development for protégés and mentors, alike. Comments highlighting this aspect were: “I liked how it showed the pairs how to find protégés’ D level, and the mentors how to give support;” “AM helps partners each deal with difficult situations;” “Because it encourages communication and collaboration between a mentor and protégé both partners will grow;” “It helps partners see where each other are at, as well as ourselves. We have to communicate and co-operate;” and “The model helps both succeed; the mentor succeeds when the protégé grows.”

Is cross-disciplinary. A fourth positive category emerging from the experts’ assessments of AM emphasized the model’s evident applicability across disciplines. Illustrative statements in this category were: “It even helps me be a better peer, wife, and mother;” “I think all mentors should learn it;” “AM is versatile enough for all professions;” “It shows how to mentor in every situation;” and “It has wide applicability and is complementary to all that we understand about adult learners in any ‘apprenticeship’ setting.”

5.2. Cautionary Aspects

One-third of the experts’ submissions referred to caveats or areas of caution regarding the use of AM. Sample comments illustrating the four sub-themes that emerged in this caution category are enumerated below.

Sufficient time needed. Nearly 30% of the experts’ comments advised users to ensure the provision of adequate time for participants to become fully acquainted with AM, its rationale, terminology, and procedures. Typical statements here were: “I need to try it out;” “You need time to get used to it;” “I like it, but before I can really judge it, I need more time to see more practical examples and to practice with it, myself;” “Before applying it, you will need to make sure you clarify the procedures so partners can see where both the mentor and mentee are;” “People will need time to understand and practice it and work out any initial misunderstanding;” and “Participants must not rush, but be given enough time to keep the pace and the conversation up in the long haul.”

Consideration of other factors. Approximately one percent of the cautionary comments advised AM users not to ignore the influence of other contextual elements that could possibly affect the mentorship process. For instance, experts stated: “Don’t forget to take into account the history and cultural backgrounds of each partner…there may be unspoken traditions involved;” “The context may shape the partners’ actions and behaviors unexpectedly;” “The customs and religious practices of [our country] are very strong, and will hold sway in people’s reactions, despite any outside model;” “There are rural and gender constraints in [country] that cannot be changed;” and “It won’t work unless trust is developed between and among all the partners.”

Possibility of resistance. A sub-theme related to the one just mentioned appeared in one percent of the cautionary comments, and it was that participants may not only passively ignore the model, but may actively oppose it. Typical comments articulating this warning were: “Many of our leaders would not accept this model, because women have been culturally silenced here;” “It might not work, because women have traditionally not been allowed to be leaders in [location];” “and “I think leaders may be reluctant to use it, because of the following limitations: peoples’ biases and negative attitudes, top-down management perception, micro-managing style, and the existence of gender/age/social/ethnic/education barriers, here;” and “Some mentors may see it as too prescriptive, and may not be willing to spend the considerable time and energy needed to work with the protégé in the daily swings of practicing.”

AM design faults. A fourth theme that emerged from one percent of the written submissions identified perceived weaknesses in the model’s graphic design. Examples of experts’ comments reflecting this weakness were: “The four large arrows should be double-headed to show reciprocity;” “The quadrants force you to categorize people, but people cannot be put into boxes;” “The two axes don’t follow conventional procedures of vertices in graphs beginning on the left at zero;” and “There is a lingering danger of seeing only four possible positions in the grids, when in reality there are numerous possible combinations.”

6. Discussion

The process the authors used in soliciting the experts’ assessments of the AM model did not follow the prescribed Delphi process described in the early literature [23] . However, the authors did rely on three key principals espoused by the Delphi technique and its adaptations, namely: that experts’ knowledge tends to be more accurate than that of novices; that group judgments tend to be more reliable than the view of any one individual; and that a group’s collaborative dialogue/discussion tends to clarify members’ thinking.

The findings reported here also support previous research on Adaptive Mentorship showing that when mentors and their protégés appropriately apply the model, then existing mentoring difficulties can be reduced, and potential ones can be de-fused [8] [10] [14] . The key word here is “appropriately,” because even though AM’s potential is evident, users must acknowledge the caveats. Certain conditions must be met, such as [10] : a) providing adequate time and support for mentoring partners to become well acquainted with AM; b) recognizing that some individuals may not want to use AM; and c) acknowledging that unexpected and/or irreversible events/circumstances and contextual factors may hinder AM’s effectiveness.

7. Concluding Thoughts

To rely solely on expert panels to guarantee perfect program and policy decisions is unwarranted and untenable. For example, this assertion was illustrated by the recent aviation incident involving the missing Malaysian Airlines Flight MH370, which attracted worldwide attention. Because the officials (many of whom were considered distinguished in their fields) failed to find the airliner within the first two months of searching, the three main countries involved collaborated to re-establish a new panel of experts to re-examine the accumulated data, to expand the search area, and to better co-ordinate all efforts [39] . According to the head of the restructured search operation, the goal of the new panel was:

…to go back and have a look at all of the data that has been gathered, all of the analysis that has been done and make sure there’s no flaws in it, the assumptions are right, the analysis is right and the deductions and conclusions are right [40] .

These events demonstrate that: a) although expert groups may possess superior knowledge and specialized skills in a particular domain, they are not inerrant; and b) although over time their analysis may begin to converge towards a satisfactory solution, they must continue to exercise patience, humility, and interdependence as they deliberate.

The experts’ perspectives synthesized in this present paper, together with the findings reported in previous AM research [8] [10] [12] [13] , all emphasize that the Adaptive Mentorship model is not a panacea that can eradicate all mentoring problems, but rather that it has been shown to be efficacious in assisting mentorship participants:

・ to stabilize their conception of the whole mentorship enterprise;

・ to clarify their comprehension of the spectrum of typical actions and responses emerging from protégés and mentors in the routines of mentoring practice;

・ to use the model as a possible “third party” for assisting mentorship partners to identify potential miscommunications, misinterpretations, and conflicts; and

・ to re-frame such misjudgments into opportunities for participants to adapt their respective responses according to AM’s principles of practice, thereby de-fusing and/or de-escalating potential conflicts.

In the light of this discussion, we the authors extend an invitation to readers interested in enhancing their mentorship programs to consider whether the AM model warrants their further attention to help inform mentoring practices in their respective settings.

References

  1. Cowan, F. and Flint, S. (2012) The Importance of Mentoring for Junior Doctors. British Medical Journal Careers. http://careers.bmj.com/careers/advice/view-article.html?id=20009962
  2. Lanphier, G. (2012) Skilled Labour in Demand Worldwide. The Regina Leader-Post. http://www.leaderpost.com/business/Skilled+labour+demand+worldwide/7755311/story.html
  3. O’Brien, P. and Gostin, L. (2011) Health Worker Shortages and Global Justice. The Milbank Memorial Fund. New York. http://www.milbank.org/publications/milbank-reports/158-reports-health-worker-shortages-and-global-justice-2#Foreword
  4. Berliner, D. (1986) In Pursuit of the Expert Pedagogue. Educational Researcher, 15, 5-13. http://edr.sagepub.com/content/15/2/4
  5. Shulman, L. (1986) Those Who Understand: Knowledge Growth in Teachers. Educational Researcher, 15, 4-14. http://edr.sagepub.com/content/15/2/4
  6. Shulman, L. (1987) Knowledge and Teaching: Foundations of the New Reform. Harvard Educational Review, 57, 1-22. http://her.hepg.org/content/j463w79r56455411/fulltext.pdf
  7. Ralph, E., Walker, K. and Wimmer, R. (2010) The Practicum in Professional Education: Canadian Perspectives. Temeron/Detselig, Calgary.
  8. Ralph, E. and Walker, K. (2011) Adapting Mentorship across the Professions: Fresh Insights and Perspectives. Temeron/Detselig, Calgary.
  9. Chun, J., Sosik, J. and Yun, N. (2012) A Longitudinal Study of Mentor and Protégé Outcomes in Formal Mentoring Relationships. Journal of Organizational Behavior, 33, 1071-1094. http://onlinelibrary.wiley.com/doi/10.1002/job.1781/pdf
  10. Ralph, E. and Walker, K. (2013) The Efficacy of the Adaptive Mentorship Model. Open Journal of Leadership, 2, 21- 26. http://file.scirp.org/Html/32622.html
  11. Ericsson , K., Charness, N., Feltovich, P. and Hoffman , R. (2006) The Cambridge Handbook of Expertise and Expert Performance. Cambridge University Press, Cambridge. http://dx.doi.org/10.1017/CBO9780511816796
  12. Ralph . E. and Walker , K. (2010) Enhancing Mentors’ Effectiveness: The Promise of the Adaptive Mentorship© Model. McGill Journal of Education, 45, 205-218. http://mje.mcgill.ca/index.php/MJE/article/view/4653 http://dx.doi.org/10.7202/045604ar
  13. Ralph , E. and Walker , K. (2012) Enhancing Mentoring across the Disciplines: Via the Adaptive Mentorship Model. Bridges: Reflecting the Scholarship of Teaching & Learning at the University of Saskatchewan, 10, 9-11. http://www.usask.ca/gmcte/sites/default/files/Bridges%20May%202012%20WEB.pdf
  14. Ralph , E. (1998) Developing Practitioners: A Handbook of Contextual Supervision. New Forums Press, Stillwater.
  15. Dror, I. (2011) The Paradox of Human Expertise: Why Experts Get It Wrong. In: Kapur, N., Ed., The Paradoxical Brain, Cambridge University Press, Cambridge, 177-188. http://cognitiveconsultantsinternational.com/Dror_PB_paradoxical_human_expertise.pdf
  16. Kurz-Milcke, E. and Gigerenzer, G. (2004) Experts in Science and Society. Kluwer, New York. http://dx.doi.org/10.1007/b105826
  17. de Meyrick , J. (2003) The Delphi Method and Health Research. Health Education, 103, 7-16 http://www.emeraldinsight.com/journals.htm?issn=0965-4283&volume=103&issue=1&articleid=871821&show=html&PHPSESSID=s5sfi9qj2tdm5fotc8633ik153 http://dx.doi.org/10.1108/09654280310459112
  18. Selinger, E. and Crease, R. (2006) The Philosophy of Expertise. Columbia University Press, New York.
  19. Hertz, N. (2010) Why We Rely Too Much on Experts. http://www.smartplanet.com/blog/smart-takes/why-we-rely-too-much-on-experts/
  20. Palgonuri (2012) Should We Rely on Experts? http://www.studymode.com/essays/Should-We-Rely-On-Experts-998077.html
  21. Hakkarainen, K., Palonen, T., Paavola, S. and Lehtinen, E. (2004) Communities of Networked Expertise: Professional and Educational Perspectives. Emerald Group Publishing Limited, Bingley. http://www.citeulike.org/group/382/article/311064
  22. Hsu, C. and Sandford, B. (2007) The Delphi Technique: Making Sense of Consensus. Practical Assessment, Research & Evaluation, 12, 1-8. http://pareonline.net/pdf/v12n10.pdf
  23. Dalkey, N. and Helmer, O. (1963) An Experimental Application of the Delphi Method to the Use of Experts. Management Science, 9, 458-467. http://search.proquest.com/abicomplete/docview/205816685/fulltextPDF/45265BD047B4798PQ/1?accountid=14739
  24. Linstone, H. and Turoff, M. (1975) The Delphi Method: Techniques and Applications. Addison-Wesley, Boston.
  25. Rand Corporation (n.d.) Delphi Method. http://www.rand.org/topics/delphi-method.reports.0.html
  26. Watkins , R. (2011) The Delphi Technique. 137-143. http://ryanrwatkins.com/na/guidebook/Delphi%20technique.pdf
  27. Hyrkäs, K., Appelqvist-Schmidlechner, K. and Oksa, L. (2003) Validating an Instrument for Clinical Supervision Using an Expert Panel. International Journal of Nursing Studies, 40, 619-625. http://www.sciencedirect.com/science/article/pii/S0020748903000361 http://dx.doi.org/10.1016/S0020-7489(03)00036-1
  28. Mathew , S., Field, W. and French, B. (2011) Content Validation Using an Expert Panel: Assessment Process for Assistive Technology Adopted by Farmers with Disabilities. Journal of Agricultural Safety and Health, 17, 227-241. http://elibrary.asabe.org/azdez.asp?JID=3&AID=38184&ConfID=j2011&v=17&i=3&T=2&redirType http://dx.doi.org/10.13031/2013.38184
  29. Basger, B., Chen , T. and Moles, R. (2012) Validation of Prescribing Appropriateness Criteria for Older Australians Using the RAND/UCLA Appropriateness Method. British Medical Journal Open, 2, Article ID: e001431. http://bmjopen.bmj.com/content/2/5/e001431.full
  30. Navarro, A., Voetsch, K., Liburd, L., Giles, H. and Collins, J. (2007) Charting the Future of Community Health Promotion: Recommendations from the National Expert Panel on Community Health Promotion. Preventing Chronic Disease, 4. http://www.cdc.gov/pcd/issues/2007/jul/07_0013.htm
  31. Powell , C. (2003) The Delphi Technique: Myths and Realities. Journal of Advanced Nursing, 41, 376-382. http://rachel.org/files/document/The_Delphi_Technique_Myths_and_Realities.pdf http://dx.doi.org/10.1046/j.1365-2648.2003.02537.x
  32. Rowe , G. and Wright, G. (2001) Expert Opinions in Forecasting: The Rof the Delphi Technique. In: Armstrong , J., Ed., Principles of Forecasting: A Handbook for Researchers and Practitioners, Kluwer, Boston, 125-144. http://www3.nd.edu/~busiforc/handouts/Other%20Articles/expertopinions.pdf
  33. Slocum , N. (2005) Participatory Methods Toolkit: A Practitioner’s Manual on the Expert Panel. King Baudouin Foundation and the Flemish Institute for Science and Technology Assessment, Brussels. http://www.kbs-frb.be/uploadedFiles/KBS-FRB/Files/EN/PUB_1540_Toolkit_7_ExpertPanel.pdf
  34. Ralph , E. (1993) Sensitive, Sensible Practicum Supervision: A Contextual Application in Saskatchewan. The Alberta Journal of Educational Research, 39, 283-296.
  35. Zais, R. (1976) Curriculum: Principles and Foundations. Crowell Harper & Row, New York.
  36. Leedy, P. and Ellis Ormrod , J. (2013) Practical Research: Planning and Design. 10th Edition, Pearson , Toronto.
  37. Gay , L., Mills, G. and Airasian, P. (2012) Educational Research: Competencies for Analysis and Applications. 10th Edition, Pearson , Toronto.
  38. Ralph , E. (2005) Enhancing Managers’ Supervisory Effectiveness: A Promising Model. The Journal of Management Development, 24, 267-284. http://www.emeraldinsight.com/journals.htm?articleid=1463402 http://dx.doi.org/10.1046/j.1365-2648.2003.02537.x
  39. Hussein , H. (2014) Malaysia Sets up New MH370P. http://www.news.nom.co/malaysia-sets-up-new-mh370-panel-9494084-news/
  40. Truss (2014) International Panel of Experts to Re-Examine All Gathered Data on MH370. The Times of India, New Delhi. http://timesofindia.indiatimes.com/world/rest-of-world/International-panel-of-experts-to-re-examine-all-gathered-data-on-MH370/articleshow/34675627.cms
  41. European Commission (2014) Expert Panels. Development and Cooperation, Europe Aid Directorate-General, Brussels. http://ec.europa.eu/europeaid/evaluation/methodology/examples/too_pan_res_en.pdf
  42. Hadler, G. (2013) Solving Problems Using a Group: Advantages and Disadvantages. ITS Education, Hong Kong. http://www.itseducation.asia/groups.htm
  43. Aas, G.H, Askling, B., Dittrich, K., Froestad, W., Haug, P., Hofgaard, L . K. , Moitus, S., Pyykko, R., Sorskar, A. K. (2009) Assessing Educational Quality: Knowledge Production and the Role of Experts. http://www.enqa.eu/indirme/papers-and-reports/workshop-and-seminar/Assessing_educational_quality_wr6.pdf
  44. Kingman, D., Yoder, A., Hodge, N., Ortega , R. and Field, W. (2005) Utilizing Expert Panels in Agricultural Safety and Health Research. Journal of Agricultural Safety and Health, 11, 61-74. http://www.fivehokies.com/Evaluation/Evaluation%20Approaches/Expertise%20Oriented/Utilizing%20Expert%20Panels%20in%20Agricultural%20Safety%20and%20Health%20Research.pdf http://dx.doi.org/10.13031/2013.17897
  45. McKenna , H. (1994) The Delphi Technique: A Worthwhile Research Approach for Nursing? Journal of Advanced Nursing, 19, 1221-1225. http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.1994.tb01207.x/pdf http://dx.doi.org/10.1111/j.1365-2648.1994.tb01207.x