Comparison of Assessment Scores of Candidates for Communication Skills in an OSCE, by Examiners, Candidates and Simulated Patients

Abdul Sattar Khan, Riaz Qureshı, Hamit Acemoğlu, Syed Shabi-ul-Hassan

Department of Family Medicine, Ataturk University, Erzurum, Turkey.
Department of Family Medicine, King Saud University, Riyadh, Saudi Arabia.
Department of Family Medicine, Riyadh Military Hospital, Riyadh, Saudi Arabia.
Department of Medical Education, Ataturk University, Erzurum, Turkey.

DOI: 10.4236/ce.2012.326141                

Abstract

Though OSCE method has been verified by several researchers for the appropriate assessment of competence in clinical skills, yet medical educationists have some concerns regarding the value of assessment of communication skills and empathy by this method. Hence, we sought to assess the extent of differences, if any, among the examiners, the candidates and the simulated patients (SPs) for communication skills. A total of 23 general practitioners, who were preparing for their postgraduate clinical examination, participated in a practice OSCE on seven stations in this study. The examiners observed and evaluated the candidates during the whole consultation, using the pre-tested checklist including 15 items with a global rating scale. The simulated patients also evaluated the candidates at the end of consultation, using the same checklist. There were significant differences in the assessment scores of candidates by the examiners, the candidates themselves and the simulated patients regarding all aspects of communication skills. However, introduction to the patient’s scenario of some non-verbal communication did not show any significant difference (p-value => 0.05). The correlation between examiners and SPs (r = 0.07, p = 0.7) and SPs and candidates (r = 0.01, p = 0.95) was very low and not significant. Cronbach’s alpha was 0.968 across items, whereas among seven stations it was 0.931. This study has shown a significance difference in assessment scores of candidates by examiners, SPs and candidates themselves. In conclusion, there is a need for further research regarding the active role of SPs in summative assessments.

Keywords

Assessment by Simulated Patients; Empathy; Communication Skills; OSCE

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Allen, R., Heard, J., & Savidge, M. (1998). Global ratings versus checklist scoring in an OSCE. Academic Medicine, 73, 597-598. doi:10.1097/00001888-199805000-00067
[2]	Bianchi, F., Stobbe, K., & Eva, K. (2008). Comparing academic performance of medical students in distributed learning sites: The McMaster experience. Medical Teacher, 30, 67-71. doi:10.1080/01421590701754144
[3]	Bokken, L., Linssen, T., Scherpbier, A., Van der Vleuten, C., & Rethans, J. J. (2009). Feedback by simulated patients in undergraduate medical education: A systematic review of the literature. Medical Education, 43, 202-210. doi:10.1111/j.1365-2923.2008.03268.x
[4]	Bokken, L., van Dalen, J., & Rethans, J. J. (2010). The case of “Miss Jacobs”: Adolescent simulated patients and the quality of their role playing, feedback, and personal impact. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 5, 315-319.
[5]	Brannick, M. T., Erol-Korkmaz, H. T., & Prewett, M. (2011). A systematic review of the reliability of objective structured clinical examination scores. Medical Education, 45, 1181-1189. doi:10.1111/j.1365-2923.2011.04075.x
[6]	Chumley, H. S. (2008). What does an OSCE checklist measure? Family Medicine, 40, 589-591.
[7]	Fischbeck, S., Mauch, M., Leschnik, E., Beutel, M. E., & Laubach, W. (2011). Assessment of communication skills with an OSCE among first year medical students. Psychotherapie, Psychosomatik, Medizinische Psychologie, 61, 465-471. doi:10.1055/s-0031-1291277
[8]	Harden, R. M., & Gleeson, F. A. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13, 41-54. doi:10.1111/j.1365-2923.1979.tb00918.x
[9]	Hatala, R., Marr, S., Cuncic, C., & Bacchus, C. M. (2011). Modification of an OSCE format to enhance patient continuity in a highstakes assessment of clinical performance. BMC Medical Education, 11. doi:10.1186/1472-6920-11-23
[10]	Jahan, F., Sadaf, S., Bhanji, S., Naeem, N., & Qureshi, R. (2011). Clinical skills assessment: Comparison of student and examiner assessment in an objective structured clinical examination. Education for Health (Abingdon, England), 24, 421.
[11]	Mazor, K. M., Ockene, J. K., Rogers, H. J., Carlin, M. M., & Quirk, M. E. (2005). The relationship between checklist scores on a communication OSCE and analogue patients’ perceptions of communication. Advances in Health Sciences Education, 10, 37-51. doi:10.1007/s10459-004-1790-2
[12]	McNaughton, N., Tiberius, R., & Hodges, B. (1999). Effects of portraying psychologically and emotionally complex standardized patient roles. Teaching and Learning in Medicine, 11, 135-141. doi:10.1207/S15328015TL110303
[13]	Moineau, G., Power, B., Pion, A. M., Wood, T. J., & Humphrey-Murto, S. (2011). Comparison of student examiner to faculty examiner scoring and feedback in an OSCE. Medical Education, 45, 183-191. doi:10.1111/j.1365-2923.2010.03800.x
[14]	Pierre R. B., Wierenga A., Barton M., Thame K., Branday J. M., & Christie C. D. C. Student self-assessment in a paediatric objective structured clinical examination. West Indian Medical Journal, 54, 144-148.
[15]	Regehr G., Eva K. (2006). Self-assessment, self-direction and selfregulating professional. Clinical Orthopedics Related Research, 449, 34- 48.
[16]	Regehr, G., Freeman, R., Robb, A., Missiha, N., & Heisey, R. (1999). OSCE performance evaluations made by standardized patients: Comparing checklist and global rating scores. Academic Medicine, 74, 135-137. doi:10.1097/00001888-199910000-00064
[17]	Robb K. V., Rothman A. (1985). The assessment of history-taking and physical examination skills in general internal medicine residents using a checklist. Ann Royal Coll Phys Surg Canada, 20, 45-48.
[18]	Rosen, K. R. (2008). The history of medical simulation. Journal of Critical Care, 23, 157-166. doi: doi:10.1016/j.jcrc.2007.12.004
[19]	Schwartzman, E., Hsu, D. I., Law, A. V., & Chung, E. P. (2011a). Assessment of patient communication skills during OSCE: Examining effectiveness of a training program in minimizing inter-grader variability. Patient Education and Counseling, 83, 472-477. doi:10.1016/j.pec.2011.04.001
[20]	Schwartzman, E., Hsu, D. I., Law, A. V., & Chung, E. P. (2011b). Assessment of patient communication skills during OSCE: Examining effectiveness of a training program in minimizing inter-grader variability. Patient Education and Counseling, 83, 472-477. doi:10.1016/j.pec.2011.04.001
[21]	Siaja, M., Romi, D., & Prka, Z. (2006). Medical students’ clinical skills do not match their teachers’ expectations: Survey at Zagreb University School of Medicine, Croatia. Croatian Medical Journal, 47, 169-175.
[22]	Taras, M. (2002). Using assessment for learning and learning from assessment. Assessment Evaluation in Higher Education, 27, 501- 510. doi:10.1080/0260293022000020273
[23]	Thistlethwaite, J. E. (2002). Developing an OSCE station to assess the ability of medical students to share information and decisions with patients: Issues relating to interrater reliability and the use of simulated patients. Education for Health, 15, 170-179. doi:10.1080/13576280210133099
[24]	Townsend, A. H., McIlvenny, S., Miller, C. J., & Dunn, E. V. (2001). The use of an objective structured clinical examination (OSCE) for formative and summative assessment in a general practice clinical attachment and its relationship to final medical school examination performance. Medical Education, 35, 841-846. doi:10.1046/j.1365-2923.2001.00957.x
[25]	Watson, M. C., Skelton, J. R., Bond, C. M. (2004). Simulated patients in the community pharmacy setting. Pharmacy World & Science, 26, 32-37. doi:10.1023/B:PHAR.0000013467.61875.ce