ABSTRACT

Introduction: Given the shortage of primary care physicians, faculties of medicine have been seeking ways to encourage medical students to opt for family medicine as their career choice. The Faculty of Medicine in Geneva decided to incorporate a mandatory one-month rotation during clerkship in a primary care physician’s private office. Such a learning environment offers a privileged opportunity for the student but represents a considerable challenge for clinical teachers. With the goal of identifying training priorities, we sought to investigate the differences in perception of clinical supervision held by clinical teachers and by students. Methods: Cognitive apprenticeship is a well-established theoretical framework for studies on clinical supervision. The Maastricht Clinical Teaching Questionnaire, based on this theory, measures five dimensions of clinical supervision. We used this validated questionnaire to compare student assessment and supervisor self-assessment of the clinical supervision. A total of 275 students over two years assessed their clinical supervision experience provided by a group of 128 clinical supervisors. This hierarchical configuration of the collected data (students were grouped according to the supervisor they had assessed) implied using a linear mixed-effects model approach to assess the significance of observed scores. Results: The results clearly show that students consistently scored their supervisor higher on all five dimensions than the supervisors did themselves. Although it appears that students and supervisors do not focus on the same issues pertaining to clinical supervision, and student expectations might not coincide with supervisors’, the fact remains that they both experience the tasks described in each of the five dimensions together. Conclusion: The results of our study point to some improvements to the training of clinical supervisors.

Keywords:

Clinical Supervision, Self-Assessment, Maastricht Clinical Teaching Questionnaire, Linear Mixed Models Approach, Cognitive Apprenticeship, Primary Care, Faculty Development

1. Introduction

The Faculty of Medicine of the University of Geneva considers it important that medical students, regardless of their ultimate career choice, be exposed to independent general practice. Consequently, the Faculty decided to incorporate a mandatory one-month rotation during clerkship in a primary care physician’s private office (OFSP, 2012). The ambulatory setting offers a privileged learning environment that allows both the observation and the follow-up of acute and chronic illnesses. It also fosters learning of preventive medicine, medical interview communication techniques, and the management of the many psychosocial dimensions of illness (Dent, 2005; Irby, 1995; Ramani & Leinster, 2008). The expected benefits of such a rotation include the personalized access to a population of patients for a sustained period of time and the development of a mentoring relationship with an experienced physician.

The primary care outpatient ambulatory setting represents a considerable challenge as a learning environment for the supervisor and the student. This is because, typically, supervisors have little time for planned and structured teaching, interactions between students and physicians are brief, and supervisors have little or no control over the organization of the working hours, the flow of patients, and the clinical situations (Ramani & Leinster, 2008). We were interested in delving deeper into this specific reality to uncover areas where clinical supervisors might be provided with greater support.

A preliminary study carried out in Geneva (Muller-Juge et al., 2018) showed that primary care physicians have strong motivation to welcome and integrate students in their work environment. This is mainly because they wished to promote primary care practice, and because they enjoyed being the “witnesses” to the “first times” of the budding physicians (the first sutures, first case of pneumonia, etc.). In this perspective, these primary care physicians consider themselves role models and mentors to their students. At the same time, it has quickly become apparent that a more structured approach was required for the supervision of students in such a clinical setting. Muller-Juge et al. (2018) highlighted that these physicians faced significant challenges when they had to adapt to the student’s presence in their workspace. Supervisors didn’t anticipate the cognitive load required to explain their clinical reasoning processes, the supervision planning, and the emotional load not only of having to “share” their patients with a student, but also to take on the responsibility of their emotional reactions (Muller-Juge et al., 2018). It also appeared that these primary care physicians were surprised to find out that it was a challenge to conjugate their two roles: the role of a clinician who takes care of patients, and the role of teacher (Audétat, Grégoire, Fernandez, & Laurin, 2017).

This situation led us to consider developing and delivering a targeted longitudinal Faculty Development program for these primary care physicians. This program includes an initial three-hour workshop required to supervise and welcome students in their practice. They are then invited to targeted workshops and required to participate in a yearly half-day continuous training organised by the primary care unit (UIGP) of the Faculty of Medicine (McLean, Cilliers, & van Wyk, 2010; Steinert & Mann, 2006). During this half-day, aimed at developing a community of practice and to foster the ability to transfer in their context, participants learn needed competencies, as peer-teachers guide them through relevant content, demonstrations, practices and feedback (Yelon, Ford, & Anderson, 2014).

1.1. Our Conceptual Framework

Stalmeijer’s (Stalmeijer et al., 2013) work on clinical supervision focuses on the teacher’s actions to create a safe learning environment, in which students feel free to ask questions and are not under pressure if they fail or make a mistake. The quality of these interactions between supervisor and supervisee should allow feedback and observations, both offered and received in a constructive manner, and promote students’ development of autonomy and reflexive thinking. Clinical teachers are also expected to demonstrate relevant skills, to be good role models, and to develop their thought process out loud, explaining their clinical reasoning. Following this model, supervisors will be encouraged once they feel confident enough to delegate more tasks to the student. Hence, cognitive apprenticeship describes a continuum from shadowing (the learner follows the teacher everywhere, has little to do but observe the teacher’s demonstrations and explanations, to fading (the teacher gradually recognizes the student’s autonomy and progressively reduces support, i.e. a scaffolding, until the end of the training). By doing so, it would allow supervisors to explore and identify some learning gaps regarding specific areas or practice.

Out of this body of work, Stalmeijer and her colleagues developed the five dimension model—the Masstricht Clinical Teaching Questionnaire (MCTQ)—to assess the quality of the clinical supervision, that is role modeling, coaching, articulating, exploration, and learning environment (Stalmeijer, Dolmans, Wolfhagen, & Scherpbier, 2009). Role modeling is embodied by teachers when they actively demonstrate and describe their reasoning while they are working. Coaching refers to teachers observing students and providing specific and concrete feedback on their performance. Articulation involves teachers questioning students (for example about their clinical reasoning) and encouraging them to ask questions. Exploration is aimed at encouraging students to formulate and pursue personal learning goals. Learning environment is about setting a constructive learning climate, which clinical teachers can foster by showing an interest in students’ learning and making students feel respected. All five dimensions require different pedagogical competencies.

1.2. Aim of Our Study

The learning needs’ assessment regarding clinical supervision and the necessity to develop strategies that foster competency development in a complex environment such as primary care outpatient ambulatory settings constitutes our main goal (O’Sullivan & Irby, 2011). As shown above, there are tensions surrounding the dual professional identity that need to be resolved and there is a clear need to develop mentoring relationships with clerks who are inexperienced and who might not be aiming to practice primary care. While clinicians are glad to welcome students in their office, the complexity inherent in the rotation make the task of supervising clerks very arduous. Thus, a more nuanced portrayal of the situation, gleaned from multiple perspectives, might allow us to identify relevant training priorities. Hence, perceptions of the components of the 5 dimensions of the MCTQ held by students and their supervisors were contrasted so that the differences might shed light into specific training needs. To the best of our knowledge, there is little research that has focused on an assessment based on students’ and teachers’ dual perceptions of the quality of clinical supervision.

Hence, our research question was to ascertain whether the supervisors’ self-assessment coincide with the learners’ assessment regarding the behaviors associated with the five dimensions. On which dimensions do perceptions converge and on which do they diverge?

2. Methods

This study was a prospective observational study based on self-administered questionnaires filled out by students and the clinical supervisors.

2.1. Measure

We chose the Maastricht Clinical Teaching Questionnaire (MCTQ) developed by Stalmeijer et al., which is a valid and reliable evaluation instrument (Stalmeijer, Dolmans, Wolfhagen, Muijtjens, & Scherpbier, 2010). It has been also used as a self-assessment instrument for veterinary clinical teachers (Boerboom et al., 2011), and for clinical supervisors from acute and community settings in Australia (Bearman et al., 2018).

Figure 1 presents the MCTQ for the students, which has 14 items grouped in five dimensions. The questionnaire was reworded so as to make it suitable as a self-evaluation tool for the clinical teachers. Both questionnaires consist of essentially the same questions, only the wording was modified from “My clinical teacher …” to “I …” For example the item “My clinical teacher showed that he or she respected me” was turned into “I feel able to show respect for my student.”

in the self-evaluation questionnaire. Each item is assessed with 5-point Likert scale where 1 = fully disagree, 5 = fully agree).

We were aware of the limitations of self-reported data, in particular that they may be widely divergent from real practice (Norman, Norcini, & Bordage, 2014). However, self-reported data indicate how clinical supervisors themselves identify “on the ground” areas for improvement. In this perspective, it is more closely aligned with a pedagogic strategy of self-reporting data rather than the ability to self-assess (Eva & Regehr, 2008).

2.2. Participants

All students at the University of Geneva Faculty of Medicine are required to enroll in a one-month mandatory rotation in outpatient primary care. During this rotation, they are paired up with a physician working in a private practice, who is responsible for supervising their work.

We conducted this research during the academic years (autumn to summer term) of 2016-17 and 2017-18. Students (N = 275 for the two years) were asked to fill in the questionnaire once they had completed their rotation. Supervisors (N = 128) were asked to fill in the questionnaire via Qualtrix at the end of the rotation.

2.3. Analysis

As a first step summary statistics (given as mean were computed ± standard deviation) for every dimension for the students’ questionnaire, respectively the supervisors’ questionnaire. Should one item be missing, we just took the mean of the available items of the same dimension for the evaluation of the mean. We then investigated, using t-tests and for every dimension, the potential differences between the mean of the evaluations by the students and the auto-evaluations.

The basic analyses mentioned above do not take a potential supervisor effect, i.e. for a given supervisor the evaluations and auto-evaluation may be very similar, whereas for another the evaluations could be much higher than the auto-evaluation. Since there were potentially several evaluations available for every supervisor, we thus investigated the link between the evaluations and the auto-evaluation using a linear-mixed effects model, taking the evaluation as the dependent variable, the auto-evaluation as the independent variable, and incorporating a supervisor random effect (on both the intercept and the slope). A perfect theoretical adequacy between them would thus lead to a regression of slope one and intercept zero, and a minimal between-supervisor variability. Sub analyses, using the same methodology, were also done for each of the five dimensions.

All the analyses were performed with R version 3.3.3 (The R Foundation for Statistical Computing, Vienna, Austria).

3. Results

A total 275 questionnaire completed by the students and 50 completed by the supervisors were used in the analysis. Supervisors may have supervised more than one student, so that 50 auto-evaluations were completed by 33 supervisors.

3.1. Evaluations and Auto-Evaluations

Table 1 presents the averages (± standard deviation) for each of the five dimensions of the Stalmeijer model obtained from the 275 students and the 33 supervisors.

All average scores were higher than 3, with the higher scores observed for the climate, and the lower for the clinical reasoning. Apart from coaching, the evaluations made by the students scored higher than the self-evaluation by supervisors.

3.2. Association between Evaluations and Auto-Evaluations

The linear mixed-effects model showed an average evaluation of 3.54 ± 0.45 (p < 0.0001) with no evidence of a link with the auto-evaluation (p = 0.882), see Figure 2. The evidence of a significant difference between-supervisor variability was weak (p = 0.054; 9.1% of variance explained).

Results were similar for all five dimensions of Stalmeijer’s model.

4. Discussion

Both students and supervisors answered the Maastricht Clinical Teaching Questionnaire. The results clearly show that students consistently scored their supervisor higher on all five dimensions than the supervisors did themselves. Several reasons may explain these findings: First, the difference in scores may be indicative

that trainees and supervisors need to take more time to discuss and express each other’s needs and expectations for the rotation. It seems to be still rare for supervisors to make explicit what they expect from their students. Second, students are quite used to judging their teachers and supervisors in the academic context. They regularly fill in forms on which they express their satisfaction (or not) with different aspects of the teaching and learning experience. The context of the private practice is somehow different: clinical supervisors in private practice are not used to be assessed, neither to assess themselves, and as our results show, do so more severely than students.

The results are such that we do not really know whether the rotation went as well as the students said or as badly as the supervisors said. This discrepancy may point to a lack of reflexive habit of exploring one’s teaching performance. In this sense, our results suggest that we need to reinforce teachers’ reflexive practice in order to have a more balanced view of their teaching practice.

It is interesting to note that the students gave their highest scores on the dimension of learning environment, and the dimension of role modeling, (see Table 1). As did the clinical teachers: This is congruent with the Bearman’s results (Bearman et al., 2018). It is satisfying that both students and clinical teachers recognized the significance of the relationship in the clinical teaching. Kilminster, Cottrell, Grant, and Jolly (2007) identified these type of skills or qualities as contributing to effective supervision. Teachers gave their lowest scores on the articulation dimension while students assessed it higher. This leads us to hypothesize that the support for the development of clinical reasoning, at the heart of the medical profession, could be improved. These results could be interpreted as a call to improve skills in intervening and supporting students as they struggle to acquire clinical reasoning skills. These skills would support adoption of a professional approach to teaching rather than the intuitive approach that appears to be reflected in these findings as well as in other results (Atkinson, Ajjawi, & Cooling, 2011; Audétat, Laurin, Dory, Charlin, & Nendaz, 2017).

It is also worth reporting that both supervisors and students score themselves low on the exploration dimension. Encouraging students to formulate and pursue personal learning goals is often overlooked; these findings are congruent with other research highlighting that supervisors tend to fail to formally conclude the supervision and struggle to explain to the student what has been learned and what remains to be learned (Pereira Miozzari, Rieder, Sader, Sommer, & Audétat, 2020 ).

Finally, students gave a low score on the coaching dimension, while clinical teachers gave themselves a good score on the same dimension. These findings confirm previous results showing that teachers usually believe they provide regular and sufficient feedback, but this is not how it is perceived by learners (Cantillon & Sargeant, 2008; Liberman, Liberman, Steinert, McLeod, & Meterissian, 2005). Providing specific and concrete feedback is the cornerstone of effective clinical teaching (Hesketh & Laidlaw, 2002). Clinical teachers should consider feedback as a crucial skill to be acquired through repeated practice and augmented by reflection on their own performance (Cantillon & Sargeant, 2008). Such critical skills could also be the subject of focused training or coaching sessions.

Strengths and Weaknesses of Our Study

The major limitation of the study is the relatively low number of self-assessments: 33 out of 128. The questionnaire was disseminated via institutional e-mail and there was no obligation for the clinical teachers to fill it out. Another approach should be sought in which more incentive is provided to the supervisors to take the time to fill out the questionnaire.

5. Conclusion

The results of our study point to some improvements to the training of clinical supervisors. Although it appears that students and supervisors do focus on the same issues pertaining to clinical supervision, and student expectations might not coincide with supervisors’, the fact remains that they experience the same tasks described in each of the five dimensions of the model. The divergence of opinion contains many insights into what happens and more importantly on what could be improved upon.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

Cite this paper

Audétat, M.-C., Cerutti, B., Rieder, A., & Fernandez, N. (2020). Perceptions of Clinical Supervision in Ambulatory Settings: Do Supervisee and Supervisors See Eye to Eye? Creative Education, 11, 297-306. https://doi.org/10.4236/ce.2020.113023

References