Attitudes of Nursing Faculty Members and Graduates towards the Objective Structured Clinical Examination ( OSCE )

The aim of this study was to evaluate the attitudes of faculty and graduates of college of nursing towards Objective Structured Clinical Examination (OSCE) as a valid and reliable method of clinical competency assessment in nursing ten years after its inception. Core nursing courses are based on instructional teaching methods and “hands-on” approach to impact cognitive, psychomotor skills and clinical judgments. Different clinical competency assessment methods are used globally; however, most of them are subjective. A descriptive survey using 16-item five-point likert scale questionnaire was conducted. The study sample consisted of 140 participants: 20 faculty members, 27 graduates of the Bachelor of Science in Nursing (BSN) Degree and 93 graduates of the Associate Degree of Nursing (ADN) programs with OSCE experience during their training. Data collection was carried out between October and November 2015. Eighty percent of faculty members, 74% of BSN and 62.3% of ADN graduates agreed that OSCE represented an objective evaluation method for psychomotor skills. Majority of the graduates perceived their OSCE experience positively although stressful. However, they have suggested the introduction of trial/mock OSCE prior to each exam to minimize the stress associated with it. In conclusion, there appears to be no single “gold-standard” assessment tool for clinical competency. OSCE assesses student nurses’ psychomotor skills in a non-clinical environment, therefore without risks to real patients. In combination with other assessment methods in the clinical settings, OSCE will provide a more comprehensive student psychomotor skill evaluation. OSCE experiences gave new nursing graduates confidence to work as registered nurses in health care settings.


Introduction
Nursing education is not only based on imparting knowledge in the cognitive domain but also in the affective and psychomotor domains.Nursing involves "hands-on" psychomotor skills and clinical judgments in the affective and psychomotor domains which are acquired through laboratory sessions in the educational institutions and experiences in different clinical settings.In the clinical settings, nursing students are supervised by nurse educators, trainers and clinical instructors and they work alongside other registered nurses to acquire and improve their psychomotor skills on real patients.In order to ascertain high professional standards, nursing students should demonstrate mastery in cognitive, affective and psychomotor domains during training.Traditionally, most nursing programs evaluate students' psychomotor skills using practical examination in the form of skill check on real patients in clinical settings [1] [2].Procedures on which students are assessed are usually not predetermined because they are based on the care that their assigned patients require at the time of their skill check.It is also impossible for all the students to be examined on the same nursing procedures and on the same patients.Utilizing different patients, different procedures, with different difficulty levels introduce some elements of subjectivity in the assessment process.Some researchers have suggested the use of simulated patients for evaluating clinical performance of newly qualified nurses to ensure they have obtained beginning level of competence for graduates of nursing programs [3].Most nursing institutions in addition to their faculty members and their clinical instructors utilize preceptors in the clinical settings for the training of students.These preceptors are usually senior registered nurses with no prior training on measurement and evaluation of nursing skills; therefore their assessment of students' practical skills may be questionable because they may be based on their relationship and general impression made from their interactions with the students [4].The disadvantage of the traditional method is that if a student is unlucky and does not perform well in two or three skill checks, he or she is likely to fail.Whereas, in OSCE with six or more stations, if a student does badly in a couple of stations, but does well in the other stations, he or she is likely to pass.The College of Nursing (CON), Kuwait, introduced OSCE to complement the traditional psychomotor skill evaluation for Fundamental of Nursing course for the first set of the Bachelor of Science in Nursing Degree (BSN) students in April 2005.Thereafter, all nursing courses in both BSN and Associate Degree of Nursing (ADN) programs started implementing OSCE for 10% of the course marks to complement their existing multiple clinical competence assessments tools in the form of psychomotor skill check, writing of Nursing Care Plans, client health teaching and other activities in the clinical settings which account for 30% of the course marks, while the remaining 60% of the nursing course marks are assigned to theoretical examinations.The CON utilizes multi stations OSCE of 5 minutes duration per station, the questions include both demonstrations of psychomotor skills and answering of written psychomotor skill questions.The evaluation methods for the demonstration stations are by specific task oriented checklist generated by the individual nurse educator, based on the steps for the procedure using recommended nursing skill textbooks.The uses of OSCE as a method of assessing all the students on the same sets of questions, using the same answer checklist for grading knowledge and skills on the same mannequins, models and trained actors as patients, within the same time frame in a simulated environment by trained nurse educators have no doubt a moderating effect on the bias created by hospital based skill check assessment.OSCE in CON is a summative evaluation.Nurse educators are expected to develop mastery of their competencies in their specialty through continuous education, research and working alongside and under the supervision of senior colleagues.The process of preparing and conducting OSCE gives younger nurse educators the opportunity to identify their personal nursing competency gaps.The process of conducting OSCE as an assessment tool for clinical competence emphasizes to both students and faculty members the importance placed on psychomotor skills in healthcare professions [5] [6].
Evaluating students in the cognitive domains is usually straight forward; however, evaluating the affective and psychomotor domains can be challenging bearing in mind that the focus of these domains are on the concept of clinical competence assessment."Competence" is a vague concept defined differently depending on the person's motives [4].Clinical competence is an essential component of the health care industries.The search for an ideal clinical competence assessment tool for undergraduate medical education led to the development of OSCE in Dundee [7] and it is widely used in both undergraduate and post graduate medical education world-wide.Many other health care and none health care professions have adapted OSCE to fit their professions in terms of objectivity, reliability and validity in measuring clinical & professional competences.OSCEs have been reported to be feasible with good objectivity, reliability and validity indices [8]- [12].OSCE is also now internationally recognized as a measure of clinical competence that focuses on outcomes through observable behaviours [13].OSCE can be used in either a summative, formative or mixed mode [14].Both formative and mixed modes OSCEs are ideal for teaching, mock exams and review courses where the candidates are still preparing to be assessed.However, the summative mode is ideal for evaluation and detection of competency gaps that need to be corrected for safe clinical practice.The use of OSCE in assessing professional clinical competences is necessary to fulfill the requirements for effective nursing education in a non-threatening, non-clinical environment.OSCE is flexible, versatile, and has the ability to be standardized in any given scenario allowing the assessment to focus on clear outcome criteria [13] [14].OSCE stations can either test specific psychomotor skills or require the candidate to answer written questions related to psychomotor skills.Photographs of OSCE stations in emergency and critical care modules have been provided by Wallace & Zahran [15] while Shadia Yousef has shown photo album of pediatrics OSCE [16].Rushford's study visit to 16 nursing institutions identified four nursing OSCE designs: multi station OSCEs, scenario based OSCE, simulated assessment with random allocation, "top to toe" assessment of all body systems [11].Three of the four OSCE designs allow candidate to rotate through all stations answering the questions or performing tasks specified during the uniform timed period.Stations may contain actors, models, mannequins, medications, instruments, or equipment for nursing procedures.The stations with examiners are usually skill stations and the examiners have a marking tool with which to grade the candidate according to the steps for performing the task.Some investigators have queried the objectivity of assessment tools like the checklist for not sufficiently differentiating candidates' mastery of skills which have led to the suggestions of including global rating to checklists [11].Another researcher investigated the use of electronic marking of OSCE using global rating scale to overcome examiners' bias and providing candidates with instant feedback [17].Grading of each step of the check list into four categories according to student's degree of dexterity: zero (0) mark, 1/4 mark, 1/2 mark and full (1) mark have been used by some researchers [2].Students have admitted that OSCEs have assisted them to develop self-confidence in their psychomotor skills thus preparing them for practice after graduation [14] [18].OSCE is also said to reduced candidate's chance of being examined on an easy or difficult skills, as every candidate gets tested on all the stations, thus decreasing assessment bias [11] [19].The most frequently reported disadvantages of OSCE by most investigators are increased level of stress experienced by students during OSCE, financial implication of running OSCE and the demand on the lecturers to ensure validity and reliability of the OSCE process [2] [4] [9]- [12] [14] [17] [21].
In the United Kingdom (UK), The Nursing and Midwifery Council (NMC) recommended OSCE as a means of assessing students' knowledge, and also for evaluating nursing competency [22] [23].However, there were concerns over the artificial conditions under which OSCEs are conducted as they "do not necessarily extrapolate to performance in clinical practice" [24].
In Canada, nursing regulating bodies for registering and licensing professional nurses are now adopting summative OSCE for assessing competency levels of internationally trained professionals' applying for registration in order to identify any competency gaps [25].
In Africa, the department of Nursing Science at the University of Zambia, introduced OSCE in 2012/2013 academic year, for assessing clinical competence for their generic nursing students [26].Similarly, there was a report from Nigeria stating that OSCE is only being used in Midwifery examinations [2].
Search for studies on OSCE in Nursing, in the Arabian Gulf region, revealed pediatric OSCE photo albums at King Abdul Aziz University, Jeddah, Saudi Arabia [16], OSCE in Maternity and Community Health Nursing: Saudi Nursing Student's Perspective [20].Third year nursing students' perception of their first OSCE experience conducted in December 2013 at Prince Khalid Bin Sultan College [21].A workshop on conducting OSCE, organized for nurse educators at the Sultanate of Oman.The aim of the workshop was to train and encourage nurse educators from the nursing institutes on how to conduct OSCE with the hope that OSCE will be incorporated in their nursing curriculum as an assessment tool for clinical skills [27].Nursing students especially in their first year have reported dreading their first OSCE.An article by a first year student at the University of Southampton vividly describes the way many students feel about their first OSCE in these words, "the very thought of OSCE strikes terror into my heart" [28].Street & Hamilton also described first OSCE as "a daunting task" [29].One of the strategies suggested to overcome the fear of OSCE is by adequate preparation or coaching of the students before OSCE [28] [29].We hope that reporting our 10 years' experience of OSCE from the perspective of our nursing graduates will encourage other nursing institutions to adopt the use of OSCE, even if only as a trial.

Study Objective
The objective of this study was to evaluate the attitude and perception of CON faculty and graduates towards their OSCE experiences as an objective method of clinical competency and skill assessment in nursing.
The study will be testing the following null hypotheses: The main hypothesis: There is no significant difference in the perception of the 3 groups (faculty members, BSN and ADN graduates regarding OSCE as an objective tool for clinical competence assessment.

Sub hypotheses:
1: There is no significant difference in the perception of the three groups, that OSCE assesses all students objectively.
2: There is no significant difference in the perception of faculty, BSN and ADN graduates that OSCE prepares nursing students after graduation to cope with the challenges of their roles as registered nurses in the healthcare industries.

Methods and Participants
A cross-sectional survey method was used in collecting data for this study.

Participants:
A convenient sample of 140 subjects from three main groups: 20 nursing faculty members involved in conducting OSCEs, 27 BSN graduates and 93 ADN graduates who were assessed using OSCE during their training programs participated in the study.
Inclusion criteria were graduates or faculty of the CON, with experience in either conducting or taking OSCEs during their training in CON and must be registered and practicing nurses in health care facilities.Although there are over 1000 CON graduates currently employed in the different Ministry of health hospitals, only 500 of them graduated in the last 10 years after the introduction of OSCE.Out of these, 125 returned their questionnaires but 5 questionnaires were incomplete and were discarded.The response rate was therefore 25% of graduates and 100% of faculty members.
Exclusion criteria were graduates from CON before the introduction of OSCE and graduates after the introduction of OSCE but who are not currently working as registered nurses.

Data Collection
Data collection was in October and November 2015.
Anonymous self-administered questionnaire designed in English language was used for data collection.It consisted of two sections: section one consisted of 8 questions on participants' bio-demographic characteristics.The second section consisted of 16-item structured questionnaire using five point likert scale: where 1 means "strongly disagree" and 5 means "strongly agree" to enquire about their perception of the objectivity and usefulness of OSCE in assessing knowledge, attitude, skills and preparation for entry into practice as trained nurses.Scores of 4 and 5 were considered positive responses.There was one open ended questions for participants to give their suggestions regarding the content and or conduct of OSCE at the CON.Content validity was assessed by 3 faculty members, and thereafter the questionnaire was pilot tested by 10 graduates and 2 faculty members, ambiguous statements and questions were rephrased for clarity.Data collection for graduates was done in their different hospitals.The researcher having received permission from the hospital nursing directors met the graduates at the lecture hall of the staff development unit in each hospital.The purpose of the study was explained to them and questionnaires were distributed.They were asked to clarify any doubt with the researcher who was physically present during the data collection and all the completed questionnaires were collected by the researcher.Reliability for the 24 items on the questionnaire by Cronbach's alpha was 0.843.

Ethical Considerations
Institutional approval was sought and obtained from the CON research committee.All nursing faculty members were contacted individually; their consents were sought and also received.Questionnaires were given to them individually and completed questionnaires were returned to the researcher.The nursing directors at the ministry of health hospitals were contacted and access sort to distribute questionnaires to graduates of CON who had ex-perience in OSCEs during their training.The nursing directors gave approval and also instructed their staff development units to compile lists of CON graduates and organize days for them to meet with the researcher.

Data Analysis
Statistical Package for Social Sciences (SPSS) version 21 for word was used to analyse the data.Descriptive statistics used for demographic data of participants were frequencies and percentages.In testing the null hypotheses of this study, Kruskal-Wallis and Mann-Whitney U tests were the preferred choice of statistical analysis because of the unequal sample sizes of the three groups of participants (20 faculty members, 27 BSN graduates and 93 ADN graduates) at the mean significance of P = 0.05 level.

Participants
The study participants' age group ranged from 21 to more than 60 years, and their gender distribution was 110 (78.6%) females and 30 (21.4%) males.Twenty (14.3%) were faculty members, 93 (66.4%) were graduates of the ADN program while 27 (19.3%)were graduates of BSN degree program.Participants' years of nursing experience ranged from less than 5 years to more than 21 years.Majority of our graduates working at the ministry of health hospitals are staff nurses, but holders of BSN and post graduate degrees with many years of experience have administrative positions from head nurses to directors of nursing.
Table 1 shows majority of the respondents in the three groups were in agreement with the following three of the five survey statements: OSCE being an objective method of assessing clinical competence (80% of faculty members, 74% of BSN graduates and 62.3% of the ADN graduates).The second survey statement was: "OSCE is a useful inclusion in nursing curriculum" (85% of faculty members, 92.6% of BSN graduates and 79.6% ADN graduates) and finally, the third survey statement: "OSCE helps students develop confidence in practicing taught skills in the clinical settings" was agreed by 70% of faculty members, 63% BSN graduates and 69.8% of ADN graduates.Three other survey statements portraying OSCE as transparent, bias free, fair to all students and it should continue to be summative were not agreed by most participants: 90% of faculty members said OSCE was transparent, bias free and fair to all students.63% of BSN graduates agreed that OSCE is transparent and bias free and 77.8% said it was fair.However, only 46.2% and 49.5% of ADN graduates agreed on OSCE as transparent and bias free, and being fair to all students.Finally, on the conduct of OSCE, while 80% of faculty members said it should continue to be summative, only 51.8% of BSN graduates and 36.6% of ADN graduates felt it should continue to be summative.

Testing the Null Hypotheses
Main Hypothesis: There is no significant difference in the perception of the 3 groups (faculty members, BSN and ADN graduates regarding OSCE as an objective tool for clinical competence assessment.
Null Hypothesis 1: There is no significant difference in the perception of the three groups, that OSCE assesses all students objectively.
The result of Kruskal-Wallis test in evaluating the differences among the 3 groups was significant.Χ 2 (df 2, N = 140) = 8.421, P = 0.015.Follow-up test to evaluate pair wise differences among the 3 groups controlling for Type 1 error across tests by using Bonferroni approach indicated significant differences between faculty and ADN groups (U = 564.5,Z = −2.632,P = 0.008).However, between BSN and ADN groups, no significant difference was noted (U = 1036.000,Z = −1.735,P = 0.083) Also, between faculty and BSN groups, no significant difference was noted (U = 225.000,Z = −946, P = 0.344).Because the P-value between the two groups of graduates and between faculty and graduates groups are not significant, the null hypothesis is accepted.
Null Hypothesis 2: The perception that OSCE prepares students after graduation to cope with the challenges of working in health care industries is the same across the three groups of participants: faculty, BSN and ADN graduates.
Kruskal-Wallis test to evaluate the differences among faculty, BSN and ADN graduates on "OSCE prepares students for the challenges of working as registered nurses was significant".Χ 2 (df 2, N = 140) = 46.001,P = 0.000.Follow-up test to evaluate pair wise differences among the 3 groups controlling for Type 1 error across tests by using Bonferroni approach.The results indicated significant differences between faculty and ADN groups (U = 51.500,Z = −6.660,P = 0.000).Also, between faculty and BSN group, (U = 23.000,Z = −5.514,P = 0.000).However, there is no significant difference in the perception between the 2 groups of graduates (BSN and ADN) (U = M1178.5,Z = −795, P = 0.427).Because the P-value between the 2 groups of graduate is more than 0.05, the null hypothesis is accepted.

Analysis of the Comments and Suggestions Made by Participants
Multiple comments and suggestion were given by the participants regarding the contents and conduct of OSCEs.The main suggestions from the faculty members were regarding equipment and content of OSCE questions.One faculty member suggested the use of more interactive mannequins and other faculty members wanted more observation of psycho motor skill stations.The graduates were more concerned about the 5 minutes timing of each station which they felt was insufficient to demonstrate the required skills.They were also concerned about stress experienced during OSCEs and strategies to reduce anxiety during OSCE examinations.Some were concerned about the questions on the stations and suggested that the focus should be more on the performance of nursing skills in preference to writing answers to questions.Twenty two (15.7%) of the participants' comment on the 5 minutes timing of each stations included: the need to increase the time, the need to unify the questions on each station, some were said to be more difficult than others, therefore requiring more time.Some participants would like time given at the end of the OSCE before submitting the answer booklets for students to re-check their answers before submission.Time should be given for changing stations and the timers should be more flexible in allowing one or two extra minutes per station.To sum up the time issue a participant stated "the time limitation causes stress and confusion for the students during OSCEs".Another participant stated "OSCE causes depression for the students because of stress, not enough time and no questions allowed".Fifteen (10.7%) of the participants were concerned about strategies to minimize anxiety and stress during OSCEs.They all suggested the need for the subject teacher to arrange for mock or trial OSCEs using mixed mode before the real OSCE to give students practice on the type of questions for the subject.
Fifteen (10.7%) of the participants were concerned about the content of OSCE.They claim the questions should only consist of demonstrations and observation of psychomotor skills, and no inclusion of knowledge (writing) answers.One participant has suggested the inclusion of some fun questions to minimize anxiety and panic.Another participant suggested that students should be involved in the preparation of OSCE exams, and if they are part of the OSCE experience, there will be no fear or stress of OSCEs.
Regarding OSCE format whether it should remain summative or be changed to mixed mode.Majority of the faculty and BSN graduates were in agreement that mix mode OSCE is inadvisable.The reasons given included: one or two minutes after each station will not be enough for meaningful feedback under tense exam conditions.Pointing out mistakes to the students while the exam is still in progress will interrupt their thought process and make them more confused for the remaining stations.Some students may become defensive and argumentative, insisting their actions were how they observed the procedure being performed in the hospitals.Discussion can also disrupt the serene exam environment and distract the other students in other stations.Finally, some students who may be performing below standard may become emotional and decide to stop their OSCE without completing all the stations if they feel they were already failing.

Discussion
In the present study, most of the participants: 80% of faculty members, 74% of BSN graduates and 62.3% of the ADN graduates agreed that OSCE represented an objective evaluation method for psycho-motor nursing skills (Table 1).The result of the main hypothesis tested revealed as predicted, a P-value of <0.117 showed there was no significant difference between the three groups perception as to OSCE being an objective method of assessing clinical competence when compared to traditional clinical skill checks.The latter method is graded based on general impression made from their interaction with the students and without any marking scheme.Nursing OSCEs utilize task specific checklist which they believe minimizes subjectivity.A researcher claimed that some authors have queried the objectivity of the checklist, where examiners tick the steps for performing the task [11].Some other investigators also agree that ticking the checklist does not sufficiently differentiates candidates' mastery of skills and have recommended including global rating to checklists and have suggested grading of each step of the check list into four categories: zero (0) mark, 1/4 mark, 1/2 mark and full (1) mark according to degree of performance of each step [2].Some investigators are of the opinion that OSCE is fairer than the traditional approach due to the standardisation of tasks that have to be performed by the students.They also claimed that the use of structured marking sheets contribute to improvements in reliability and content validity [10].The use of electronic marking scheme with global rating scale was being explored as a means to diffuse examiners bias during OSCE grading [15].The different viewpoints of researchers regarding OSCE makes it difficult to conclude that OSCE is the "Gold standard" for clinical competence assessment Survey statements: OSCE is fair, transparent and bias free, were supported by 90% of faculty members.However, only 63% and 77.8% of BSN graduates agreed that OSCE is transparent and bias free and fair respectively Also, only 46.2% and 49.5% of ADN graduates agreed with the statements.The result of testing the hypothesis showed the P-value of 0.012 for fairness of OSCE, and 0.025 for transparency and bias free, meant that not all participants agreed that OSCE being a fair, transparent and bias free assessment tool for students' psychomotor skills contrary to one researcher's report which showed decreased bias in assessment of clinical competence by using OSCE [19].In the present study, the survey statement: OSCE helps students develop confidence in practicing taught skills in the clinical settings,was agreed by 70% of faculty members, 63% BSN graduates and 69.8% of ADN graduates also, the P-value was 0.391.Majority of the ADN graduates are working as registered nurses in clinical settings and are therefore in a better position to state how OSCE assisted them as new registered nurses.This findings support the view point of some researchers [14] [18], which agreed that graduates with OSCE experience develop more confidence in their psychomotor skills when joining the work force and they feel better equipped for the challenges of working as registered nurses in different healthcare settings.
Analysis of the open ended questions showed that 10.7% of the student would not like the real OSCE exam to be formative, because getting immediate negative feedback may affect their performance in the remaining stations.However, they would like general feedback pointing out the different errors by the whole group anonymously at the completion of the OSCE examination.They also would like mock formative OSCE to prepare them for their summative OSCE exams supporting the findings some researchers [14] [18].
In this study 10.7% of the participants were concerned about minimizing anxiety and stress during OSCEs.Most of the literature reviewed reported participants stress was one of the disadvantages of OSCEs [1] [4] [8]- [12] [14] [15] [18].Students who have reported their OSCE experiences have use the following terms to describe their feelings: "the very thought of OSCE strikes terror into my heart" and OSCE as "a daunting task" [25] [26] Students who have not prepared adequately for any form of evaluation will experience some degree of stress, be it routine quiz or final theory examination.Suffice to say that all the reports have recommended trial or mock OSCEs as the solution to minimize anxiety and stress associated with OSCEs, these trials are hoped to prepare the students adequately for their OSCE examinations.All the participants in this study also recommended the need for the subject teachers to arrange for mock or trial OSCEs using mixed mode before the real OSCE to give students practice on the type of questions for the subject.

Limitations
The relatively small sample size of faculty members, with one or two (5% -10%) of them disagreeing with most of the survey statements (Table 1), shows that they may not have previous OSCE experience before their employment at the CON.The researcher is of the opinion that these faculty members may be finding the OSCE process challenging and overwhelming for them.Secondly, it appears that these faculty members are more comfortable with the traditional clinical base psycho motor skill check.It is therefore necessary to convince them by providing them with numerous evidence based studies reporting the advantages of OSCE for assessing clinical competence in nursing and other health care professions.

Recommendations
OSCE should be combined with other forms of skill assessment, which may test competences not easily tested within the OSCE format.Nurse-educators and researchers should explore the possibility of producing an OSCE bank containing questions and answers including checklists and global rating scores that will provide nurseeducators with adequate number of questions for both mock and real OSCEs.

Conclusion
The findings of this study were not entirely conclusive in testing the main and sub-hypotheses which included: There is no significant difference in the perception of the 3 groups (faculty members, BSN and ADN graduates) regarding OSCE as the "gold-standard" for clinical competence assessment, especially in terms of transparency, bias free, fairness to all students and the summative mode of conducting it.There is no "gold-standard" for clinical competency assessment.A combination of methods is advocated, i.e.OSCE should complement other traditional psycho-motor skill assessment methods in the clinical settings.One or more sessions of formative OSCE followed by mixed OSCE should be introduced before the students are exposed to the first summative OSCE.Future research should explore testing the validity and reliability of OSCE marking sheets, by using inter-rater reliability to compare the grading of two assessors simultaneously on the same students.

Table 1 .
Participants views of OSCE.