Impact of a practical skills assessment on the individual engagement of undergraduate pharmacy students within laboratory sessions.

This study reports on the learner impact, practicability and cost effectiveness of an individual practical skills test designed to assess the interpretative and manipulative skills of undergraduate pharmacy in a laboratory setting. The reliability of the assessment tool across a 5 year period was examined and refine- ments introduced in response to constructive feedback from colleagues and learner feedback recorded via end of year Student Evaluation Questionnaires. A blended learning strategy supported the needs of multi- ple learning styles and inclusion of a formative assessment increased student confidence and improved cohort performance in the summative assessment. Future directions include the introduction of a peer learning activity as a means of reducing group sizes and providing an opportunity for the learners to develop skills in constructive critique and reflective learning.

This study reports on the learner impact, practicability and cost effectiveness of an individual practical skills test designed to assess the interpretative and manipulative skills of undergraduate pharmacy in a laboratory setting. The reliability of the assessment tool across a 5 year period was examined and refinements introduced in response to constructive feedback from colleagues and learner feedback recorded via end of year Student Evaluation Questionnaires. A blended learning strategy supported the needs of multiple learning styles and inclusion of a formative assessment increased student confidence and improved cohort performance in the summative assessment. Future directions include the introduction of a peer learning activity as a means of reducing group sizes and providing an opportunity for the learners to develop skills in constructive critique and reflective learning.
Keywords: Practical Skills Assessment; MPharm Degree; Formative Assessment; Peer Learning

The Scottish MPharm Degree
In Scotland the undergraduate pharmacy degree is a 4 year undergraduate Master's degree (QAA, 2002) accredited by the General Pharmaceutical Council (GPhC). The curriculum developed across the four years addresses core modules in pharmaceutical sciences including medicinal chemistry, pharmacognosy, pharmaceutical analysis, biochemistry, pharmaceutical microbiology, sterile products and medicine design and manufacture together with professional modules such as prescribing sciences, pharmacy practice, social pharmacy, public health and pharmaceutical care.
The GPhC standards for the initial education and training of pharmacists (GPhC, 2011) advises that the "curricula must be progressive, dealing with issues in an increasingly more complex way until the right level of understanding is reached" (Harden & Stamper, 2009), is underpinned by appropriate assessment strategies and provides opportunities reflection and peer learning. The philosophy of the GPhC standards (GPhC, 2011) is driven by a testing of 4 levels of competency defined in Miller's triangle (1990) and which describes four levels of assessment: knows; knows how; shows how (competence); and does (performance) (Figure 1).
A popular degree choice for undergraduate students, each Stage of the MPharm degree has typically between 130 -140 full time students enrolled and such large classes are the main driver for a group approach (typically 4 -5 students) being adopted in practical coursework sessions.

Modes of Assessment in Pharmaceutical Science Modules
It is widely recognized that the mode of assessment may influence the behavior of student learners (Biggs, 2003) but equally issues of class sizes and staff resources are sometimes constraints to the implementation of the ideal scenario. In the early years of the MPharm degree most modules have a minimum of two components of assessment aligned to the lecture material (e.g. written exam) and the laboratory coursework sessions for which there are a variety of assessment strategies in use.
Biochemistry is a Stage 2 (Scottish Credit Qualification Framework level 8) degree module and forms the foundations for Stage 3 modules such as Biotherapeutics and Pharmaceutical Analysis both of which require students to be numerate, competent in the handling of micropipettes and construction of calibration lines and confident in their ability to interpret and evaluate experimental data. Historically the assessment of the Biochemistry coursework sessions has been by an individual written test and an individual laboratory report. The advantages of this approach were that it enabled the assessment of core knowledge and evaluative skills required for later Stages of the degree including e.g. the preparation of a final year thesis. The disadvantages were that it gave no measure of a student's competency within a laboratory setting and there was little incentive for students to contribute individually to the group coursework activity since by sharing the data generated as a group, individuals could produce satisfactory submissions without necessarily having contributed effectively to the laboratory practicals.
Student feedback via the Student Evaluation Questionnaire (SEQ), consideration of staff resources required to mark and feedback on the two written assessments together with personal concerns regarding the inability of some students to competently handle micropipettes and manipulate calibration curves were the drivers for change. This led to the design, implementation and review of an individual practical skills test as a means of assessing the manipulative and interpretative skills of Stage 2 MPharm students undertaking Biochemistry coursework element.

Design and Operation of the Practical Skills Test
The five core skills assessed via the practical skills test were the: 1) accurate and precise use of micropipettes, 2) calculation of dilutions and preparation of calibration standards, 3) accurate representation of data using a graph, 4) use of a UV spectrophotometer, 5) evaluation of the accuracy and precision of experimental data.
The core skills were assessed via 3 independent workstation activities each of 12 minutes duration. Workstation 2 assesses core skill 3 and requires students to construct calibration lines and interpolate unknowns using exemplar data. Workstation 1 assessed core skills 1, 2 and 4 whilst workstation 3 assessed core skills 1, 2, 4 and 5. A round robin rotation was employed with 4 students simultaneously undertaking each of the 3 skills assessments. Thus 12 students complete the summative assessment in each 45 minute cycle and 5 staff was involved in the observation, marking and timing of the assessment. Post assessment the markers complete a prescriptive marking pro forma which details how effectively the student has performed key elements of the task. This approach enables students to receive individual written feedback on their summative assessment and critically has significantly reduced the time (approx. 30% decreases since 2007) assessors commit to this marking this component.
Appreciative of the stress that some learners experience in individual skills based assessments and mindful of the potential benefits that formative assessment and feedback affords (Black & William, 1998;Dunn & Mulvenon, 2009), the students have a formative assessment opportunity one week before the summative event. Since it is recognized that students may readily practice their skills in graphical analysis outwith the laboratory environment the formative exercise focuses on other ex-perimental activities designed to evaluate core skills 1, 2, 4 and 5. Immediately after completion of the formative assessment staff mark the individual student worksheets, provide individual feedback on the pass/fail outcome and provided detailed group feedback. Students are then invited to seek staff guidance if they are still unsure about the handling of micropipettes or operation of UV spectrophotometers.

Evaluation of Effectiveness and Impact upon Learners
Student feedback indicated the formative assessment was perceived as a valuable exercise and the blended approach to supporting different learner styles was appreciated (see Table  1). It is clear however that some of the challenges of group work prevail with some students expressing a desire for smaller groups (see Table 1).
Cohort performance in the formative assessment has been quite consistent with 30% -40% of the population failing to achieve a pass Grade (i.e. Grade D -Grade A, data not shown). However, following group feedback on the formative assessment, further individual instruction on the use of micropipettes; operation of the UV spectrophotometer and the one week for students to reflect on their performance there is a significant improvement in the Grade profile of each cohort (see Figure  2).
During the first two years of implementation >85% candidates achieved a Grade A in the summative assessment and consequently the difficulty level of both workstations 1 and 3 concerned with data generation was increased. These changes led to a decrease in the proportion of the population achieving Grade A (see Figure 3) suggesting the tool was more discriminatory and a plateau in the Grade A profile appears to be emerging.

Discussion
This study summarizes an assessment activity designed to support a progressive approach to the development of laboratory skills within the 4 year MPharm degree. The practical Table 1. Post module comments captured via SEQ (May 2012 cohort).

Student Identifier Comments in Student Evaluation Questionnaire (SEQ)
1 The practical skills test made me work hard to improve my technique.
2 Formative practical assessment was a good taster for what to expect in the summative.

3
The formative coursework assessment was a good indication of what to expect in the summative coursework assessment. Thought this was good as it helped reduce nerves before the assessment.

4
The fact that I knew I had a practical coursework assessment made me work hard and take an interest in coursework sessions prior to the test.

5
The coursework test was well handled; the formative gave a lot of help for the summative.

6
The majority of the experiments were for 4 or 5 people but you could actually do it on your own or in pairs so thought this was a little pointless and also meant if there were people in the group who liked to take the lead, others would simply not get a turn and be left behind.

7
The quizzes provided on Moodle which gave good examples of test and exam style questions were particularly useful both in assessing understanding of particular topics and in preparation for the examination-would have been useful to have these for other modules.
8 Very helpful when staff came round to each student during the lab coursework. I benefited greatly from this as when class is given answers as a whole its hard to keep up and fully understand.   skills assessment was developed to encourage individual student engagement, promote acquisition of core laboratory skills and is a valid instrument (Bloxham & Boyd, 2007) for assessing the module learning outcome: "To generate, manipulate and interpret experimental data relating to biochemistry." A blended learning strategy was implemented to address multiple learning styles and included significant periods of face to face laboratory instruction, online videos to enable students to view instrument demonstrations and online Articulate quizzes to enable students to self evaluate their skills in pharmaceutical science calculations. This approach enhanced the student experience, received positive student feedback (see Table 1) and supported a "learning by seeing" and "learning by doing" experience (Bruner & Olson, 1973) which was accessible to all students via the University's virtual learning environment.
The development of the practical skills assessment and asso-ciated marking pro forma required significant staff resources to ensure: 1) the tasks were appropriate for assessment of the knowledge and skills of stage 2 students, and 2) the assessment pro forma enabled an accurate record of individual student performance to be recorded in a consistent and transparent manner. However the impact of the assessment on student skills was almost immediately apparent with anecdotal feedback from academic colleagues concerned with Stage 3 modules indicating that the laboratory skills and confidence of progressing Stage 2 students had improved within the first year of implementation. Moreover there has been sharing of this good practice with academic colleagues leading to a modified form of the assessment recently being introduced within the Stage 1 Physical Pharmacy module. This vertical integration of the assessment strategy reinforces the message to students that the individual acquisition of lab skills is a pre-requisite for progression and also improves the potential for assessment of competency (Miller's Triangle, 1990) in the core skills associated with pharmaceutical sciences. External examiner feedback of the assessment has been positive and constructive whilst student feedback reflected the findings of Taras (2002) and suggested the formative assessment was both important to supporting individual learning and an effective way for some students to manage the nerves associated with the summative exercise.
It became clear however, in the first two years of application, that the summative assessment was not as discriminatory as we would have hoped for (see Figure 2) and whilst from an academic perspective this was problematic it was also important that the student learners perceived the process as rigorous and continued to be motivated and challenged by the tool (Juwah, 2000(Juwah, , 2003. Consequently in 2010 the assessment was refined to increase the level of difficulty and a reduction in Grade A awards occurred with a broader distribution of pass Grades (A-D) being achieved and no significant alteration in the proportion of Grade E or Fs being observed. Current data suggests the proportion of population achieving Grade As has stabilized (55% -63%) but further monitoring of the effectiveness and reliability of this assessment tool is desirable particularly in light of the recent introduction of a modified form of the assessment in early years of the degree course.
Future activities will be directed towards finding a solution to addressing the concerns some students express regarding individual contribution to and individual learning from a group setting (see Table 1). One possible solution which will be trialed next academic session is to include an element of peer learning within the group work, with student pairs constructively critiquing the organizational skills, team working skills and quality of experimental data generated by their peers (Juwah, 2003).

Conclusion
In conclusion this study evaluated the impact of a practical skills assessment which reflects the ethos of GPhC standards for the training and education of undergraduate pharmacy students and which has been demonstrated to be a practicable and cost effective tool which is valued by academics and the student learners. The continued challenges that group working sometimes presents are recognized but the plan is to use this as an opportunity to include peer learning and thereby further support the development of self oriented and reflective practitioners