Development of Quantitative Indicator System for Evaluating the Achievement of Surgical Nursing Course Objectives Based on OBE Concepts ()
1. Introduction
The concept of OBE (Outcomes-Based Education), which is “student-centered, output-oriented, and continuous improvement”, has gradually become the theoretical basis for the formulation of talent cultivation goals in higher education (Yang, Zhu, Luo et al., 2023). OBE firstly originated from vocational training in Canada, and was gradually recognized and applied to the field of basic education in the United States and Australia in the 1990s (Mukhopadhyay & Smith, 2010). In the 1990s, OBE was gradually recognized and applied to the field of basic education in the United States and Australia (Xu, Luo, & Lu, 2024). The training of applied nursing talents is still the main task of nursing education in China, and the theoretical guiding significance of the OBE concept on the training objectives of applied nursing talents has been widely recognized, and it has become an important guiding principle for improving teaching quality and optimizing curriculum design (Xu, Luo, & Lu, 2024; Zhang, Yang, Wang et al., 2023). The OBE concept emphasizes that the core of the learning outcomes of the students, by clarifying the expected learning objectives, designing the teaching content, methods and evaluation system, and ensuring that students can complete the education in a timely manner. System to ensure that students can achieve the expected level of competence after completing the educational process (Tan, Chong, Subramaniam et al., 2018). According to the basic requirements of reverse design and forward implementation, graduation requirements, course objectives and classroom teaching objectives need to be logically connected. The evaluation of the achievement of the course objectives is the preliminaries of the evaluation of the achievement of the graduation requirements, which needs to be determined by the lecturers in the light of the talent cultivation objectives of their universities, the academic situation and the basis of curriculum construction (Gong, Zhang, Liu et al., 2024).
Surgical Nursing is a professional core course in Nursing, aiming to improve students’ clinical nursing competence, through which students are required to be familiar with and manage the care of common surgical diseases and multi-morbidities. Our Surgical Nursing, which was awarded the national online-offline hybrid first-class undergraduate course in 2023, is offered in the 5th semester of the four-year undergraduate nursing, with a total of 112 credit hours, including 73 hours of theory and 39 hours of laboratory/attendance, for a total of 7 credits. The course has now been open for 11 semesters on the online platform, and one virtual simulation project has been developed. In the previous teaching process, the teaching team relied on qualitative analysis to evaluate the achievement of course objectives and lacked systematic quantitative tools. This study for the first time incorporates virtual simulation, designed experiments and other multivariate assessment data into the quantitative model, which supplements the insufficiency of the evaluation methods of course goal attainment under the concept of OBE. The purpose of this paper is to sort out how the teaching team constructs this quantitative index system, with a view to providing useful references and lessons for practitioners and researchers in the field of nursing education.
2. Talent Cultivation Objectives, Graduation Requirement Observation Points and Course Objectives
According to the concept of OBE, the top-level design of talent cultivation objectives needs the reverse support of graduation requirements, and 3 - 5 graduation observation points are supported according to the core courses of the specialty (Harden, 2002; Iqbal, Willis et al., 2020). This major specifically formulated 5 talent cultivation objectives, which generally fit the concept of five education in moral, intellectual, physical, aesthetic and labor. Focusing on the 5 talent cultivation objectives, this specialty sets 7 graduation requirements, including having a sense of family and country, sorting out correct professional values and professional norms, describing the knowledge of related disciplines, mastering the basic theories of the specialty and the development of the discipline, possessing the ability of nursing practice, possessing the preliminary teaching and management ability, and possessing the ability of innovation. It has been clarified through discussion by the School of Nursing Instructional Steering Committee that not all courses are required to support all graduation requirements, and that each core course in the program covers at least three graduation requirements. Surgical Nursing supports four graduation requirements, with specific course objectives including: mastery of basic theories, knowledge, and skills in surgical nursing; possession of a sense of family; teamwork, dedication, and love of injury; and clinical critical thinking. The remaining three requirements are considered irrelevant or already covered by other courses.
3. Determination of Course Assessment and Evaluation Items and Their Weights
The course assessment of Surgical Nursing needs to take into account online and offline, process and summative evaluation. Combined with the foundation of the course construction and the four course objectives identified above, the teaching team further clarified the assessment items of the course, aiming at evaluating the degree of achievement of the course objectives from multiple perspectives. The weighting of each assessment item is determined based on the accumulation of experience in the course construction process, and is determined after several rounds of collective discussion by the teaching team. The course assessment results are composed of process assessment results and final examination results, and the assessment items include six items of wisdom tree online learning (platform points), virtual simulation experiments, evaluation of teaching reform courses, design experiments, operational skills and final theory. The proportion of points for each assessment item is filled in according to the course syllabus; the level of difficulty of the assessment items in different categories of the course is filled in according to the level of performance (in percent) of the minimum expectation of students to meet the standard for assessment and evaluation; and the final examination shall not be less than 60 points. Each assessment objective corresponds to the achievement of one or more course objectives. The weights of the course assessment items and their correspondence with the course objectives are shown in Table 1, and the distribution of the scores of the course assessment items in each course objective is shown in Table 2. The scores of the various types of assessment items are all in percentage, and the sum of the scores of the same type of assessment items in different course objectives should be equal to 100. The weighting of each course assessment item is discussed by the course team and submitted to the Faculty Teaching and Learning Steering Committee for validation.
Table 1. Weighting of course assessment items and their correspondence to course objectives.
Assessment Items |
Percentage of points (Ai) |
Minimum Achievement Score (Bi) |
Matching course objectives |
Online Platform study |
15% |
80 |
Objective 1 |
Virtual Simulation Experiment training |
5% |
75 |
Objective 1, 4 |
PBL, Sandwich teaching |
5% |
80 |
Objective 1, 3, 4 |
Design Experiments |
10% |
75 |
Objective 1, 2, 3, 4 |
Skill |
15% |
70 |
Objective 1, 3 |
Final Exam |
50% |
60 |
Objective 1 |
Total |
100% |
|
|
Table 2. Distribution of course assessment items in each course objective.
Assessment Items |
Objective 1 |
Objective 2 |
Objective 3 |
Objective 4 |
Online Platform study |
80 |
10 |
0 |
10 |
Virtual Simulation Experiment training |
50 |
0 |
0 |
50 |
PBL, Sandwich teaching |
40 |
0 |
30 |
30 |
Design Experiments |
40 |
10 |
10 |
40 |
Skill |
70 |
0 |
30 |
0 |
Final Exam |
80 |
10 |
10 |
0 |
4. Matrix of Support of Assessment Results to Course Objectives and Its Achievement Calculation
In order to present the calculation process and results, the results of Surgical Nursing of 117 students in 2 classes of undergraduate nursing in the class of 2022 of our university are taken as the sample for analysis, and the calculation process of the assessment results on the support matrix of the course objectives and their degree of achievement is elaborated in detail. The average scores of the 6 assessments of the 117 students are, in order, as follows: 93 points for online learning on the Wisdom Tree (platform points), 91 points for virtual simulation experiments, 92 points for the evaluation of teaching and reform courses, Designed Experiment 82 points, Operational Skills 83 points and Final Theory 64 points (rounded to the nearest whole digit for ease of calculation). The formula for calculating the achievement of the course sub-targets is shown in Table 3, and the calculation results are shown in Table 4.
Table 3. Definition and description of formula variables.
Variables |
Definition |
Formulas |
Ti |
Course Sub-Objective Scores |
|
Ymi |
Course sub-goal attainment thresholds |
|
Pi |
Course Subgoal Grade Point Average |
|
Dmi |
Course sub-goal attainment |
|
Yz |
Threshold for Achievement of Overall Course Objectives |
|
Dzm |
Course Total Goal Attainment |
|
Note: Ai is the proportion of marks for each assessment item in the process assessment and final assessment (Table 1); Bi is the minimum level of achievement expected from students in different categories of assessment items, i.e., the minimum attainment score (Table 1); N is the total number of samples.
Table 4. Calculation of course sub-goal attainment.
Course objective |
Ti |
Ymi |
Pi |
Dmi |
Objective 1 |
71.00 |
0.67 |
52.87 |
0.74 |
Objective 2 |
7.50 |
0.66 |
5.42 |
0.72 |
Objective 3 |
12.00 |
0.68 |
9.14 |
0.76 |
Objective 4 |
9.50 |
0.77 |
8.33 |
0.88 |
Total |
100 |
|
|
|
Yz |
0.68 |
Dzm |
0.75 |
5. Discussion
5.1. Analysis of the Degree of Achievement of Course Objectives
It showed that the degree of achievement of the course objectives of Surgical Nursing is 0.75, which is higher than the threshold. From the results of course sub-objective attainment, course objective 4 (cultivate critical clinical thinking) has the highest degree of attainment, and the reasons are analyzed as follows: firstly, the appropriateness of teaching methods: critical thinking cultivation is often realized through interactive teaching such as case study, PBL teaching, virtual simulation, etc., and such methods directly train students’ analytical and decision-making abilities, and the evaluation criteria are clear (Meng, Song, Yu et al., 2023; Manas, Ismail, Mohan et al., 2024); secondly, the inclination of course resources: nursing Courses generally emphasize clinical skills training, and related teaching resources (such as virtual simulation experiments, operational assessment) are invested more to form positive feedback; Third, evaluation quantifiability: the assessment of critical thinking can be quantified by structured scoring scales (such as OSCE multi-station assessment), and the results are objective and easy to count (Martinho & Griffin, 2020). Future teaching should continue to consolidate the advantages of critical thinking development.
The lowest degree of achievement of the course objective up to 2 (cultivation of national sentiment) may be due to the analysis of the following reasons: first, the teaching objective is abstract, national sentiment involves abstract values such as professional ethics, social responsibility, and the lack of specific behavioral indicators, which leads to greater difficulty in teaching design and evaluation; second, the teaching method is single, and the current teaching may rely on theoretical lectures or slogan propaganda, the lack of practical immersion, which makes it difficult for students to internalize the values; and third, the assessment form is weak, and the existing assessment is more than one type of assessment. Weak assessment form, the existing assessment mostly adopts subjective scoring (such as “mutual evaluation between teachers and students” in the teaching reform course), which is easily influenced by the subjective preference of the evaluator and has a low weighting (e.g. only 5%). Future teaching improvements are suggested to optimize the cultivation process of the goal of national sentiment, such as integrating practical projects and strengthening the sense of responsibility through real-life scenarios (Sun, Zhang, Yu et al., 2025). In terms of evaluation tools, it is recommended to develop quantitative evaluation tools such as designing behavioral observation scales combined with qualitative evaluation.
5.2. Construction of Appraisal Items and Quantitative Index
System
The evaluation of course objective attainment is the basic component unit of graduation attainment evaluation (Taymoori & Moshki, 2014). Combined with the talent cultivation objectives and graduation requirements of our university, Surgical Nursing has formulated four course objectives compatible with them, and set six assessment items in combination with the foundation and characteristics of course construction. The teaching team has fully embodied the characteristics and requirements of “forward design, reverse implementation” under the concept of OBE through the logical sorting of talent cultivation objectives - graduation requirements - course teaching objectives - course assessment items (Harden, 2002; Barman, Silen, & Bolander, 2014). The construction of quantitative index system is a key step to realize the evaluation of course goal achievement. This course belongs to the online and offline hybrid courses, online teaching platform, virtual simulation project construction started earlier, teaching resources construction is more adequate, the teaching team in the teaching process has also carried out a variety of teaching methods, therefore, the course assessment construction covers the six items that can reflect the course objectives and the corresponding weighting ratio. The basic principle of the proportion of each assessment item is 50% for process assessment and 50% for summative assessment, taking into account the knowledge, emotion and value goal orientation.
5.3. Visualization Output of Quantitative Index Evaluation System
In order to improve the operability of the quantitative index system, the teaching team put all the calculation formulas into Excel software to form an online calculator. In practical application, teachers only need to input the actual score or average score of each assessment objective to automatically generate the course sub-objective score Ti, course sub-objective attainment threshold Ymi, course sub-objective average score Pi, course sub-objective attainment degree Dmi, and the final course total objective attainment threshold Yz and course total objective attainment degree Dzm. The development of the online calculator realizes convenient calculations of randomly sampling students or analyzing the average scores of students only. The development of the online calculator realizes the convenient calculation of randomly selecting students or analyzing only the average grades, which ensures that the whole quantitative index system of course goal achievement evaluation can truly serve the teaching and objectively and conveniently present the teaching effect.
6. Limitation
Although the construction of the quantitative index system for evaluating the achievement of course objectives in Surgical Nursing based on the concept of OBE has achieved certain results, it still faces some challenges. We didn’t collect feedback from students or faculty regarding the implementation or effectiveness of the quantitative index system. Although it was emphasied that subjective assessments must be fair in the “mutual teacher-student evaluation”, but it was difficult to avoid personal relationships affection. Also the system of defining thresholds for the achievement of individual graduation requirements for individual students has not yet been constructed.
7. Conclusion
Based on the concept of Outcome-Based Education (OBE), this study constructed a metricized evaluation system for the achievement of surgical nursing course objectives through reverse design. Taking the course evaluation data of nursing undergraduates in the class of 2022 as a sample, we systematically established the support relationship between the course objectives and the observation points of graduation requirements, and formed a series of core formulas based on the weight allocation of assessment items and the calculation of scoring values. The results of the study show that the system can objectively quantify the degree of achievement of course objectives. This model provides a generalizable scientific paradigm for the evaluation of goal attainment of core courses in nursing.
Funding
Innovation Project of Guangxi Graduate Education (JGY2024327).