Evaluation of the Effect of Chemotherapeutic Drug Training on Mobile Terminal for Neuro-Oncology Nurses Based on Kirkpatrick’s Model

Background: Since there has been training, there has been discussion about the effect of training. But training evaluation is not systematic until Kirkpatrick came up with the training evaluation model in 1959. At present, the prevailing model in the systematic summary of training evaluation is still The Kirkpatrick’s model. This model was further improved in 1994, more responsive to contemporary needs, and thus widely used all over the world. At the beginning, it was widely used in human resource management of enterprises. In recent years, this model has been gradually used in the medical field to evaluate the effect of medical training. The Kirkpatrick’s model has a systematic, integrated and persuasive evaluation system for trainees. It has good effects in the pre-service nurse training, the professional image and code of conduct nurses training, and the geriatric nurse training. At present, there are few studies on the chemotherapeutic drug training of neurologist nurses in China. In clinical work, nurses’ cognitive and practical behaviors of chemotherapeutic drug protection and drug extravasation prevention and treatment are insufficient. It directly harms the health of nursing staff and increases the complications of chemotherapy, increases pain of tumor patients, delays or interrupts chemotherapy, and aggravates the economic burden of patients. Especially, Chemotherapeutic drugs for neuro-oncology have particularity and necessity of urgent training. Objective: To investigate the effect of chemotherapeutic drug training through mobile terminal for neuro-oncology nurses based on the Kirkpatrick’s model. Methods: The training content and evaluation questionnaire for chemotherapeutic drugs were designed by nursing management personnel and senior nurses in our department according to the guidelines and common diseases requiring chemotherapy in the department. The content includes the basic knowledge of neuro-oncology chemotherapy, pharmacological knowledge, toxic and side effect of chemotherapy, etc., which are regularly pushed through the mobile terminal-WeChat. Forty nurses participated in the training and the effect is evaluated by Kirkpatrick’s model. Result: After the training, 100% of nurses were satisfied with the training content and 97.5% with the training form. The scores of nurses in learning level such as basic pharmacological knowledge, drug configuration and exposure, drug treatment and infusion, observation of toxic and side effects, and treatment of drug extravasation were significantly higher than those before the training (P

mon diseases requiring chemotherapy in the department. The content includes the basic knowledge of neuro-oncology chemotherapy, pharmacological knowledge, toxic and side effect of chemotherapy, etc., which are regularly pushed through the mobile terminal-WeChat. Forty nurses participated in the training and the effect is evaluated by Kirkpatrick's model. Result: After the training, 100% of nurses were satisfied with the training content and 97.5% with the training form. The scores of nurses in learning level such as basic pharmacological knowledge, drug configuration and exposure, drug treatment and infusion, observation of toxic and side effects, and treatment of drug extravasation were significantly higher than those before the training (P < 0.01). The scores of nurses in the behavior level such as drug allocation, health education, toxic and side effect observation and prediction, treatment of exosmosis, occupational protection were significantly higher than those before the training. After the training, the satisfaction of managers, chemotherapy physicians and chemotherapy patients on the behavior of nurses was significantly higher than that before the training (P < 0.01). Conclusion: The chemotherapeutic drug training through mobile terminal based on Kirkpatrick's model can improve the ability of neuro-oncology nurses, so as to improve the satisfaction of physicians and patients.

Introduction
Glioma is the most common nerve tumor, and the current treatment is based on a combination of surgery, radiotherapy and chemotherapy [1]. Postoperative care, chemotherapeutic medication and adverse reactions care, and patient mental and spiritual care all require more comprehensive and professional nursing knowledge and skills. In the "Nurses Training Program in Specialized Nursing Field" formulated by the Ministry of Health in 2007, it was proposed that the specialized nursing fields with strong nursing skills need to strengthen standardized training [2]. In the case of increasingly heavy clinical work, it is essential to establish an efficient training mechanism to comprehensively improve the theoretical knowledge and operational skills of disease treatment of oncology nurses. In order to train neuro-oncology specialist nurses more systematically and efficiently, we use the WeChat platform of mobile terminals. Mobile terminals are commonly used portable electronic devices such as mobile phones and notebook computers [3] [4]. In recent years, the form of official accounts and WeChat groups on the WeChat platform on mobile phones has greatly improved the accessibility of information and the efficiency of dissemination, and it is also convenient for clinical staff to use fragmented time for efficient learning in busy work. On this basis, the Kirkpatrick's model is used to evaluate the effect of training. The Korotkoff four-level evaluation model was proposed by the American scholar Donald L. Kirkpatrick in 1959, so it is also called the Kirkpatrick's model [5], including 4 levels of reaction level, learning level, behavior level and result level [6]. In this study, the training team used WeChat public accounts and WeChat groups to train 40 neuro-oncology nurses in our hospital on the basis of aggregating relevant materials and documents for a period of 2 months. The training was very effective, and the report is as follows.

Clinical Data
40 nurses in our department were selected as the research objects that have not participated in any relevant training courses in the six months before the participation. The exclusion criterion is to refuse to continue the training program or fail to submit the relevant questionnaire. The age was 23 to 49 years old, with an average of 35.80 ± 1.50 years old. There are 3 junior colleges (7.5%), 35 undergraduates (87.5%), and 2 postgraduates (5%).

Method
The training team is composed of nursing managers in our department and experienced senior nursing staff in the field of tumor chemotherapy. The members include: 2 head nurses of neuro-tumor, 3 nurses in charge, and 2 professors of neuro-tumor chemotherapy. The team members refer to the latest domestic and foreign guidelines for brain tumors, works in related fields, and standardized operations in basic nursing, combined with common diseases in the undergraduate room, design a training content for chemotherapy drugs. The content includes basic knowledge of neuro-tumor chemotherapy, pharmacological knowledge, chemotherapy toxic and side effects, extravasation treatment and nursing, chemotherapy occupational protection, questionnaire design and notification release, etc. According to the depth and difficulty of evaluation, Kirkpatrick Model divides the evaluation of training effect into four progressive levels, namely, reaction level, learning level, behavior level and result level. The reaction level is defined as nurses' feelings towards the training course and their participation in the training; The learning level is defined as the level of knowledge and skills and the change of cognitive attitude, which mainly evaluates the grasp of theoretical knowledge of nurses; Behavior level reflects the nurse clinical nursing behavior and the standard of compliance; The result level is to discuss the long-term effects of the training. The knowledge of chemotherapy drugs will be trained through WeChat public account every Friday, lasting for 2 months. After 6 months of training in the research group, according to the four levels of the Korotkoff model, the changes before and after the training of nursing staff in our department were compared.

Observation Indicators
Observe the changes before and after training at the four levels of the Kirkpatrick's model through questionnaires, and evaluate the application effect of this training on WeChat terminals. All the evaluation questionnaires in this study were prepared by the team members after consulting relevant literature and combining the actual situation of the department. The validity was verified by 5 nursing professors, and the reliability was verified by Cronbach's alpha (four dimensions 0.78 -0.87). a) Reaction level. The chemotherapeutic drug knowledge training project evaluation questionnaire designed by the members of our department's training team is used to evaluate the nurses' willingness and recognition of the training plan. There are 12 multiple-choice questions, including 2 aspects of training content and training form. Each question contains 4 questions. Options: very good, good, fair and poor. The first two options represent approval or satisfaction. b) Learning level. The manager of the training group designs the content of theoretical test questions and evaluates the performance of the nurses participating in the training before and after the training. The test questions are based on the theoretical knowledge content of the training, a total of 30 questions, and each of the 5 items has 6 questions. Each item has a maximum of 6 points and a minimum of 0 points. c) Behavior level. Assess the behavior and attitude changes of nurses in the undergraduate room before and after training, including five items including chemotherapy drug configuration, chemotherapy drug health education, side effects observation and prediction, extravasation treatment (in vivo and in vitro and environment), occupational protection, etc., 0 -12 points for each item. The higher the score, the better the behavior and attitude of the nursing staff or the stronger the chemotherapy nursing ability. d) The result level. The nursing quality of nursing staff was evaluated from three aspects: satisfaction of management staff to nursing staff, satisfaction of chemotherapy physicians, and satisfaction of patients receiving chemotherapy to nursing staff. Each questionnaire has 0 -15 points. The higher the score is, the better the evaluation is.

Statistical Processing
The SPSS 20.0 statistical software was used, and the measurement data were compared by paired t test and independent sample t test. Inspection level α = 0.05.

Reaction Level
100% of nursing staff were satisfied with the training content, and 97.5% of nursing staff were satisfied with the form of training.

Comparison of the Scores of Nurses' Learning Level Questionnaires before and after Training
There are 5 items in the learning level, a total of 30 questions. Each item has a full score of 6, with a total score of 0 -30. After training, the scores of nurses on basic pharmacological knowledge, drug configuration and contact, medication treatment and infusion, observation of side effects, and drug extravasation treatment were significantly higher than before training. The difference was statistically significant (See Table 1).

Comparison of Nurses' Behavior Level Questionnaire Scores before and after Training
There are 5 items in the behavior level and 15 questions in total, with a full score of 12 for each item and a total score of 0 -60 After training, nurses' behavior level scores in drug configuration, health education, observation and prediction of side effects, extravasation treatment, occupational protection and other items are significantly higher than before training, and there are statistical differences. Learn meaning (See Table 2).

Result Level
After the training, the scores of the management staff, chemotherapy physicians and chemotherapy patients on the nursing staff behavior level questionnaire were significantly higher than before the training, and the difference was statistically significant (See Table 3).

Discussion
In neuro-oncology chemotherapy patients, because the tumor invades the nervous system and the prognosis is often poor; at the same time, facing a series of  possible conditions caused by chemotherapy, nurses need more comprehensive knowledge of chemotherapy. In previous studies, oncology nurses often lacked knowledge of relevant knowledge, and the standardization of daily work needs to be improved [7]. In recent years, the approaches and methods of chemotherapy have been continuously improved. New chemotherapeutic drugs have been continuously launched and the past treatment norms have been continuously rewritten. The methods of use and compatibility of different drugs are changing. Therefore, whether the nursing staff grasps the relevant knowledge in time affects every link in the treatment process of doctors and patients, and even directly affects the results of treatment. In this study, 40 neuro-oncology specialist nurses were trained based on the WeChat platform, and the effect of the training was evaluated by the Korotkoff model. The Kirkpatrick's model was first proposed by Donald L. Kirkpatrick in 1959, and it is widely used in training evaluation in different fields [8]. In recent years, more and more nursing trainings in China have also chosen the Kirkpatrick's model as the evaluation standard, and its effect has been widely recognized [3] [6] [9].
Due to the lag in the improvement of the results of the results level, it also takes time to reflect on the work, so we evaluate and collect the results of the four levels after 6 months of training. The results of the reaction level after the training showed that 100% of the nursing staff was satisfied with the training content, and 97.5% of the nursing staff was satisfied with the form of training.

Limitations
Because the training is limited to only one department, the sample size is small.
This study did not involve multiple research centers, the acceptance of training programs by nurses in different hospitals will be different, and the possible effects will be different. In the following research, the model can be extended to several research centers and the sample size can be expanded, which may achieve better results.

Conclusion
In summary, the training of chemotherapy drugs for mobile terminals based on the Kirkpatrick's model has effectively improved the level of theoretical knowledge of nurses in our department, and can effectively improve the norms of clinical work of nurses, and increase the satisfaction of patients and doctors with nursing and can be promoted and applied.

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.