Does Checklist Implementation Improve Quantity of Data Transfer: An Observation in Postanesthesia Care Unit (PACU)

Background Omission of patient information in perioperative communication is closely linked to adverse events. Use of checklists to standardize the handoff in the post anesthesia care unit (PACU) has been shown to effectively reduce medical errors. Objective Our study investigates the use of a checklist to improve quantity of data transfer during handoffs in the PACU. Design A cross-sectional observational study. Setting PACU at Memorial Sloan Kettering Cancer Center (MSKCC); June 13, 2016 through July 15, 2016. Patients, other participants We observed the handoff reports between the nurses, PACU midlevel providers, anesthesia staff, and surgical staff. Intervention A physical checklist was provided to all anesthesia staff and recommended to adhere to the list at all observed PACU handoffs. Main outcome measure Quantity of reported handoff items during 60 pre- and 60 post-implementation of a checklist. Results Composite value from both surgical and anesthesia reports showed an increase in the mean report of 8.7 items from pre-implementation period to 10.9 post-implementation. Given that surgical staff reported the mean of 5.9 items pre-implementation and 5.5 items post-implementation without intervention, improvements in anesthesia staff report with intervention improved the overall handoff data transfer. Conclusions Using a physical 12-item checklist for PACU handoff increased overall data transfer.


Communication to Improve Patient Safety
Miscommunication is a major patient safety concern. In 2016, The Joint Commission reported communication error as the number one cause of all anesthesia related sentinel events for the period 2004 to 2015 [1]. In the analysis of 444 surgical malpractice claims, 60 cases involved communication breakdown. When multiple healthcare providers across departments and disciplines care for a patient, mistakes in transmitting increasingly complex patient information have been shown to lead to patient harm [2] [3]. Handoff is a transfer of information and professional responsibilities across teams [4]. Despite the wide use of an electronic health record (EHR), a verbal synchronous, face-to-face communication in realtime is a fundamental paradigm of clinical discussion that provides a critical structure and an opportunity for an interactive discussion about a patient [5] [6] [7]. For that reason, The American Society of Anesthesiologist's standard of care requires the presence of intraoperative anesthesia staff for monitoring during transport and verbal report [8].
From review of surgical malpractice claims, the highest percentage of perioperative mistakes, including 43% of all communication failures, occurs post-operatively as a result of poor handoffs [9]. Studies have shown that poor handoffs increase the risk of patient harm [2] [3] [9] [10] [11] [12] and that standardizing communication protocols for handoffs can decrease medical errors and adverse events [13] [14] [15] [16]. Lack of standard guidelines produces inconsistent reports that are vulnerable to omission of pertinent information. Our objective was to quantify the amount of information transferred. The omission of data has been associated with worse outcomes than passing on poor quality of information [17]. A complete omission of information occurred in 57% of surgical malpractice claims [9]. Observation of handoffs showed items deemed vital components of handoff were reported less than half of the times [18].
Inadequate communication in PACU has been shown to increase morbidity and mortality [19]; a review of 419 reports from Anesthetic Incident Monitoring Study (AIMS) indicated a failure of communication as the second most common contributing factor that led to adverse events in recovery units [10]. PACU is especially vulnerable to communication failures between providers because of physical transfer of patient, collaboration of multiple clinicians, and similar patient histories are common features of PACU [17] [20].

Why a Checklist?
To avoid adverse events caused by miscommunication, The Joint Commission mandated "a standardized approach to handoffs" as a patient safety goal in 2006 [21]. Checklists have been used to standardize consistency in communication among providers and to reduce morbidity and mortality in surgical settings [22]. Checklists in PACU to standardize handoff communications have also been studied to show improvement in data transfer [23] [24] [25] [26] and reduction in medical errors [13] [15] [27]. Multicenter handoff interventions using a handoff checklist in PACU showed decrease in preventable adverse events across disciplines and departments [15] [27]. In addition, use of checklists has shown improvements in nursing staff satisfaction and handoff efficiency [25].
Overall, a checklist accomplishes two goals for both intraoperative and postoperative care providers. First, it provides a guideline that defines a standard for a handoff. Second, a physical checklist is used as a reminder of items to prevent omission of information [18] [26].

Goal & Hypothesis
The goal is to establish measures to decrease perioperative miscommunication and improve patient safety through standardized PACU handoff protocol. We hypothesize that a physical checklist will increase data transfer and efficiency at our PACU, and prevent omission of pertinent patient information in handoff.

Derivation of the Checklist
According to a systemic review of 31 studies on PACU handoff, a handoff should include at minimum: patient information, anesthesia information, surgical information, current status, and care plan [17]. A published "Anesthesia Handover Checklist" by Lin and colleagues was used as the initial structural framework. The initial checklist included: Patient, Underlying diagnosis/procedure, Technique-anesthetic, Status of procedure, Past significant medical history, Allergies, Timing/expected duration, Immediate expected events next 30 min, Emergence plan, Noteworthy aspects of case, Treatment plan for post op care, Fluids/EBL, Induction events, Records available for review, Signs-vitals, and Transfer care to. These items listed out a mnemonic: PUTS PATIENT FIRST [28]. This checklist included key elements of the transfer of care measures recommended by The American Society of Anesthesiologists [29]. Following the recommendations from various studies that emphasized flexibility in making adjustments according to the implemented institutions [15] [24] [30], the pilot week was used to adjust the working checklist according to the practices at our institution (Figure 1(a)). Multiple iterations were made to test usability and strength of the checklist.
Every item was deemed equally important, and given a score of 1. Although surgical and anesthesia staff received separate grades, the primary endpoint was the total number of checklist items addressed by either department during the PACU handoff. This value ranged from the minimum of 0 to the maximum of 12. For the composite score, if an item is addressed by either a surgical or anesthesia staff, the item is considered to be addressed, and a score of 1 is allocated to the item. For department based scores, the surgical and anesthesia staff reports received separate score of 1 per item accordingly. The start and end time of the handoff was recorded for assessing the duration of handoff rounded to a whole minute. Lastly, every handoff was assigned an unidentifiable number to match the data between two observers.
Total duration of the study included 5 weeks of observation in the main PACU. The first week preceded the study to make adjustments to the checklist as a pilot period. All handovers were observed in real time by two observers. This pilot study yielded 100% consensus on "item qualified" between attending anesthesiologists and the observers. The observers were physically present at all observed handoffs between June 13, 2016 and July 15, 2016 from 10AM to 5PM. Immediately after each handoff, two observers resolved any differences in assessment and arrived at 100% consensus.

Pre-Implementation Period
Second and third weeks served as the control period. Observers gathered data before the checklist was implemented to gauge the current quality of PACU handoff. Both anesthesia and surgical staff being observed were not informed of the reason for the presence of observers in PACU to avoid isolated improvements in handoff behavior.

Post-Implementation Period
After pre-implementation period and prior to the official implementation of the checklist, all anesthesia staff-including attending anesthesiologist, Certified Registered Nurse Anesthetists (CRNAs), resident physicians, and CRNA students-were made aware of the study. They were provided an electronic and physical copy of the checklist. An A4-sized laminated checklist was available by every patient bed. Additionally, ID badge-sized checklists were distributed to every provider ( Figure 1(b)). During the last two weeks of the study, anesthesia staff were asked to use the checklist by the observers before each report. Surgical staff were not part of the intervention although data were collected from their reports.

Statistical Analysis
A traditional randomized study would randomize each handoff 1:1 to the group with or without a checklist. However, this approach would require some staff to unlearn the checklist hints after being exposed to it. Because of these logistical and feasibility issues, our study is based on convenience sampling in that the first two weeks of the study was preimplementation of the checklist, while the second two weeks was post-implementation. This approach assumes that the case mix (patient and surgical characteristics) is similar between the two phases of the study, which is reasonable in this high volume cancer center. Power calculation was performed prior to the study to determine the minimal detectable difference (MDD) necessary to achieve 80% and 90% power for a two-sided t-test, given 50 patients in each arm and type I error rate of 0.05. The MDD refers to the smallest treatment effect that can be identified assuming a known sample size. We assume mean of 5 items completed in the pre-implementation phase, with a conservative standard deviation (SD) of 5. The MDD are 2.85 and 3.25 for 80% and 90% power. This translates to an assumed mean of 7.85 and 8.25 checklist items completed in the post-implementation phase. If instead SD is 2, the MDD changes to 1.14 and 1.30 for 80% and 90% power.
We examined each item individually to identify the proportion of handoffs addressed for the specific item between the two groups using Fisher's exact test. The secondary outcome of duration of PACU handoff is compared between the two groups using Wilcoxon rank sum test. As exploratory analyses, we compare the total number of items addressed in the postimplementation phase by consistency status to assess whether consistency impacts the quality of handoff. All analyses were repeated with the component scores which included the items addressed by the anesthesia provider only. All statistical tests were two-sided at alpha level of 0.05, performed using Stata 13 (Stata Corp, College Station, TX).

Data Transfer
We observed a total of 120 PACU handoffs. 60 handoffs were each observed preimplementation and post-implementation of the checklist. Composite values analyzed items as addressed by either surgical or anesthesia staff. Department based values analyzed items addressed by surgical and anesthesia staff separately. Pre-implementation of a checklist, the composite value showed a mean of 8.7 (SD = 1.5) items reported out of a total of 12 items on the checklist, and post-implementation the median report increased to 10.9 items (Table 1) (Figure 2). When anesthesia staff reports were analyzed independently, the mean reported items increased from 4.8 (SD = 1.6) to 8.9 (SD = 2.0) items post-implementation of the checklist (Table 1). Interestingly, surgical staff report stayed relatively consistent at mean of 5.9 items pre and 5.5 items post-implementation periods. In the analysis of anesthesia staff reports, items that were consistently reported at low numbers despite the checklist were: PACU Plans, Disposition-Expected Duration in PACU, Underlying Diagnosis, and Procedure Done (Table 2).

Secondary Analysis
Duration of each handoff is the total time of both surgical and anesthesia staff reports rounded to a minute. Post-implementation, the median duration of handoff, is increased by one minute (Table 3). However, in comparison to the increase in the number of reported items with longer handoffs during the pre-implementation period, duration of the handoffs was independent of the number of reported items post-implementation (Table 4).

Discussion
Our results demonstrate that, in the setting studied, the use of a checklist improved the overall quantity of data transfer during PACU handoff. A checklist was introduced and implemented for two weeks. During the post-implementation period, more items were reported in all intervals of "handoff duration" in comparison to pre-implementation period. Using a checklist to prevent omission of patient information during handoff is important because miscommunication from multiple care transfer has been shown to increase patient harm [2] [3] [9] [10] [11] [12]. To mitigate these adverse events, a use of checklist in PACU has shown not only an increase in data transfer [23] [24] [25] [26] but also a decrease in medical errors [13] [14] [15]. Our study adds that a checklist reminds the staff of defined standard of items to report in order to minimize information omission during PACU handoff. Furthermore, the quantity of minimum data transferred during handoff is independent of the quality of the reports, which is more closely associated with complexity of patient history and not an objective of this study.

Quantity of Data Transfer
There are two important reasons for analyzing the data as composite values and department based values. First, PACU handoff is provided by both the surgical and anesthesia team at our institution. The median number of items reported by surgical staff stayed the same at a median of 6 items out of the total 12 during post-implementation period. This data indicates that the increase in overall improvement in handoff during post-implementation can be attributed to improvements in anesthesia reports without unintended observational influence on the quality of surgical staff reports. Quality of anesthesia reports as a standalone report is also important because collaborative report may not be a standard of practice at all institutions. In many institutions, only anesthesia gives report during PACU handoff. As an ideal standard, anesthesia staff should be able to adequately report surgical information in case anesthesia staff is the only informant of intraoperative events.
Second, comparing composite and department based values helped us to identify items pertinent to either surgical or anesthesia staff during handoffs. Items related to surgical procedure improved the least in the anesthesia staff reports (i.e. PACU plans, Disposition-Expected Duration in PACU, Underlying diagnosis, and Procedure done) (Figure 3). Because compliance of reporting surgical information by anesthesia staff was lowest in a similar study, the data was used to recommend presence of surgical staff during handoffs [31]. Items relevant to the practice of anesthesia improved the most in anesthesia reports and composite value (i.e. Allergies, Anesthesia Technique, and Airway) (Figure 4). Considering that prior to implementation of a checklist, about half of the reports did not include these anesthesia specific information, a checklist reduced omission of at least the most relevant information.

Duration
Contradictory to our hypothesis that a checklist would reduce the duration of a handoff, we observed an overall increase in median time spent during a handoff. Previous studies have shown conflicting reports on the effect of handoff duration after implementing a checklist [23] [25] [26] [31].
The lack of training in using the checklist led to providers stumbling or pausing during the report. Majority of the informal feedback from the anesthesia staff were disturbances to their original "flow" with a different order of items and some unfamiliar items on the checklist. Despite the foreseen improvements in handoff with a physical checklist [26], multimodal staff training models could improve incorporating a new checklist to practice [24] [25] [27] [32]. Second, we recommend multiple Plan-Do-Study-Act cycles24 and incorporating staff feedback18 to design a checklist that better fits the context of each institution.

Limitations
Hawthorne effect is the influence of the presence of an observer on observed behavior. During pre-implementation phase, anesthesia staff was given limited details on what we were observing. We cannot overlook the influence of the presence of the two observers during all observed handoffs. It is likely that data transfer improved for our control group simply because of Hawthorne effect. For instance, we can assume that the observed handoffs were more comprehensive than the baseline handoffs before our study. It follows then, that the actual improvement in handoff is indeed better than we reported. We expect to find more pronounced improvement if our results were subject to the Hawthorne effect.
Although prior studies have correlated adverse events with poor handoff [2] [3], direct comparison to patient outcome with a handoff checklist in a randomized trial is ideal but difficult to accomplish.33 Thus, deriving from previous studies, we assume that our improvement in data transfer will decrease adverse events and improve patient outcome. Secondly, although complexity of the patient history may take longer to report, patient population was not defined or restricted [33]. Patients may vary greatly in their phase of cancer treatment at our institution. Our data is based on the local context and exact results cannot be generalized for other institutions without further multicenter investigation [34]. Thirdly, creating definitive categories of medications or tools with the help of practicing anesthesiologists, observers were taught to convert qualitative data from observations to quantitative data during the pilot week. Despite the 100% consensus made between two observers after every handoff, we cannot neglect the human variables in information gathering.

Conclusion
In conclusion, implementation of a physical checklist for PACU handoff increased overall data transfer and prevented omission of patient information. Report duration did not have an impact on overall data transfer. For future directions, we recommend incorporating staff feedback into Plan-Do-Study-Act cycles for an improved checklist to ensure compliance and familiarize the staff with the use of a checklist through multimodal training modules.

Acknowledgments
1) The authors would like to thank the anesthesia staff at Memorial Sloan Kettering Cancer Center for complying with the intervention during the study period.
2) Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R25CA020449. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
3) This research was funded in part through the NIH/NCI Cancer Center Support Grant P30 CA00874.

PACU
Post Anesthesia Care Unit       Table 3 Summary of Handoff Duration (minute) a .

N Mean SD Median Min Max
Pre 60 2.  Table 4 Breakdown of handoff duration a,b