A Validation Pilot Study Comparing Telemedicine Images to a Face-to-Face Airway Exam for Conducting the Anesthesia Preoperative Airway Evaluation

Telemedicine is a branch of healthcare that uses communication technology to deliver medical information and services between patients and healthcare providers. The applicability of telemedicine is vast and increasingly relevant. There is a lack of research on utilizing telemedicine for remote evaluation of the airway. The primary aim of this pilot study was to validate a telemedical airway exam as a viable alternative to an in-person evaluation. Three anesthesiologists evaluated 48 volunteers by telemedicine and live examination. The telemedical exam consisted of transmitting still images of four established, predictive parameters of difficult airways: Mallampati score; neck extension; ability to prognath; and thyromental distance. Each subject’s telemedical and face-to-face scores were compared to determine their degree of correlation. Still images were taken using standardized positioning with four pictures of mouth opening, neck extension, prognath, and thyromental distance. Data were analyzed using Wilcoxon signed-rank tests and free-marginal multirater kappa analysis. Average respective scores for live versus telemedicine examination were as follows: Mallampati scores were 1.73 versus 2.54; neck extension


Introduction
Telemedicine is a branch of healthcare that uses communication technology, such as smartphones, video conferencing, and photographs, to provide patients and healthcare providers with medical information and services [1] [2]. Population growth has outpaced the availability of healthcare providers in the United States. Telemedicine has been deemed a useful tool for patient care in remote locations, management of chronic and perioperative conditions, and monitoring of special needs patients and intensive care units [3] [4] [5]. As the COVID-19 pandemic continues globally, the burden on both the inpatient and outpatient healthcare systems has reached unprecedented levels; thus, noninvasive preoperative diagnostics have been successfully adopted to limit this burden and ensure the safety of patients and providers [6].
Preoperative anesthetic assessment is achieved via thorough examination of patients including medical history, physical examination, airway assessment, laboratory workup, and perioperative risk assessment [7]. A review of the medical literature shows evidence that current telemedical technologies can be used to obtain portions of the preoperative assessment [2] [8] [9] [10]. It has been well-described in the past about how telemedicine is a useful tool in obtaining patients' medical history, provide preoperative instructions, and explaining the anesthetic plan. Telemedicine's growing role and potential future in anesthesia as a form of expert consultations in emergent situations has been discussed with the growth of technology through newer smartphones, tablets, and 5G networks to ensure seamlessness in communication between patients and providers [11]. The camera quality in smartphones continues to grow rapidly each year; consequently, capturing images and videos in 4K quality is allowing viewers of these media to immerse themselves into experiences that are becoming increasingly lifelike. Even further, virtual reality technology is rapidly developing and has shown various uses throughout medicine and healthcare; this may be another route for telemedicine in the future. Anesthesia telemedicine studies in the past have shown increased patient satisfaction, decreased patient cost in both time and money, and no significant difference in case cancellation rates with telemedical preoperative visits [9] [12] [13] [14] [15]. Studies have even shown increased patient and provider satisfaction, specifically after inclusion of still images into the electronic medical record to supplement the in-person airway exam, in lieu of the traditional written assessment of patients' airway exams [16]. Despite this burgeoning growth in technology, there is little statistical evidence Currently, patients need to schedule an appointment to meet with an anesthesia provider to have an airway exam performed. This may present hardships for those patients with limited mobility, living in rural areas, undergoing economic hardship, and/or with special needs. As an example of the latter, some individuals may become uncooperative when exposed to the stresses of visiting a large medical center. Patients who cannot receive an airway exam prior to the day of surgery may have increased perioperative morbidity and mortality. Having a validated tool for remote assessment such as a telemedical airway exam will eliminate these barriers to safe and thorough preoperative care. While there have been studies done in the past supporting the use of still images in the preoperative airway examination, conclusions have been limited to suggest incorporating these as a supplement for providers to review. No study has shown a remote assessment alone to be noninferior to an in-person examination.
The goal of this study was to determine whether telemedicine was a reliable and consistent alternative to in-person exams for evaluation of airway parameters by anesthesia providers. Still, images were compared to a live, face-to-face evaluation by three independent, board-certified anesthesiologists. Mallampati score, neck extension, ability to prognath, and thyromental distance, were evaluated for the airway examination. These four parameters were selected because the available literature indicates that, when combined, they are effective in predicting successful intubation [17].

Study Design and Subjects
This prospective blinded study was approved by the Institutional Review Board (Pro2012002417) of the New Jersey Medical School, Rutgers University, Newark, New Jersey, USA, and registered at clincaltrials.gov (code NCT0193750). Informed consent was obtained from all participants in the study. The primary aim was to demonstrate that telemedicine can be used with equal efficacy to obtain airway parameters for preoperative airway evaluation.
Volunteer subjects (n = 48) were evaluated for four airway parameters (Mallampati score, neck extension, ability to prognath, and thyromental distance) by telemedicine (still images) and by live examination by three independent boardcertified anesthesiologists. Each subject's telemedical and live office scores were plotted to determine the degree of correlation between telemedicine and faceto-face examination of the study subject's airway. The still images were taken at a standard distance and height, of mouth opening, neck extension, prognath, and thyromental distance.

Data Capture and Measurements
A room with four divided spaces was used for each of the three independent Open Journal of Anesthesiology evaluators. Each volunteer visited three stations for face-to-face examination.
Mallampati score was evaluated in accordance with the I-V classification system. Neck extension was scored between 0 -4 which corresponded to a range of motion spanning 0 -60 degrees. Ability or inability to prognath was scored as either a 1 or 0, respectively. Thyromental distance was scored by the approximate number of fingerbreadths needed to bridge the distance. At the fourth station, an iPhone 6 (Apple, Inc., Cupertino, California) was mounted on a tripod. The built-in flash was used on all photos and automatic photo exposure was activated. Additional light was provided by an adjustable table lamp with a 60-watt incandescent bulb. A picture was taken for each of the Mallampati view, neck extension, ability to prognath, and thyromental distance. Each viewing station had an adjustable chair to ensure proper height and a giant protractor to measure neck extension. Study subjects were instructed on how to place their fingers under their chin to demonstrate their thyromental distance. The pictures were deidentified. The images were sent to the same three independent anesthesiologists one week later for evaluation of Mallampati score, neck extension, ability to prognath and thyromental distance using the abovementioned scales.

Statistical Analysis
Data from the telemedical and face-to-face images was entered into an Excel (version 16.29; Microsoft Corporation, Redmond, Washington) spreadsheet for analysis. Comparisons were performed using Wilcoxon signed-rank test to analyze the two study groups. Inter-rater agreement among the three independent anesthesiologists was calculated using free-marginal multirater kappa methodology found using an online Kappa calculator: https://www.justusrandolph.net/kappa/. Free-marginal multi-rater kappa is an extension of the bi-rater free-marginal kappa and a variation of Fleiss' fixed-marginal kappa [18]. It can take values from 1 to −1. Values from 0 to 1 indicate agreement better than chance, a value of 0 indicates a level of agreement that could have been expected by chance and values from −1 to 0 indicate levels of agreement that are worse than chance. Differences between the average live airway assessment scores and the average telemedical evaluations were considered to be statistically significant if the p-value was ≤0.05. All the material was stored in a safe computer which could be accessed by the study team members only.

Results
This prospective pilot study enrolled 48 study subjects to undergo an airway exam via telemedical images and face-to-face evaluation. The average (mean) Mallampati   Results regarding the objective measurement parameters used in preoperative airway grading were mixed, in terms of differences between face-to-face and telemedical evaluation. Despite the small sample size (n = 48), there were no statistically significant differences (CI 95% and p-value ≤ 0.05) between the study groups for ability to prognath and thyromental distance (Table 2). However, there was a statistically significant difference found between the Mallampati scores and neck extension, based on the Wilcoxon signed-rank tests ( Table 2).
To assess inter-rater reliability among the three independent anesthesiologists for the four different outcome measures assessed in this study, free-marginal multirater kappa methodology was applied ( Table 3). Degree of agreement based on kappa score was determined to fall into one of six categories: very good  Table 3).

Discussion
The successful and thorough examination of a patient's airway is critical to an anesthesiologist. Airway management is one of the most important aspects of a successful anesthetic, and proper preparation depends on the preoperative examination. The ability of an anesthesia provider to evaluate a patient's airway utilizing telemedical images expands health care delivery to a variety of patients with limited health care access. Obtaining airway exam data prior to the date of surgery increases patient safety, decreases same day surgery cancellation, and allows anesthesia providers to prepare for a difficult airway.
This pilot study was designed to determine if a telemedical airway exam can substitute for a face-to-face evaluation. Our data shows that the exam methods are comparable for ability to prognath and thyromental distance ( Figure 1 and Table 2). However, differences in the assessment of study subject's Mallampati score and neck extension were found to be large when comparing still images to a live exam ( Figure 1 and Table 2).
There are several likely explanations for the differences found in Mallampati scores and neck extension. First and foremost, adequate lighting was a significant issue. All three examiners noted that lighting was problematic when scoring the telemedical Mallampati images. They stated that it was difficult to obtain an accurate assessment because the study subjects' oropharynx was too dark in the photos. This issue will likely be resolved as camera quality rises and technology continues to improve. Other options include adding light clips to cellphones or other devices used to capture the images. For example, there are many commercially available "ring lights" that come with a stand and cellphone holder that can be powered with a remote control and have adjustable lighting settings. Setting these up with a camera in a standardized manner in offices to capture images of the airway may help obtain better views of the oropharynx. A second reason for the difference in Mallampati scores between the two groups is that the  [19]. Our study did not have the patient repeat the Mallampati test, so this may be another reason for the lack of correlation between the face-to-face score and the telemedical score.
Regarding the apparent differences in neck extension grades, extension of the neck is a dynamic motion and obtaining a true understanding of a patient's motor abilities will understandably be more difficult with a still image. Our study did not include a ruler or protractor for reference with the still images, which may have made it more difficult to gauge the angle of neck extension of the subjects for the anesthesiologists evaluating the images. Future studies may also want to consider the use of videos, either live or prerecorded, in examining the airway for a more dynamic remote examination. Prerecorded videos may be advantageous for patients to be able to perform at their own convenience. However, while live videos may require an anesthesiologist to be present the whole time, they also ensure that the video that is captured is adequate for comprehensive preoperative evaluation. This may possibly be a more robust method of mimicking a live assessment, provided there is a convenient, HIPAA-compliant method of documenting and sharing video files.
Of note, the percent agreement and free-marginal kappa statistic for both the Mallampati score and neck extension score were considerably lower than those for the thyromental distance and ability to prognath, both for live exams and telemedicine evaluations ( Table 3). The small sample size of this study (n = 48) and the lower inter-rater agreement rates for Mallampati score and neck extension highlight that it may be difficult to obtain consistent ratings for these parameters in daily practice. These kappa values and agreements rates were not different from each other when comparing live exams with telemedicine, signifying that raters had similar reliability with both approaches in all four measurements ( Figure 2 or camera in an area with adequate lighting in their own homes for a true remote evaluation. Advances in camera technology may make this more feasible soon, but certain patients may be limited in their functional capacity to be able to get high quality images. In communities of a generally lower socioeconomic status, access to the proper instrumentation, environment, and the literacy required to complete the tasks needed to properly record adequate images or videos of the airway may be limited. As this is a pilot study where n = 48, confidence intervals of 95% (p-values ≤ 0.05) applied to the data do not indicate validity of the study. Rather, trends in the data are more representative and provide credibility for the results. Confidence intervals such as 75% or 85% are often employed to provide more accurate estimation of hypothesis testing in pilot studies [20]. This is a limitation of the study, and future studies with larger sample sizes must be done for a stronger diagnostic test validation analysis.
This study's data shows that a telemedical airway exam is a reliable tool that enables anesthesia providers to obtain essential patient information prior to the day of surgery and without having a patient visit the preadmissions testing clin-