Evaluation of Videoconferencing Systems for Remote Medical Education ()
1. Introduction
Telemedicine can be divided into two categories, one between doctors and patients for remote consultation, and the other between doctors for remote education. Over 10 years of experience with the use of telemedicine for remote education has shown us that remote medical education differs from other types of remote education (Shimizu et al., 2014). For example, remote medical education requires the frequent use of moving images to show new techniques and procedures. In addition, remote medical education requires higher-quality images than other types of remote education because image quality for the former should be able to show clear images for accurate diagnosis and precise anatomy.
Transmission of moving images is technologically more challenging than transmission of still pictures because the former requires the continuous control of 30 frames per second, rather than single images. Lack of continuous control can cause the images to become blurred, making them inadequate for medical education. Additionally, compression algorithms automatically degrade image quality. Another major obstacle has been the initial cost of special videoconferencing systems that yield unsatisfactory image quality, or the much higher costs charged to use satellite imaging for better image quality (Rabenstein et al., 2002; Rafiq et al., 2004).
The digital video transport system (DVTS), developed by the WIDE (Widely Integrated Distributed Environment) Project in Japan in 1998, satisfied requirements for adequate image quality and a lower cost. The DVTS is a free software program installed on a personal computer (PC), which was found to preserve the original quality of moving images while avoiding complicated compression processes (Ogawa et al., 1999). Because it consumes a bandwidth as high as 30 Mbps under perfectly stable conditions, however, it could not be used on the commercial internet. Research and Education Network (REN), initially called “Computer networking for scientists” was coupled with DVTS to enable the low cost transmission of high-quality moving images for medical purposes (Jennings et al., 1986; Shimizu et al., 2006).
Use of this system soon expanded throughout Asia and beyond, with remote medical education programs between Japan and Korea introduced in 2003 (Carati et al., 2006; Huang et al., 2008; Hahm et al., 2009; Shimizu et al., 2009). As of February 2014, we had connected with 300 hospitals and other facilities in 45 countries and regions, with the broadcast of 425 programs. Contents included a variety of fields, including surgery, endoscopy, healthcare, urology, and nursing, as well as classes for medical students (Eto et al., 2007; Kaltenbach et al., 2009; Lee et al., 2012; Huang et al., 2014).
More recent technological developments include the emergence of high-definition (HD) images in the medical community. When we started this project, all images were standard definition (SD), which fit the DVTS very well. At present, however, many medical devices are HD compatible. Another recent development is the dramatic expansion of needs for telemedicine, with hospitals not connected to the REN wanting to participate. The popularity of mobile devices is another major trend in the medical community. These new needs and demands have resulted in the refinement of compression systems to a level satisfactory for medical education and have led to the development of two additional videoconferencing systems, HD-H.323 and Vidyo®. The selection criteria for these systems include: 1) global availability and support, 2) transmission of moving medical images of satisfactory quality, and 3) affordable cost. The H.323 has been recommended protocol by the International Telecommunication Union, which defines protocols for providing audio-visual communication sessions on any packet network (Reid et al., 1999). H.323 compatibles are produced by several companies, such as Polycom (San Jose, CA, USA), Lifesize (Austin, TX, USA), Cisco (San Jose, CA, USA), and Sony (Tokyo, Japan). The Vidyo system (Hackensack, NJ, USA) is based on the H.264 standard with VidyoDesktop software providing the ability to connect from a PC and enabling users to work on hardware and devices already owned (Kwon et al., 2006). Moreover, the VidyoDesktop is compatible with other audio-visual devices and is built to run on Windows, MacOS or Linux.
Although the three systems have various theoretical advantages and disadvantages, they have never been compared by users (Cao et al., 2012). We here report preliminary feedback on user evaluation of image quality, sound, preparations, and cost. These findings may lead to further improvements in technological support for international collaborations on remote medical education.
2. Methods
The questionnaire shown in Table 1 was administered to participants during the panel discussion, “Technical reviews for the perfect quality of service” at the Seventh Asia Telemedicine Symposium, held on December 14
2013, in Bangkok, Thailand. The participants answered questions using answer pads (Instant Respond System, Model HiTeach, Taipei, Taiwan), and the results were recorded for subsequent analyses.
3. Results
The study assessed 32 participants: nine (28%) medical doctors, 13 (41%) engineers, and 10 (31%) “others”. Of 30 (94%) valid answers, 27 (90%) reported that their institutions were connected to a REN, whereas the other three (10%) said that their institutions were not. Of the respondents, 21/30 (70%) had experience using the DVTS, 22/24 (92%) had used the HD-H.323, and 21/29 (72%) had used the Vidyo. The HD-H.323 system was installed at 16/20 (80%) of the participating institutions.
Of 22 (69%) valid answers on best image quality, nine subjects (41%) picked the DVTS, eight (36%) picked the HD-H.323, and five (23%) picked the Vidyo system (Figure 1). Of 27 (84%) valid answers on the ease of preparation, 21 subjects (78%) picked Vidyo, six (22%) picked HD-H.323 and none picked DVTS (Figure 2).
Of the 29 participants (91%) who evaluated the biggest problem with the DVTS, 11 (38%) reported sound as the most problematic, followed by preparation (28%, 8/29) and cost (24%, 7/29), and finally image quality (1/29, 3%). Valid responses on the HD-H323 were received from 28 (88％) participants, with 18 (64%) stating that the biggest problem was cost (64%, 18/28), followed by sound (18%, 5/28) and preparation (7%, 2/28). None selected image but 47% (9/19) reported that image quality differed between one-to-one and multiple connections. Valid responses on the Vidyo system were received from 29 (91％) participants, with the largest problems being sound and cost, each reported by eight subjects (28%, 8/29), followed by image (17%, 5/29), with preparation being reported the least (7%, 2/29) (Figure 3).
When compared by category, problematic image quality was reported by five of 29 subjects (17%) for Vidyo, one of 29 (3%) for DVTS, and zero of 28 (0%) for HD-H.323. Sound as the biggest problem with these systems
Figure 3. Biggest problems cited by participants for the (a) DVTS, (b) HD-H.323, and (c) Vidyo systems.
was selected by 8 of 29 (28%), 11 of 29 (38%), and five of 28 (18%) participants, respectively, and preparation as the biggest problem was greater for DVTS (28%, 8/29) than for the other two systems (7% each). Cost as the biggest problem for the DVTS, HD-H.323, and Vidyo systems was selected by 24% (7/29), 64% (18/28), and 28% (8/29) of participants, respectively. In total, for these three systems, 20% designated image as the biggest problem, 84% designated sound, 42% said preparation, and 116% said cost (Figure 4).
Overall satisfaction rates for these three systems were 70% (14/20) for DVTS, 86% (18/21) for HD-H.323, and 77% (20/26) for Vidyo. Of 18 participants who selected the best system, nine (50%) selected Vidyo, eight (44%) selected HD-H.323, and one (6%) picked DVTS.
Figure 4. Comparison of problems by category; Image, sound, preparation, cost.
4. Discussion
One of the major aims of this study was to determine whether DVTS images, which are SD, were inferior to images on the other two systems, which are HD. However, the participants in this study selected the DVTS system as providing the highest quality images, followed by the HD-H.323 system. This lack of inferiority for the DVTS was likely associated with the continued preference for DVTS use. The DVTS uses minimal compression on SD, whereas the HD quality of the other two systems is considerably compressed, resulting in the inevitable degradation of image quality. The low ranking of the Vidyo system for image quality may be due to the effects on image quality of the performance of client computers and the need for costly additional equipment for sending images. Compared with their other aspects, however, image quality was rarely regarded as the biggest problem for all three systems.
Respondents reported that the HD-H.323 system was superior to the other two systems in sound quality. Fewer participants reported that sound was the biggest problem with the HD-H.323 than for the other two systems. Echo-cancellers are usually installed in HD-H.323 systems, effectively preventing echo in most situations. Although an echo-cancelling system has also been installed in Vidyo systems, these echo cancellers are software-based and do not seem to work well. In contrast, DVTS contains no echo-canceller, with a sound mixer or a microphone with an on-off switch used to prevent echo (Shimizu et al., 2011).
Ease of setup was greater for the Vidyo than for the other two systems. This may be due in part to the ability of users of this system to use their own PCs. In contrast, the HD-H.323 system requires specialized equipment. Although most participants in this survey were at institutions equipped with the HD-H.323, many other hospitals do not yet have HD-H.323 compatibility. The low-bandwidth requirement of the Vidyo system should also be an advantage over the DVTS, which requires a huge bandwidth and thereby limits the availability of the system. The lack of need for a global IP is another advantage of Vidyo, making setup easier than for the other two systems.
Cost was by far the biggest problem with the HD-H.323 system. Moreover, a considerable number of responders picked cost as a major issue for the Vidyo and DVTS systems.
All these results seem to characterize the three systems very well. The disadvantages of the DVTS include sound and time-consuming preparation, with imaging and cost being smaller problems. Responders seem to be satisfied with all the elements of the HD.H.323, except for cost. Thus, institutions that can afford or are already equipped with this system should be satisfied with its imaging, sound, and ease of setup. The two major problems with the Vidyo system were sound and cost, whereas it was much easier to set up and was selected by the participants in this study as their favorite system. Overall satisfaction rates were relatively high for all three systems, but each still has drawbacks.
This study had several limitations. First, there were no statistical comparisons because of the small number of participants. Second, the survey results were obtained from answer pads, making it difficult to determine the reasons for these results.
In conclusion, the three systems chosen for remote medical education, requiring the transmission of high-quality moving images, were well accepted and useful. Further studies are needed to determine the characteristics of each system properly and to select the optimal system for each program, depending on the contents or system availability.
Acknowledgments
The authors thank the entire medical and engineering staff for their cooperation and expertise in supporting program organization and network preparation. This project was funded in part by Kyushu University Interdisciplinary Programs in Education and Projects in Research Development No. 25727, Grant of The Clinical Research Promotion Foundation, and Grant-in-Aid No. 23256005 for Scientific Research from the Japan Society for the Promotion of Science.