Evaluation of Videoconferencing Systems for Remote Medical Education

Remote education in medicine has two distinct differences over remote education in other fields. The first is the frequent use of moving images to show techniques and procedures, and the second is the necessity of very high image quality for accurate diagnosis or precise anatomy. The emergence of digital video transport systems (DVTS) has surmounted technological limitations and made remote medical education practical. However, to meet new demands for better quality and wider availability, two additional systems, HD-H.323 and Vidyo, have been developed. Feedbacks from users of these systems are crucial for accurate evaluation and further improvements. We designed questionnaires evaluating image quality, preparations, major problems, and overall satisfaction with each of these systems. These questionnaires, along with answer pads, were administered to 32 participants at the Seventh Asia Telemedicine Symposium, held in Bangkok, Thailand, on December 14, 2013. Of 22 (69%) valid answers on best image quality, nine (41%) chose DVTS, eight (36%) chose HD-H.323, and five (23%) chose Vidyo. Of 27 (84%) valid responses on ease of preparation, 21 (78%) picked Vidyo, six (22%) picked HD-H.323 and none picked DVTS. The biggest problems with DVTS were sound (11/29, 38%) and ease of preparation (8/29, 28%), the biggest problem with HD-H.323 was cost (18/28, 64%), and the biggest problems with Vidyo were sound (8/29, 28%) and cost (8/29, 28%). Overall satisfaction rates were 70% (14/20) for DVTS, 86% (18/21) for HD-H.323, and 77% (20/26) for Vidyo. In conclusion, the three technologies currently used for remote medical education were all satisfactory. Because each has its own advantages and disadvantages, the choice of the most appropriate system should depend on the program being broadcast and the availability of equipment and network at participating stations.


Introduction
Telemedicine can be divided into two categories, one between doctors and patients for remote consultation, and the other between doctors for remote education.Over 10 years of experience with the use of telemedicine for remote education has shown us that remote medical education differs from other types of remote education (Shimizu et al., 2014).For example, remote medical education requires the frequent use of moving images to show new techniques and procedures.In addition, remote medical education requires higher-quality images than other types of remote education because image quality for the former should be able to show clear images for accurate diagnosis and precise anatomy.
Transmission of moving images is technologically more challenging than transmission of still pictures because the former requires the continuous control of 30 frames per second, rather than single images.Lack of continuous control can cause the images to become blurred, making them inadequate for medical education.Additionally, compression algorithms automatically degrade image quality.Another major obstacle has been the initial cost of special videoconferencing systems that yield unsatisfactory image quality, or the much higher costs charged to use satellite imaging for better image quality (Rabenstein et al., 2002;Rafiq et al., 2004).
The digital video transport system (DVTS), developed by the WIDE (Widely Integrated Distributed Environment) Project in Japan in 1998, satisfied requirements for adequate image quality and a lower cost.The DVTS is a free software program installed on a personal computer (PC), which was found to preserve the original quality of moving images while avoiding complicated compression processes (Ogawa et al., 1999).Because it consumes a bandwidth as high as 30 Mbps under perfectly stable conditions, however, it could not be used on the commercial internet.Research and Education Network (REN), initially called "Computer networking for scientists" was coupled with DVTS to enable the low cost transmission of high-quality moving images for medical purposes (Jennings et al., 1986;Shimizu et al., 2006).
Use of this system soon expanded throughout Asia and beyond, with remote medical education programs between Japan and Korea introduced in 2003 (Carati et al., 2006;Huang et al., 2008;Hahm et al., 2009;Shimizu et al., 2009).As of February 2014, we had connected with 300 hospitals and other facilities in 45 countries and regions, with the broadcast of 425 programs.Contents included a variety of fields, including surgery, endoscopy, healthcare, urology, and nursing, as well as classes for medical students (Eto et al., 2007;Kaltenbach et al., 2009;Lee et al., 2012;Huang et al., 2014).
More recent technological developments include the emergence of high-definition (HD) images in the medical community.When we started this project, all images were standard definition (SD), which fit the DVTS very well.At present, however, many medical devices are HD compatible.Another recent development is the dramatic expansion of needs for telemedicine, with hospitals not connected to the REN wanting to participate.The popularity of mobile devices is another major trend in the medical community.These new needs and demands have resulted in the refinement of compression systems to a level satisfactory for medical education and have led to the development of two additional videoconferencing systems, HD-H.323 and Vidyo®.The selection criteria for these systems include: 1) global availability and support, 2) transmission of moving medical images of satisfactory quality, and 3) affordable cost.The H.323 has been recommended protocol by the International Telecommunication Union, which defines protocols for providing audio-visual communication sessions on any packet network (Reid et al., 1999).H.323 compatibles are produced by several companies, such as Polycom (San Jose, CA, USA), Lifesize (Austin, TX, USA), Cisco (San Jose, CA, USA), and Sony (Tokyo, Japan).The Vidyo system (Hackensack, NJ, USA) is based on the H.264 standard with VidyoDesktop software providing the ability to connect from a PC and enabling users to work on hardware and devices already owned (Kwon et al., 2006).Moreover, the VidyoDesktop is compatible with other audio-visual devices and is built to run on Windows, MacOS or Linux.
Although the three systems have various theoretical advantages and disadvantages, they have never been compared by users (Cao et al., 2012).We here report preliminary feedback on user evaluation of image quality, sound, preparations, and cost.These findings may lead to further improvements in technological support for international collaborations on remote medical education.

Methods
The questionnaire shown in Table 1 was administered to participants during the panel discussion, "Technical reviews for the perfect quality of service" at the Seventh Asia Telemedicine Symposium, held on December 14, 2013, in Bangkok, Thailand.The participants answered questions using answer pads (Instant Respond System, Model HiTeach, Taipei, Taiwan), and the results were recorded for subsequent analyses.

Discussion
One of the major aims of this study was to determine whether DVTS images, which are SD, were inferior to images on the other two systems, which are HD.However, the participants in this study selected the DVTS system as providing the highest quality images, followed by the HD-H.323system.This lack of inferiority for the DVTS was likely associated with the continued preference for DVTS use.The DVTS uses minimal compression on SD, whereas the HD quality of the other two systems is considerably compressed, resulting in the inevitable degradation of image quality.The low ranking of the Vidyo system for image quality may be due to the effects on image quality of the performance of client computers and the need for costly additional equipment for sending images.Compared with their other aspects, however, image quality was rarely regarded as the biggest problem for all three systems.
Respondents reported that the HD-H.323system was superior to the other two systems in sound quality.Fewer participants reported that sound was the biggest problem with the HD-H.323than for the other two systems.Echo-cancellers are usually installed in HD-H.323 systems, effectively preventing echo in most situations.Although an echo-cancelling system has also been installed in Vidyo systems, these echo cancellers are software-based and do not seem to work well.In contrast, DVTS contains no echo-canceller, with a sound mixer or a microphone with an on-off switch used to prevent echo (Shimizu et al., 2011).
Ease of setup was greater for the Vidyo than for the other two systems.This may be due in part to the ability of users of this system to use their own PCs.In contrast, the HD-H.323system requires specialized equipment.Although most participants in this survey were at institutions equipped with the HD-H.323,many other hospitals do not yet have HD-H.323compatibility.The low-bandwidth requirement of the Vidyo system should also be an advantage over the DVTS, which requires a huge bandwidth and thereby limits the availability of the system.The lack of need for a global IP is another advantage of Vidyo, making setup easier than for the other two systems.
Cost was by far the biggest problem with the HD-H.323system.Moreover, a considerable number of responders picked cost as a major issue for the Vidyo and DVTS systems.
All these results seem to characterize the three systems very well.The disadvantages of the DVTS include sound and time-consuming preparation, with imaging and cost being smaller problems.Responders seem to be satisfied with all the elements of the HD.H.323, except for cost.Thus, institutions that can afford or are already equipped with this system should be satisfied with its imaging, sound, and ease of setup.The two major problems with the Vidyo system were sound and cost, whereas it was much easier to set up and was selected by the participants in this study as their favorite system.Overall satisfaction rates were relatively high for all three systems, but each still has drawbacks.
This study had several limitations.First, there were no statistical comparisons because of the small number of participants.Second, the survey results were obtained from answer pads, making it difficult to determine the reasons for these results.
In conclusion, the three systems chosen for remote medical education, requiring the transmission of high-quality moving images, were well accepted and useful.Further studies are needed to determine the characteristics of each system properly and to select the optimal system for each program, depending on the contents or system availability.

Figure 4 .
Figure 4. Comparison of problems by category; Image, sound, preparation, cost.