Evaluation of an Autonomous Vehicle User Interface for Sensory Impaired Users ()
1. Introduction
Currently, the issue of independent mobility remains a significant challenge for individuals with impairments, who often encounter limited or inaccessible options for transportation. However, emerging automated and assistance systems in vehicles hold promise in not only enhancing safety and comfort for all drivers but also enabling persons with impairments to full personal mobility. To realize this vision of inclusivity, autonomous vehicles (AVs) must be designed with principles that prioritize accessibility and accommodate diverse needs. This can be a daunting and particularly challenging task especially regarding enabling independent travel and use of AVs for persons with vision and/or hearing impairments. Persons with reduced vision and/or hearing have special requirements related to AVs, as defined by ITS America [1]. For persons with vision impairments, these are audio and non-visual ways of communication with the vehicle; options for facilitating the orientation in space and locating the vehicle; and information about road conditions and the surroundings, detecting possible obstacles when reaching the desired location. For persons with hearing impairments, the special requirements are clear visual ways of communication with the vehicle; converting all important sound signals to clear visual notifications and warnings; and access to aid devices for the hearing impaired (text-to-speech converters or vice versa).
According to the listed requirements, it is clear that a solution for reducing the transportation limitations of persons with sensory impairments can mainly be achieved through a different approach to designing the vehicle user interface (UI). This paper is focused precisely on inclusive UI solutions that could be effective as AV interface solutions for passengers with vision and/or hearing impairments.
2. Background and Motivation for Study
Reviewed literature showed an increasing trend of exploring user behaviors in AVs and most research includes using vehicle simulations to: analyze user behaviors in AVs to provide ideas about methods for increasing users’ trust in the AV system [2]; investigate user behaviors in critical situations [3]; provide insights into the users’ perceptions and behaviors [4]; identify informational preferences in the AV [5]; explore user requirements in an AV and suggest specific interior design and multimodal UI for facilitating in-vehicle [6] etc.
With regard to persons with impairments and the use of AVs, researchers examine attitudes towards AVs among these individuals, their trust in autonomous systems, acceptance of AVs, and willingness to own and/or use AVs [7]-[9]. Research shows persons with impairments express positive attitudes towards AVs and are willing to use self-driving transportation modes even at a higher cost, provided they are accessible and safe for everyday use [10]-[12].
Nonetheless, a small number of previous studies investigate how specific interior design and vehicle-passenger communication system solutions influence the AV usability by persons with certain types of impairments. Proposing and testing solutions is a crucial step that can help gather feedback and generate recommendations for the inclusive design of AVs. General guidelines were found in the form of listed opportunities for universal design of AVs and fully accessible AV checklists [1] [13]. However, more detailed information and design specifications are very limited. Since this research focused on persons with vision and hearing impairments and the use of AVs, few research papers with closely related topics were identified as most relevant.
Firstly, in the research of Kempapidis et al. [14], individuals with a vision impairment participated in AV test rides with the goal to determine the emotions and reactions among participants. The results showed a positive response to the experience using an AV and indicated that improvements in obstacle detection and smoother braking systems could help to eliminate negative aspects of the user experience.
Secondly, the case study of Ranjbar et al. [15], investigated if vibrotactile aid could enable persons with vision impairments and hearing impairments use AVs. The results show that persons with sensory impairments can travel independently with an adequate adaptation of the given information through auditory, tactile, or visual information channels.
Lastly, the experiment by Brinkley et al. [16], involves testing a prototype for an AV human-machine interface (HMI) designed for visually impaired users. Results show how the participants expressed increased trust in the AV and increased usability when interacting with the inclusive AV HMI.
The small number of previous researches that suggest concepts for inclusive in-vehicle communication systems (user interfaces) that can provide independent use of AVs by persons with sensory impairments indicate that further studies are needed in this area. Therefore, this study is aimed to provide insights into the specific UI features that affect the accessibility and usability AVs for people with sensory impairments.
3. Objectives and Research Questions
This study aimed to address three main questions:
Does the adoption of inclusive design principles for the graphical interface of the AV improve usability for sensory-impaired users?
Does the adoption of additional visual information through a head-up-display (HUD) in the AV improve usability for sensory impaired users? and
Does the adoption of a voice control option with auditory (natural language) feedback in the AV improve usability for sensory impaired users?
Addressing these questions was essential to understand the interactions between sensory impaired users and AVs, evaluate the inclusively designed visual and sound UI through capturing the user experiences, identify challenges and provide guidelines for further development of inclusive UI solutions for sensory impaired users.
4. Method
To analyze the possible benefits of applying inclusive principles when designing AV UIs, an inclusive UI was developed, containing both visual and sound enhanced communication modes, and evaluated using an immersive static driving simulator.
4.1. Interface Design
The developed UI for sensory impaired users was based on the most common approach for designing inclusive interfaces - multimodality. According to research, the optimal combination for persons with reduced vision and/or hearing is to use visual elements combined with auditory messages and tactile cues (female voice is preferred) [17]. Therefore, a visual interface was designed as an app for in-vehicle communication containing not only screen options, but also an option for voice control with auditory (natural language) feedback. In addition, HUD messages were designed as a combination of notification icons and sounds intended to be used as additional features to grab the passenger’s attention. General and specific guidelines for designing the visual and sound interface elements for persons with reduced vision and hearing were combined to generate the final solution [18].
The final interface was designed using Adobe XD. Initially, three variants were generated which underwent a first simulation that provided important feedback on how to optimize the final UI version [19]. The optimized UI version was then used in the driving simulator. The final design consisted of a simple wireframe containing a header, menu bar and options section which allowed the most important content to be positioned centrally on the screen. The navigation items were not designed to be constantly displayed on the screen in order to free space for larger buttons and text. Excessive text was avoided and replaced with universal, filled icons. The UI was designed with a high “AAA” contrast between the icons/text and the background. The high level of contrast “AAA” means the used contrast ratio is 7:1 to allow maximum inclusivity (the minimum required level of contrast ratio is 3:1) [20]. In addition, pre-determined choice options were included as selection buttons in order to simplify the use of the available options. Samples of the designed UI are provided in Section 4.2, Table 1.
4.2. Participants and Equipment
In total, 32 participants (4 females), from 24 to 52 years old (Мage = 31.75, SD = 6.46) without reported sensory impairment, were involved in the experiment. To simulate a vision acuity loss, participants were given glasses equipped with specially designed lenses to induce a blurred vision (Inclusive Design Toolkit, University of Cambridge) (Figure 1(a)). The study participants were given three pairs of the simulation glasses, which made their vision about 0.49 logMAR worse [21]. To imitate the reduced hearing capacity, the audio files of the user interface that represented the virtual in-vehicle assistant were processed with a filter based on the approximate hearing curve an 80-year-old person with typical age-related sensor neuronal hearing loss [22] [23].
To recreate the ride in an AV a driving simulator was used. It consists of a section of a vehicle cabin (including front seats, cockpit, dashboard, windscreen and mirrors, steering wheels and pedals, spatial audio) with a cylindrical projection screen (5m diameter) in front of it (Figure 1(b)). The use of the simulator helped for evaluation of the designed UI and was chosen because previous research has shown it is a valid tool for investigating human factors in the evaluation of in-vehicle systems [24].
(a) Cambridge simulation glasses (b) Driving simulator -Virtual Vehicle GmbH
Figure 1. Equipment used for the simulation: Vision impairment simulation glasses and driving simulator.
A portable device (10-inch) tablet was included in the simulator, through which the developed UI (app) was presented to the participants (Figure 1(b) and Figure 2). Two apps were used in this study - one being the inclusive variant and the other one being a regular interface designed with no inclusive features. The voice messages were implemented only in the inclusive UI as an additional accessibility feature. The two UI variants were included in order to capture the difference between using inclusive features and the possible benefits of its use for persons with sensory impairments. Differences between the standard and inclusive UI are given in Table 1. The screens in front of the cabin were also used as a virtual HUD displaying additional information regarding the ride and notification messages (Figure 2).
Table 1. The main differences between the standard and inclusive UI are as follows
Design techniques and principles |
User interface type |
Standard UI |
Inclusive UI |
Wireframe and organization into areas |
Always visible menu/Header displaying date, time, weather, info about locked doors, and logo/Title of selected option bar - small/Central control options display divided in 1, 2 or 4 sections/Back button |
Menu visible only on home page/Header displaying time, info about locked doors, and logo /Title of selected option bar - large/Central control options display not divided/Back button |
Important and high priority information |
Always positioned centrally |
Always positioned centrally |
Use of text, text size |
Text excluded where icons are self-explanatory, font sizes used 42, 48 and 30 pt, italic used |
Text included in combination with all icons, font sizes used 72, 54 and 48 pt, italic not used |
Contrast of colours |
4.0:1/3.2:1/3.4:1/3.5:1 |
9.9:1/8.7:1/7.3:1 |
Button sizes (for the 1920x1200px screen on the 10-inch tablet) |
Menu option buttons 230 × 174 px/Buttons for adjustments 253 × 253 px and 175 × 175 px/Play music buttons 153 × 153 px/Back button 93 × 68 px/Yes or No and Start or Reroute buttons 250 × 100 px |
Menu option buttons 396 × 396 px/Buttons for adjustments 340 × 340 px/Play music buttons 153 × 153 px/Back button 675 × 150 px/Yes or No and Start or Reroute buttons 250 × 100 px |
Icons |
Outlined |
Filled |
Choice options |
Adjustment options also available in the form of slider buttons |
Pre-determined only choice options only in the form of clickable buttons |
Audio |
Excluded |
Included as a voice control option through a simplified screen reader and keyword input required by user to select the options |
4.3. Experimental Design
The experimental design consisted of a 4x2 factorial design with two independent variables: “simulated sensory condition” and “UI design”. The sensory condition was treated as a between-subject factor, with 4 levels: normal vision and hearing, normal vision with hearing impairment, impaired vision with normal hearing, impaired vision and hearing. The UI design was treated as a within-subject factor, with two levels: standard graphics, inclusive graphics. This design resulted in 8 experimental conditions, as described in Table 2. Each participant was randomly assigned to one of the four groups defined by the sensory conditions, and had to interact with both UI designs.
Table 2. Experimental design - simulated sensory impairments and UI characteristics.
Sensory condition (between factor) |
User interface design (within factor) |
Standard UI |
Inclusive UI |
Group 1 - No impairment |
Standard graphics No audio |
Inclusive graphics Normal audio |
Group 2 - Hearing impairment |
Standard graphics No audio |
Inclusive graphics Degraded audio |
Group 3 - Vision impairment |
Standard graphics Simulation glasses No audio |
Inclusive graphics Simulation glasses Normal audio |
Group 4 - Vision & hearing impairment |
Standard graphics Simulation glasses No audio |
Inclusive graphics Simulation glasses Degraded audio |
4.4. Testing Procedure
Before beginning the experiment, each participant was provided with instructions and asked to fill out an informed consent form. An internal review board had previously approved the ethical aspects of the study. Our study adheres to the guidelines of the Declaration of Helsinki. The ethical aspects of the study were reviewed and approved by Virtual Vehicle Research IRB with protocol n. 22001. The participants were asked to imagine a situation in which they are a person with reduced visual and/or hearing acuity, i.e., the ability to discriminate details, on an AV for shared rides (an explanation was provided to them regarding what an AV is). Participants from the vision-impairment and Vision & hearing - impairment group were provided with the simulation glasses and were asked to keep them on during the whole experiment. Participants from the Hearing-impairment group and Vision & hearing - impairment group were also explained that in order to simulate their hearing acuity loss they will be listening to modified sound files during the whole experiment. Finally, they were required to enter the simulator and take a seat.
Participants were encouraged to use the UI displayed on the tablet in the cabin and the options of the interface which included: navigation, comfort settings, seat adjustment, climate control, light adjustments, and multimedia for listening to music. Additional information regarding the options were not provided in the participants’ instructions in order to determine if despite their “sensory impairment” they can understand and recognize all the information without having previous knowledge about them. It was emphasized to the participants that to achieve the goal it is very important for them to explore and use all the available features of the interface to get a good impression of it. They were encouraged to use the clickable (i.e., touch) buttons and, if possible, use the voice control option. They were also informed that messages are to appear on the screens in front of them as if they were projected onto the windshield to simulate a HUD. This approach was used to allow a free and spontaneous exploration of the UI functions by the participants and trigger realistic, natural reactions throughout the experiment. The choice was to exclude specific task-related performance measures like, e.g., completion time, since the main goal was to receive feedback about the overall user satisfaction in self-initiated interactions with the interface.
After entering the simulator, the participants were guided by the UI provided on the portable device ((10-inch) tablet) in front of them. Two rides were performed per participant and each lasted about 8 minutes. The route and scenario were the same for all groups and rides. The simulation began after every participant “started the ride” through the UI and the driving environment appeared on the screens in front of the cabin (Figure 2).
(a) Participant with a simulated sensory impairment interacting with the regular UI
(b) Participant with a simulated sensory impairment interacting with the inclusive UI
Figure 2. Participants during the simulation interacting with the designed UIs. Vehicle speed is shown on a virtual HUD on the simulator windshield.
4.5. Data Collection
The participants’ opinions, impressions and experiences during and after each simulation were collected through 1) a 3-part questionnaire and 2) an interview. The questionnaire consisted of 3 different Likert scales:
The first 12 Likert questions evaluate the trust in the interface and were a modified version of the “Checklist for trust between people and automation” [25];
The second 10-question Likert scale evaluate the usability of the interface and was a modified version of the “System Usability Scale (SUS)” [26]; and
The third Likert scale contained additional questions for collecting more information regarding the overall feeling of safety during the ride and special features of the UI: inclusive graphics (usability of buttons, icons, text, pre-determined choice options, etc.), HUD messages (understandability and effectiveness for grasping the attention), voice messages (helpfulness for sensory impaired users).
After the drive, participants exited the simulator and were interviewed in order to understand their impressions during the whole interaction with the UI in greater qualitative detail. An experimenter annotated their comments.
The collected results are presented and discussed in the following section.
5. Results
5.1. Questionnaire Results
The collected data from the first two sets of Likert questions (“Checklist for trust between people and automation” and “System Usability Scale”) were analyzed according to the proposed methods from the relevant sources stated above. Both scales provided a model for assessing trust and usability of the UI based on statements with both a positive and negative tone that measure polar opposites along a single dimension. The results are shown in Figure 3 and Figure 4.
Based on the charts, we can see that the inclusive UI received the highest score for both trust and usability from the “vision-impairment group” compared to all other groups with simulated sensory impairment. For the “Vision-impairment group”, the “Checklist for trust between people and automation” score increased from 16 (while using the non-inclusive, standard version) to 20.125 (while using the inclusive version), which is the highest score compared to all other groups. The SUS score for this group increased from 60.625/100 (non-inclusive) to 69.6875/100 (inclusive). This SUS score was second-highest after the SUS score of the non-inclusive UI from the “No-impairment group” (71.5625). Since the non-inclusive UI for the “No-impairment group” equals a standard UI for a standard user, i.e., a reference score (or baseline), the fact that the inclusive UI scores very near that value with visually impaired users indicates a great achievement, that almost entirely compensates for the loss of usability induced by the sensory impairment.
The “No-impairment group” preferred the non-inclusive UI since they rated it as more usable, based on the SUS score, which was expected. However, it was
Figure 3. Bar chart representing results from all groups and test rides to the “Checklist for trust between people and automation” questionnaire.
Figure 4. Bar chart representing results from all groups and test rides to the “System Usability Scale” questionnaire.
interesting to find out that the participants from the “No-impairment group” felt they trust the autonomous system slightly more during the ride when interacting with the inclusive graphics. The regular UI did score higher on the SUS, but the results from the checklist for trust showed an opposite result. This means that even though the non-inclusive UI did seem more appealing and useable for this category of users that had no sensory disabilities, they rated the inclusive graphics as more trustworthy.
The “Hearing-impairment group”, interestingly, showed a preference of the standard UI once more. The standard UI with no voice messages received a higher SUS and trust between people and automation score.
The results from the “Vision & hearing - impairment group” showed more usability and trust of the inclusive variant, but the data was not as evident as it was with the “Vision-impairment group”. For example, the SUS score, in this case, increased from 58.125/100 (non-inclusive) to 63.75/100 (inclusive), but this usability score (in the inclusive case) is lower in comparison to the total SUS score of nearly 70/100 (inclusive) calculated from the answers of the participants in the “Vision-impairment group”.
Data collected from the additional Likert questions are shown in Figure 5 to Figure 10, with statistical parameters (minimum, lower quartile, median, upper quartile, and maximum). Data was analyzed by a two-way repeated measures ANOVA to compare means between groups, interfaces and their interactions, and post-hoc multiple comparison Tukey tests were used to compare the means of observations between interfaces from the same impairment groups.
The main result is that the inclusive UI received the highest usability rate from the “Vision-impairment group”.
For the other impairment groups there was no significant improvement in the usability when interacting with the inclusive UI. The mean scores to the question “How well were you able to see and select the graphics?” regarding the inclusive UI were 4 (“Hearing-impairment group”) and 3 (“Vision and hearing impairment group”) showing no improvement compared to using the standard UI (Figure 5). Different impairment groups showed no significant difference between interfaces, F = 1.65, p > 0.05 (p = 0.18). There was no significant difference between the two interfaces, F = 0.8, p > 0.05 (p = 0.37). There was also no interaction effect present, F = 1.13, p > 0.05 (p = 0.34).
However, based on the multiple comparisons test, (adjusted p value = 0.049) we can see a significant evidence that the UI with inclusive graphics were more useable to persons with a simulated vision impairment compared to the non-inclusive graphics. The result to the same question for the “Vision-impairment group” (Figure 5) increased from 3.125/5 (non-inclusive) to 4.25/5 (inclusive). The mean score to this question was the highest in this group in comparison to all other groups with a simulated impairment (Figure 5).
In addition, the feeling of safety during the ride was highest in the “Vision-impairment group” when interacting with the inclusive UI (Figure 6). There was a significant difference between impairment groups (F = 28, p < 0.01). The overall feeling of safety during the ride while using the inclusive UI variant in this group was quite high – 4.875/5 (Figure 6) which is another positive indicator that the inclusively designed UI is suitable for persons with impaired vision helping them feel safe while traveling.
Figure 5. Box plot displaying statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the two test rides (1 & 2) of each of the four study groups (“No-impairment group”, “Hearing-impairment group”, “Vision-impairment group” and “Vision & hearing impairment group”) to the question “How well were you able to see and select the graphics?”.
Figure 6. Box plot displaying statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the test ride with the inclusive UI (2) of the three study groups with a simulated impairment (“Hearing-impairment group”, “Vision-impairment group” and “Vision & hearing impairment group”) to the question “How safe did you feel while driving an autonomous vehicle with a (simulated) impairment?”.
Regarding the effectiveness of the HUD messages, the scores to the question “Were the messages that appeared outside the vehicle (on the big screen) attracting your attention?” were average (around 3) showing no major difference between groups and interfaces (Figure 7).
The effectiveness of the HUD messages that appeared on the screen in front of the simulator received a mean score of around 3.5/5 from the “No-impairment group” for being good for attracting the passengers’ attention.
The effectiveness of the HUD messages decreased in the scenario where degraded audio messages were included (“Hearing-impairment group”) indicating that the participants with a simulated hearing impairment experienced a cognitive overload. While focusing on hearing the audio, they did not receive these pop-up messages as well as when they were shared with them only as graphical symbols on the screen. The result was similar with the group with a simulated vision and hearing impairment (“Vision & hearing impairment group”).
The HUD messages received a highest mean score from the “Vision-impairment group” (Figure 7). The resulting score for the question “Were the messages that appeared outside the vehicle (on the big vertical screen) attracting your attention?” was 4/5, which is higher than the other cases.
Figure 7. Box plot displaying statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the two test rides (1 & 2) of each of the four study groups (“No-impairment group”, “Hearing-impairment group”, “Vision-impairment group” and “Vision & hearing impairment group”) to the question “Were the messages that appeared outside the vehicle (on the big screen) attracting your attention?”.
The results related with the voice messages are given in Figure 8 and Figure 9. The answers to the questions “Was the content of the voice messages understandable?” and “How easy was it for you to hear the voice messages?” are relatively low, especially from the groups where a hearing impairment was simulated.
Figure 8. Box plot displaying robust statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the test ride with the inclusive UI (2) of the three study groups with a simulated impairment (“Hearing-impairment group”, “Vision-impairment group” and “Vision & hearing impairment group”) to the question “Was the content of the voice messages understandable?”.
Figure 9. Box plot displaying statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the test ride with the inclusive UI (2) of the three study groups with a simulated impairment (“Hearing-impairment group”, “Vision-impairment group” and “Vision & hearing impairment group”) to the question “How easy was it for you to hear the voice messages?”.
The received mean score from the “Hearing-impairment group” to the question “Was the content of the voice messages understandable?” was 3.125/5 (Figure 8) and to the question “How easy was it for you to hear the voice messages?” – 3.25/5 (Figure 9). These results matched the obtained scores from the first two Likert-scales where this group showed a preference of the standard UI with no voice messages. Later, the answers from the interview questions provided the reasons for these results and explained that persons were annoyed by the voice messages because they were not able to hear them well – “acoustic was irritating because I did not understand properly”, “audio was bad, but I understand that I was in hearing impairment group”, etc. The effect was similar with the “Vision & hearing - impairment group”. Based on the previously analyzed answers from the “Hearing-impairment group” where it became obvious that some of the voice messages have a negative effect that decreases the usability of the interface and the feeling of trust, it was concluded that the voice messages have the same effect in this case as well. As a result, the overall feeling of safety and trust was low despite the inclusive design of the interface – 2.125/5 (Figure 6).
On a positive note, the graphical design of the UI on the tablet received a high mean score of 4.5/5 for being useful to compensate the lack of information due to the hearing impairment (Figure 10).
Figure 10. Box plot displaying statistical parameters (minimum, lower quartile, median, upper quartile, and maximum) of answers from the two test rides (1 & 2) of the two study groups with a simulated hearing impairment (“Hearing-impairment group” and “Vision & hearing impairment group”) to the question “Was the visual information on the tablet useful for you to compensate the possible lack of information due to the hearing impairment?”.
5.2. Interview Results
The participants’ comments from the interview session were very beneficial to pinpoint all advantages and disadvantages of the UI and gather ideas for further improvement and development.
Answers to the open-type questions indicated an overall positive impression of the inclusive UI. Most of the participants stated that the easiest tasks for them were to use the menu items due to the clear layout and interactive structure. They felt the design was familiar and therefore easy to use. Participants also liked that they can use all the options with only 1 - 2 clicks. Some of the participants from the “No-impairment group” and “Vision-impairment group” stated that they enjoyed the voice messages and the option to use voice control as an alternative to clicking the buttons because they found it practical. Participants with a with simulated vision impairment explained that the UI provided on the tablet was easy to use and the options were visible.
When participants were asked what was the hardest task and what they disliked about the UI the most common answers were related with the voice messages. The participants explained that the voice messages were sometimes too long and by the end of the message, they tended to forget what type of input or keywords are required from them in order to use the options. This was especially emphasized when there was a hearing impairment simulated. Participants from the groups with a hearing impairment felt a bit frustrated that they were not able to hear the messages well and this interfered with their driving experience – “the options were read aloud to support the driver but took concentration and attention instead of supporting because of the hearing impairment”. Participants stated they would have preferred shorter messages and an option to repeat them.
6. Discussion
Analyzed results showed that the inclusive visual interface did play an important role in improving the usability for persons with sensory impairments. The visually impaired group was able to see and select the graphics meaning the ergonomic and inclusive principles were properly combined to increase the usability and trust in the system for visually impaired users.
Regarding the use of HUD messages as an additional visual interface, results showed that is not particularly beneficial for better capturing the attention of the sensory impaired users in critical situations, thus improving the usability.
The audio messages as an additional feature of the UI did not increase the usability for persons with a hearing impairment or both a vision and hearing impairment as it was initially expected. The persons who were not able to hear the messages felt annoyed and distracted and as a result less satisfied during the rides. This means that for hearing-impaired users the sounds that might be useful are short notification sounds combined with tactile modes of communication as a replacement of sound messages and virtual assistants. This would, however require additional testing and remains as a subject for further work on this topic.
Furthermore, since the UI testing with focus groups and a driving simulator in this research was limited due to several aspects, there are some other opportunities for further research which are open.
Firstly, the actual feeling of driving in an AV might not be accurately captured with driving simulators. The participants in the experiment were aware that they were using a driving simulator in a virtual environment. This awareness may have had some impact on their opinions of trust and comfort during interactions while the simulations lasted. Secondly, even though the disability simulation methods were effective to simulate specific sensory impairments, it remains difficult to fully understand what it is like to live with a particular disability. That is why it is necessary to conduct future testing of the developed concepts with focus groups with participants belonging to categories with different disabilities.
7. Conclusions
In this research, we conducted an explorative study of an AV UI developed for passengers with loss of acuity in vision, hearing or both. The interface was designed as a combination of an inclusive in-vehicle visual UI (app), voice messages and a HUD. The main goal was to provide insights into the specific interface features that affect the accessibility and usability of AVs for people with sensory impairments.
To evaluate the developed interface, the “Wizard-of-Oz” method was used where an AV ride was simulated through a driving simulator and focus groups with simulated visual, hearing, as well as combined acuity loss (simulated through vision impairment simulation glasses and specially filtered audio messages) participated in the experiment. The experimental design consisted of a 2x4 factorial design with two independent variables: “UI design” (within) and “simulated sensory condition” (between). The exploration of user experiences was done through survey data. The participants responded to three different structured Likert scales and open-type interview questions. Based on the results obtained, generally positive user experiences were reported while technical challenges were highlighted. Several main conclusions were drawn and discussed in this paper, based on which validated guidelines for the further development of inclusive UI solutions were provided.
The three initially defined research questions (given in section 3) were addressed individually:
(1) Does the adoption of inclusive design principles for the graphical interface of the AV improve usability for sensory impaired users?
The specially designed graphics of the inclusive interface proved to be much more user-friendly for those with visual impairments compared to the graphics of the standard UI. The ergonomic and inclusive design principles were effectively integrated in the UI resulting in enhanced usability and trust in the system for visually impaired users showing they can be used in future inclusive UIs.
(2) Does the adoption of additional visual information through a head-up-display (HUD) in the AV improve usability for sensory impaired users?
There was no significant evidence that HUD messages improved the accessibility for sensory impaired users. However, the participants did not rate it as unnecessary or unlikeable either, meaning that it can be included in AVs, but it is likely that it will not improve the inclusivity of the interior significantly.
(3) Does the adoption of a voice control option with auditory (natural language) feedback in the AV improve usability for sensory impaired users?
Based on the collected answers, the use of sound messages as an addition to the inclusive visual UI showed to be good for improving the AV usability for persons with vision impairments, but not for users with hearing impairments. This became clear from the answers of participants from the “Hearing-impairment group” and “Vision & hearing impairment group” who felt a cognitive overload from the distorted sound messages. That is why short notification sounds and tactile messages could be a better option to increase the UI usability for persons with hearing acuity loss.
In conclusion, we believe the suggested and evaluated concepts for inclusive in-vehicle communication systems in this paper provide a solid base for further studies in this area. The designed interface and received user feedback resulted in specific guidelines for further development of inclusive AV models, which is something that is still not fully provided in literature sources and other available research results.
We also hope this research draws more attention to the importance of applying human-centric and inclusive design principles in the development of autonomous vehicles to allow them to achieve their full potential and help persons with impairments overcome mobility barriers.
Acknowledgements
This work was partially financed by Virtual Vehicle Research GmbH, a COMET K2 Competence Centers for Excellent Technologies funded by the Austrian Federal Ministry for Climate Action (BMK), the Austrian Federal Ministry for Labor and Economy (BMAW), the Province of Styria (Dept. 12) and the Styrian Business Promotion Agency (SFG).
The authors would like to acknowledge Hassaan Islam, Virtual Vehicle Research GmbH, for his assistance in setting up the driving simulator for the experiment, and Manuela Prior, Virtual Vehicle Research GmbH, for her help to successfully conduct the study with the focus group and collect data. The authors would also like to thank Aleksandra Aleksovska, Faculty of Mechanical Engineering – Skopje, for her cooperation and help to develop the inclusive UI, Prof. Dr. Sofija Sidorenko, Faculty of Mechanical Engineering - Skopje, and Dr. Anton Fuchs, Virtual Vehicle Research GmbH, for their support and guidance.