Preliminary Study for Feasibility of Driver Agent in Actual Car Environment
—Driver Agent for Encouraging Safe Driving Behavior (3) ()
1. Introduction
In recent years, Japan has become a more aged society and the proportion of drivers of 65 years of age and over has increased. Moreover, the number of traffic accidents caused by elderly drivers in Japan has been increasing and it is becoming a social problem. Drivers aged between 65 and 74 years of age are more likely to cause an accident than any other age group [1] [2]. Extant studies have indicated that one of the primary reasons behind the increase in the number of accidents caused by elderly drivers is the impact of aging on cognitive, visual, and physical functions. However, a lack of this mode of transportation can diminish the quality of life of the elderly and increase the possibility of them developing dementia [3]. Moreover, high-level disparity exists between the evident changes in biological functions of individuals of the same age, which means it is inappropriate to determine driving capability based solely on age. Therefore, it is necessary to determine a method suitable for evaluating driving capability and to design support methods in line with an individual’s requirements. Previous attempts have generally focused on the use of information display devices (such as small displays, car navigators, and head-up displays) and presentation methods based on sounds, voice, and vibration [4]. Extant studies have also attempted to improve driver behavior [5] to deal with negative adaptation [6] or false recognition of sensors. In addition, a few studies have also developed communication robots for cars [7] [8], where agents and robots accompany drivers [9] [10]. However, most previous research has been conducted using a driving simulator (DS) rather than an actual car environment.
The goal of our research is to reduce the number of traffic accidents caused by drivers, particularly the elderly. Moreover, previous research has suggested that both encouraging self-awareness in driving behavior and driving with fellow passengers have potential to reduce the traffic accident rate [11]. Thus, this study proposed using a driver agent in the form of a compact communication robot to encourage safer driving by helping drivers recognize their own behaviors. This agent provides the driving support during driving by attention awakenings and the feedback support based on driving behavior evaluation. Our previous experiments [12] have confirmed that the proposed agent can improve driving behavior using DS. Therefore, to confirm the feasibility of our agent in actual car environment, we conducted first experiment of “driving with robot agent” using actual car.
In this paper, we first introduce our previous experiments in Section 2, and system of proposed agent in Section 3. Then, in Section 4, we describe a set of preliminary experiments using our agent in an actual car environment designed to evaluate the level of its acceptability and distraction based on subjective evaluation and analysis of driver fixation points. Finally, we discuss the results derived from the experiments.
2. Related Work
2.1. Effect of Form of Driving Support Agent
Three forms of driving support agent have been considered: a voice agent such as a car navigation system, a visual agent displayed on an LCD monitor around a dashboard or on a smartphone, and a robot set around a dashboard. A previous experiment was conducted where these three agent forms provided the same driving support to elderly and non-elderly drivers [13].
The results suggested that the robot form was significantly more noticeable, familiar, and acceptable than the other two agent forms to both elderly and non-elderly drivers. In particular, the elderly found sudden vocal support difficult to understand, whereas the robot motion induced a sound that indicated when the agent was about to offer support. This feature could be seen as advanced notice of the offer of support via a mode other than vision, which could help drivers focus their attention on the offered support. For non-elderly drivers, coping with vocal support was not a difficulty; however, they found the visual agent too distracting, which led them to evaluate the form as least acceptable.
The robot form is a physical object and has stronger presence than the other forms. Analysis of driver fixation points during driving indicated that the presence of an agent does not necessarily lead to huge disturbance while driving. For the elderly, fixation points during driving diverged most with the voice agent and converged most with the robot agent. It has been reported that the accident rate could be reduced considerably if elderly drivers were accompanied by a fellow passenger, which has become known as the fellow passenger effect [14] [15]. The results revealed that the divergence of fixation points whilst driving was suppressed if the form of the agent was presented more clearly. This implies that the robot agent might trigger the fellow passenger effect because elderly drivers tend to consider a robot as a fellow passenger.
2.2. Driving Agent Improvement of Driving Behavior
The proposed agent would be expected to improve driving behavior via two support functions: driving support and review support. Thus, a DS experiment was conducted in which elderly and non-elderly drivers were presented with three different supports: driving support only, review support only, and their combined use [12]. We analyzed the changes both in driving performance over three weeks and in subjective evaluation of the agent. Driving performance was evaluated using three indices: safe confirmation time at an intersection with a stop sign, and minimum passing speed and maximum width in pedestrian/parked car avoidance. For example, after three weeks use of combined support, the safe confirmation time of elderly drivers increased from 1.7 s to 3.6 s and that of non-elderly drivers increased from 1.9 s to 4.2 s. Moreover, the passing speed of elderly drivers reduced from 31.9 km/h to 16.1 km/h and that of non-elderly drivers reduced from 31.5 km/h to 18.9 km/h. The results for all three conditions revealed that use of an agent improved the driving behavior for both elderly and non-elderly drivers, and that the combined use of driving support and review support was most effective. Furthermore, analysis of the relationship between the biofunctions of elderly drivers and the improvement effect suggested that elderly drivers, whose cognitive or visual function were impaired because of aging, tend to take compensatory action based on the agent support [16].
The results of the subjective evaluation regarding acceptability and distraction revealed that elderly drivers rated feedback support highest, whereas non-elderly drivers rated driving support highest. The result implies that elderly drivers tend to desire driving evaluation and feedback because they are concerned about their driving behavior. In contrast, non-elderly drivers, who generally have confidence in their own driving ability, tend to accept a new service or technology for fun and safer driving.
3. Driver Agent
3.1. Concept
Parker et al. [17] suggested that driving behavior is determined based on the driving situation and the driving model acquired from the experiences of the drivers themselves. Thus, drivers will revert to the same driving behavior if the driver model does not change. Analysis of the relationship between the biofunctions of drivers and the collision rate suggests that elderly drivers who have self-awareness of their driving ability tend to drive more safely than do drivers without such self-awareness [11]. To change a driver model to a safer one is to make drivers aware of their own driving behavior (i.e., self-awareness).
In this study, with the aim of reducing the accident rate for elderly drivers, we proposed a driver agent system that provides driving support advice during driving and review support to encourage changes in driving behavior through self-awareness. Moreover, as an agent form, we selected a commercially available communication robot designed for home use, and we expected its acceptability to increase based on a sense of reliability and familiarity gained through daily usage.
3.2. Architecture of Driver Agent
For the objective of this research, we are developing a driver agent system for an actual car environment. Figure 1 shows the architecture of our proposed agent
![]()
Figure 1. Architecture of driver agent system.
system. The system consists of a smartphone (Android), a portable communication robot, and a cloud. As the hub of the system, we developed a smartphone application that receives data from a camera, Controller Area Network (CAN), and other devices via Wi-Fi or Bluetooth, and uploads these data automatically to the cloud service. Furthermore, this application connects to the robot and to a turntable for the robot, and it relays action commands from the cloud to the robot and the turntable. The system is being developed as a common service for communication robots that have the functions of speaking and motion. However, the size of the robot is an important consideration regarding its use in an actual car. Hence, we chose the RoBoHoN (SHARP Co., Ltd.) model as the robot for use in our system. On the cloud, there are map data that include details of intersections and the speed limit of each road, support models consisting of several rules for controlling the agent, and driving evaluation algorithms for each traffic scene.
The driver agent has two primary functions [13]. One is the driving support function that arouses driver attention and suggests revision of the current driving operation based on uploaded data. The support content is determined based on the support model with reference to driving instructor data. The traffic scenes for the first support model were selected by referring to a report of traffic accidents associated with elderly drivers [1] [3], i.e., intersections with a stop sign, parked car/pedestrian avoidance, and traffic confluence. The small robot provides support using voice and motion based on the action command received from the cloud. Moreover, the turntable changes the robot’s yaw based on the type of the support being provided. The second function is review support, which evaluates driver behavior and provides feedback. After a period of driving, the application automatically receives driving evaluations and it offers feedback to the driver as a review list including one or more good scenes and fewer than three bad scenes. When the user selects one of the scenes to view, a 10-s movie is played and the robot vocalizes the comment to the driver. Through these supports, the driver agent is able to make the driver aware of their own driving behavior and to encourage them to improve it.
3.3. Position and Gesture of Agent in an Actual Car
The position of the robot in an actual car is an important issue for consideration. There are certain Japanese laws and guidelines regarding the position and use of electronic devices in a car. The Japan Automobile Manufacturers Association Inc. has certain guidelines relating to the position of a car navigation system [18]: 1) the device should be placed under 10˚ from the viewpoint of the driver, 2) the driver should be able to see the scene ahead in their peripheral field of view when gazing at the device, and 3) the device should not obstruct the driver seeing the automobile’s instruments. Moreover, for the agent to generate the fellow passenger effect, we presumed that placement of the agent to the left of the driver would be optimal because the driver’s seat is usually on the right-hand side of the vehicle in Japan. Therefore, we defined the position of the robot as being to the front and left of the driver (Figure 2).
The orientation and motion of the agent are also important factors to consider in ensuring the driver is not distracted. If the robot faces toward the driver during driving, the driver might be attracted to look at the robot’s face, which could possibly constitute a distraction [19]. Thus, the robot usually faces forward (away from the driver) when it is not providing support. To express driving support, several gestures were used by the robot in previous experiments [13], i.e., it moved its right arm up and down twice to express speed reduction. However, a driver might struggle to recognize the gestures of a small robot whilst driving, and this difficulty could increase the risk of a traffic accident. Furthermore, if the robot has several types of gesture, a driver might become distracted in trying to recognize a specific gesture. However, a previous study [13] has reported that robot motion and the sound associated with the motion of the robot give advanced notice of the offer of driving support, which reduces the mental workload and the reaction time of the driver [20]. Therefore, in this study, the agent used the same gesture for all types of support.
4. Experiment
In this study, we conducted preliminary experiments using an actual car to evaluate the acceptability of and distraction by our proposed agent. To this end, we analyzed both the subjective evaluation and the fixation points of the subjects during driving.
The experiments were performed after obtaining the approval of the Nagoya University Ethics Committee. In advance of the experiments, we held a meeting with the corresponding department in Aichi Prefecture, Aichi Prefectural Police Headquarters, and Chubu District Transport Bureau regarding our agent system and the use of a robot in an actual car. As a result, we obtained approval from each department for us to conduct our experiments on public roads.
![]()
Figure 2. Position of the robot in an actual car environment.
4.1. Method
The subjects drove a car for approximately five to eight minutes around a predefined experimental course on the campus of Nagoya University (Figure 3). The course included an intersection with a stop sign, and pedestrian and parked car avoidance scenes, although the locations and numbers of pedestrians and parked cars around the course were not controlled by the experimenter. However, sufficient numbers of both scenes were encountered during the experiments. Ten individuals (six males and four females) with an average age of 41.4 years participated in the experiments. As the preliminary experiments were the first trial of using an agent in an actual car environment, we opted to avoid using elderly subjects to reduce the risk of a traffic accident.
4.2. Experimental Conditions
In the experiments, we defined two experimental conditions: driving with the agent and driving without the agent. The robot agent was placed to the front and left of the driver (left side in Figure 2). Under the condition of driving with the agent, the agent provided driving support to the driver. In these experiments, the driving support comprised arousing driver attention. This support involved approach notifications regarding the intersection with a stop sign, pedestrians, and parked cars. On approaching each hazard, the agent provided support through vocalization and motion.
By design, our agent system is controlled automatically based on GPS and map data. However, there are no map data available for the campus of Nagoya University. Therefore, in our experiments, the agent was controlled by Wizard of Oz (WoZ). To control the agent manually, we defined two rules regarding driving support. One concerned the priority of information. In a real and uncontrolled environment, several types of traffic scene often occur at the same time.
![]()
Figure 3. Experimental course in Nagoya University.
For example, a pedestrian and a parked car might be recognized at an intersection with a stop sign. In our experiments, the agent prioritized information regarding the intersection over that concerning pedestrians or parked cars. This was done because the location of the intersection was static and easily recognized on the map. In dealing with information on pedestrians and parked cars, priority was given to whichever was closer. The second rule concerned the reduction of frequency of information provision. Previous research has shown that support offered too frequently can annoy the driver. Therefore, if the same traffic situation was found to continue, the agent only provided information regarding the first one and it withheld other information for five seconds. For example, if there were three pedestrians in front of the car, the agent would provide an approach notification regarding the closest one but it would omit issuing notifications regarding the other pedestrians.
4.3. Procedure
All subjects participated in experiments under both experimental conditions. The procedure of the experiment was as follows. Each subject communicated with RoBoHoN for 15 minutes before driving the car in order to familiarize themselves with the robot. After that, the subject drove the car around the practice course until they became proficient in driving the PRIUS (Toyota Co., Ltd.). They then drove around the experimental course once without the agent and twice with the agent.
Logged data were collected from the CAN and the sensors of the smartphone (e.g., speed of car, acceleration, and GPS). Whilst driving, all subjects were equipped Tobii Pro Glasses 2 (Tobii Co., Ltd.) to collect data on driver fixation points. Moreover, when having completed the experiment, the subjects answered a questionnaire regarding the agent.
4.4. Hypotheses
In previous experiments using DSs, subjective evaluation of the acceptability of the agent used was high. Thus, we expected little significant difference between the previous results obtained using DSs and our results derived using an actual car. Furthermore, in the previous experiment, there was no significant difference between the proportion of driving instances that the subject gazed at the designated agent area with an agent and the proportion without an agent. Thus, we expected that the difference between proportions while driving on a real road would be small.
5. Results
5.1. Subjective Evaluation of Driver Agent
The results of the average subjective evaluation of driving with the agent are shown in Figure 4. The subjects assigned subjective scores between one and seven to the following questions about the agent: Q1: favorability, Q2: reliability,
![]()
Figure 4. Subjective evaluation scores: “Actual Car” represents scores of our experiments; “DS” represents scores of previous DS experiments [13].
Q3: familiarity, Q4: desire to use, Q5: usability, Q6: contribution to safe driving, Q7: comfortability, Q8: not causing annoyance, and Q9: not causing distraction. The columns denoted “Actual Car” in Figure 4 represent the results of our experiments. For comparison, the columns denoted “DS” in Figure 4 show the results of using an agent for driving support in previous DS experiments with nine subjects [13]. We conducted a Welch’s t-test on each of the nine questions. The results revealed no significant difference between the two conditions, but the reliability score of Actual Car was lower than that of DS (p < 0.1). An introspection report from the subjects revealed certain comments about the frequency of the support offered and the support provided when many target objects were detected. The subjects answered that they consciously made efforts to drive safely because of the agent support. Moreover, they reported that they felt the motion of the agent representative of a human passenger. Six subjects stated that they wanted the agent to offer support more frequently and to provide additional types of support. Conversely, when encountering a number of pedestrians, the agent provided notification in the form “there is a pedestrian” about the nearest target, but some subjects found it curious that there was no information regarding the other pedestrians. Moreover, delay in providing information about objects that appeared suddenly also confused the subjects.
The aim of this study is to implement the driver agent in the real world to provide safe driving support services. Thus, we also conducted a questionnaire survey regarding the acceptable monthly cost of agent services. The average price suggested was US$13.5 per month. In addition, we conducted a Godspeed questionnaire [21] to evaluate the impression of the robot for considering whether there is the negative bias caused by the appearance or impression of the robot. The results of this questionnaire are shown in Figure 5. All the average scores were above score 3. In particular, likability was rated highest over score 4. Subjects reported positive impressions about the robot, which might have affected the results of other subjective evaluations. We also conducted a questionnaire to ask the subjects about the position of the agent in the car. We asked
![]()
Figure 5. Result of Godspeed questionnaire.
them to rank their preferred positions from first to third based on the locations shown in Figure 6. Most subjects preferred a position on the dashboard (positions 1 - 4). In our experiments, the agent was positioned at the right-hand side of position 1. However, some subjects wanted the agent placed closer to the driver’s seat because for seeing the agent easily whilst driving. Furthermore, the position below the dashboard was considered undesirable because the position was out of sight of the driver.
5.2. Fixation Points Analysis
If the existence of an agent attracts the attention of the driver whilst driving, the use of the agent could distract the driver, which might represent a problem concerning safety. Therefore, in our study, we analyzed driver fixation points and we calculated the proportion at which the subjects gazed at the agent whilst driving.
5.2.1. Method
Using Tobii Pro Glasses 2, we determined the fixation points of the subjects. The subject wore the glasses and drove the car under two conditions. The glasses have a front camera for recording front images depending on the face angle of the subject. All fixation points are recorded as X-Y coordinates on the front image. The device recognizes a fixation point at 10-ms intervals. To analyze the proportion of the gaze of the subjects toward the agent, we used a Tobii Pro Lab software automatic plotting function on the front images. Moreover, we set the area of interest (AOI) as the position of the agent in the car and we automatically counted the number of fixation points inside the AOI under the two conditions with and without the agent. By setting the AOI in both conditions and comparing the number of fixation points, the difference in the number between
![]()
![]()
Figure 6. Results of preferred position in an actual car.
the two conditions would express the proportion to which the agent attracted the attention of the driver whilst driving. The recognition accuracy of fixation points is usually affected by differences both of individual subjects and of the environment. In this study, analysis was performed on fixation points collected from seven of the subjects.
5.2.2. Results
The fixation points presented as a heat map recognized by the Tobii Pro Lab software under the conditions with and without an agent are shown in Figure 7. Areas that a driver gazed at frequently and for a long time are shown in red. It can be seen that the focus of attention of all drivers was usually the front space, although their gaze occasionally diverted to the rearview mirror or speedometer. During the experiments, each subject received driving support on 10 - 15 occasions. Only a few fixation points were observed around the position of the agent. The results of the distraction proportion under both conditions and the total duration that the subjects gazed at the AOI, calculated based on the number of fixation points, are shown in Figure 8. The accuracy of recognizing fixation points is different between subjects. Therefore, we normalized the distraction
![]()
(A) ![]()
(B)
Figure 7. Heat map of driver fixation points from seven subjects whilst driving under both conditions. (A) With agent; (B) Without agent.
proportion by calculating the proportion of each subject based on the total number of fixation points of each subject. The ratio under the condition without the agent was 0.0018, whereas the proportion with the agent was 0.0016. We conducted a Welch’s t-test on the results and found no significant difference between them. In addition, the duration that the subjects gazed at the AOI was 0.18 s under the condition without the agent and 0.21 s with the agent. Clearly, the duration of both is short and very similar.
![]()
Figure 8. Average distraction proportion and duration of driver fixation points within AOI under both conditions.
6. Discussion
The results of the subjective evaluation revealed that the acceptability of the agent was higher than score 4 (intermediate score), which is the same as in previous work [13]. In particular, although the subjects drove an actual car with an agent in these experiments, the agent was found not to annoy the subjects. There was almost no difference between the results obtained using an actual car and those derived from earlier DS experiments. Of the results derived in this study, only the reliability score of the Actual Car was lower than found following the DS experiments (p < 0.1). The trade-off between the frequency of offering driving support and risking annoyance has been confirmed in several previous studies. In this study, to reduce the frequency of supports, if the same traffic situation was found to continue, the agent was configured to provide information regarding only the first issue and to omit information on subsequent issues. However, some subjects misinterpreted these omissions regarding pedestrians or parked cars as false recognition by the agent. Thus, the subjective score regarding reliability was lower in comparison with the results derived from earlier DS experiments. In addition, we assumed that delay in offering support also reduced the appreciation of reliability. This result suggested the problem for an additional trade-off, i.e., with regard to the relationship between the reduction of the frequency and reliability of support. In an actual car environment, false recognition, delay, data noise, and sudden changes of situation occur. To design robust interaction against such difficulties and to inform the driver regarding the uncertainty and performance limitations of the agent will be objectives of our future work.
The subjects tended to like the agent to be placed close to their line of sight, even if they did not gaze at it directly whilst driving (Figure 8). Furthermore, the result of the Godspeed questionnaire showed a high likeability score for the RoBoHoN robot. The distance might be related not only to the question of agent visibility but also to a feeling of rapport between the driver and the agent like the personal space. The favorable impression of the robot might have given positive bias to the evaluations of its acceptability and level of distraction. On the other hand, the evaluation on the perceived intelligence of the robot was 3.5. When the agent acquires additional functionality and higher accuracy in the future, the mismatch between the robot concept and its functions might promote negative bias of the subjective evaluation of its reliability.
As shown in Figure 7, the greatest concentration of fixation points was toward the road in front of the car; this was followed by the rearview mirror or speedometer. Analysis of the fixation points during driving revealed that the subjects rarely gazed at the agent. Usually, the subjects drove around the experimental course for five to eight minutes. However, the distraction ratios under the condition with and without the agent were 0.18% and 0.16%, respectively, i.e., small values with no significant difference between them. Each subject received driving support on 10 - 15 occasions, however, the results suggest that the provision of these notifications did not attract the gaze of the subjects. We also analyzed the timing when the driver fixation points were recognized as being toward the position of the agent. In almost cases, the driver gazed to the left-hand side when they turned left. We have to consider safety when using an agent in actual car for realizing the system. However, the overall results implied that the presence of the agent did not attract the attention of the driver and that the possibility of driver distraction by the agent would be low. Moreover, the subjects found it possible to notice and understand the driving support offered whilst driving without having to gaze at the agent. The robot form agent produces the sound of an actuator or the turntable as advance notice of the support notifications, and the subjects reported easily noticing the support because of the sound or motion of the robot in their peripheral field of vision. Furthermore, the subjects considered the agent as a fellow passenger, and we thought this feeling was enhanced because of the presence of the robot as a physical object. These findings indicate certain advantages of using a physical robot as an agent.
Limitations: We need to conduct the experiment using elderly drivers as well. Moreover, the agent was controlled by WoZ in the current experiment. Therefore, experiments using an automatically controlled agent on a public road are required for accurate evaluation of the agent system. Thus, we are conducting further experiments using both elderly and non-elderly drivers.
7. Conclusion
In this study, we conducted a set of experiments to evaluate the acceptability of and distraction by our proposed agent using an actual car environment. We analyzed the subjective evaluation and driver fixation points of the subjects whilst driving with the agent. The results revealed positive acceptability of the agent, and the subjects generally desired greater communication from the agent whilst driving. Furthermore, from analysis of the fixation points, it was found that the subjects did not gaze directly at the agent whilst driving, which suggests that the presence of the agent in an actual car does not distract the driver. In future work, we will conduct experiments using an actual car on a public road with both elderly and non-elderly drivers.
Acknowledgements
This research was in part supported by the Center of Innovation Program (Nagoya University COI; Mobility Innovation Center) of the Japan Science and Technology Agency. We also thank T. Sato and Y. Kobayashi for developing the system.