The Influence of a User-Centred Design Focus on the Effectiveness of a User Interface for an Agricultural Machine

As agricultural machines become more complex, it is increasingly critical that special attention be directed to the design of the user interface to ensure that the operator will have an adequate understanding of the status of the machine at all times. A user-centred design focus was employed to develop two conceptual designs (UCD1 & UCD2) for a user interface for an agricultural air seeder. The two concepts were compared against an existing user interface (baseline condition) using the metrics of situation awareness (Situation Awareness Global Assessment Technique), mental workload (Integrated Workload Scale), reaction time, and subjective feedback. There were no statistically significant differences among the three user interfaces based on the metric of situation awareness; however, UCD2 was deemed to be significantly better than either UCD1 or the baseline interface on the basis of mental workload, reaction time and subjective feedback. The research has demonstrated that a user-centred design focus will generate a better user interface for an agricultural machine.


Introduction
A user interface is a means by which the user interacts with the target system. It is an important facet that helps the user to monitor, control and alter the target task environment [1]. The efficiency and effectiveness of the operation, as well as the workload and safety of the operator, depends on the information displayed to the user. Designers and researchers recommend that the typical user interface should encapsulate and emphasize the critical features of the target environment [1] [2] [3] [4]. Using the paradigm of situation awareness, [5] proposed the goal-directed task analysis to determine the critical information needs of the user. Based on the situation awareness information needs of the operator, designers and engineers can design the user interface by focusing on the essential interface elements to accomplish operational goals of the operator.
There are two important considerations for designing a user interface: 1) information requirements of the user, and 2) information presentation to the user. Information requirements of the user specify the quantity, type or variety of the information deemed necessary for the user to achieve job-related goals. For example, a car driver requires knowledge of the current speed of the car; without the knowledge of this critical information, it would be difficult to drive safely and lawfully. Information presentation demonstrates the form, look, feel and mode of the information communicated to the user so that the processing and utilization of the information can be efficient. Referring back to the previous example, the current speed of a car can be presented to the driver in many forms: using numeric text, by showing the movement of a needle on a dial gauge, using a combination of both numeric text and animated needle, or by any other means based upon the imagination and resourcefulness of the designer.
In this study, we have designed and evaluated a driver interface for a tractor air seeder system. This work was accomplished in two phases. During the first phase, individual elements of the driver interface were designed and evaluated on the basis of mental workload invoked and level of situation awareness that was enabled. Experimental results confirmed that the metrics of mental workload and situation awareness can be used by the designer to select individual interface elements [6]. During the second phase of the study, individual elements of the tractor air seeder interface were further modified based on knowledge gained in the first phase of the study and then integrated into a complex interface. Two versions of a user interface were developed and evaluated against a pre-existing interface that had been used previously as part of a tractor-air seeder simulator. This manuscript discusses the findings of the second phase of the study.

Evaluating User Interfaces
Design of a user interface according to the user's goals and information requirements is half the battle towards building an effective interface. Although designers apply many interface design guidelines based on human factors principles, it is still common that certain aspects of the interface may not work as intended. Evaluation of a user interface helps designers to identify ineffective features and other issues to further improve the interface. Commonly recommended interface evaluation methods include heuristic evaluation, cognitive walk-through, usability testing, and guidelines/standard inspection [7] [8]. Heu-  [7]. Heuristic evaluation requires many experts to evaluate the interface based on expertise gained over many years of professional practice. Usability testing yields data based on both objective and subjective evaluations which makes it more suitable for research and scientific studies. Multiple metrics can be used for usability testing. Situation awareness may be considered as a primary means for evaluation, as this metric is a widely accepted criterion of evaluation in many domains [9]. Reference [10] defined situation awareness as "the perception of the elements of the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future". From this definition, we can see that situation awareness consists of perception, comprehension, and projection. These are referred to as the three levels of situation awareness. Poor situation awareness could lead to ineffective, inadequate operational outcomes (perhaps leading to dangerous conditions). Reference [11] described several catastrophic airline crashes (Northwest Airlines MD-80 in 1987, US Air B-737 in 1989, or Korean Airlines Flight in 1983) which were directly or indirectly related to poor situation awareness of the operators of the automated flight system. A common reason for poor situation awareness relates to the presence of automation in the system. Automation may shift the role of the operator from "active participant" to "passive user or supervisor" [12]. In partially automated driving scenarios, adequate situation awareness is essential for the safety of the driver. During partially automated driving scenarios, human drivers are expected to take control of the situation whenever the situation demands attention (i.e. due to technology failure or technology limitation). This can be problematic as the driver has not likely been actively involved in decision-making leading up to the point of technology failure, and therefore, lacks complete understanding of the situation. It is for such reasons that it is critical to design a user interface that adequately supports the situation awareness of the user. Reference [13] developed and evaluated three interfaces for regenerative life support systems using the "ecological interface design" which considers the user-centered approach to better support the situation awareness of the operators. Results of the study have indicated that the interfaces which presented "situation-rich" information helped in better decision making.
Mental workload is another evaluation metric that has been used widely in human factors studies [14]. Mental workload can be defined as "the amount of cognitive capacity required to perform a given task" [15]. Evaluation of mental workload provides critical insights into design considerations and operational outcomes. As described by [14], most mental workload evaluation techniques can be categorized as analytical or empirical. The primary premise for this division is that analytical techniques (such as mathematical models and simulation models) do not require the operator to perform the task under investigation, while empirical techniques require the operator to perform the task under investigation. We can categorize empirical techniques into four divisions: primary task performance (e.g. time or error related), secondary task techniques (e.g. loading or subsidiary task), physiological or psychophysiological techniques (such as cardiac or brain activity, eye function), and operator opinion or subjective techniques. Further details about all these techniques can be read in [14]. Subjective techniques (self-reports by the operators) are more "sensitive" and "accurate" [16] and operators show better judgment about their workload among varied task conditions [17]. Reference [16] developed and tested a unidimensional mental workload scale called the Integrated Workload Scale (IWS). This scale has shown advantages such as simplicity, ease of administration, speed of use, and minimal obstruction with the task.
A simple characterstic such as response time can also be used to evaluate a user interface. Reference [18] (cited by [19]) used the user's response time in answering the questions related to information presented on an interface as a means for inferring the situation awareness attained by the user. Higher response time was associated with lower situation awareness. In another study related to workload, [20] reported that "accuracy decreased and reaction time increased as the difficulty of information processing requirements was increased". Higher levels of subjective workload were associated with a lower level of performance and increased reaction time. In a study comparing two interfaces in an intensive care unit [21], lower response time and higher situation awareness was observed for an "integrated" display compared to the traditional display. Reference [22] mentioned that "an increase in task load led to lower situation awareness and higher mental workload, reduced mission success and increased mission times". Overall, it can be concluded that higher response time can be associated with lower situation awareness, higher mental workload, or lower performance.

Interface Design Considerations Associated with Automation
The current trend is for automation to be incorporated into agricultural machines. Engineers are using sensor technology, combined with control systems, to automate various tasks that were previously completed manually. Although the operator may not be responsible for completing these automated tasks, the operator is still responsible for the overall operation of the machine. This implies that the operator should be provided with information that will enable him/her to fully understand the status of the machine-including the status of tasks that were completed autonomously. Therefore, the designer of a user interface should not forget to incorporate status information on tasks completed autonomously. If not, there is a chance that the operator will suffer from the so-called "out-of-the-loop" syndrome [5].

Research Objective
There is a wealth of information that has been published in numerous textbooks  [23]. From this base of knowledge, an experimental study was completed in a controlled laboratory environment in which multiple versions of display elements (pictorials/symbols) relevant to the monitoring of an agricultural air seeder were evaluated [6]. Using metrics of situation awareness and mental workload [6], we were able to identify preferred display elements. The objective of this research, therefore, is to investigate the potential benefit to the operator associated with using a display interface designed from a user-centred perspective.

Design of an Air Seeder User Interface
Reference [6] identified 12 elements or functions that are most vital to the efficient operation of an air seeder. These elements were: seed level status (tank levels), fertilizer level status (tank levels), fan RPM, seed application rate, seed depth (tool depth), fertilizer application rate, fertilizer depth (tool depth), tool pressure, blockage status, desired path of the unit, desired location of the unit, and current speed of the unit. In an earlier study completed in the Agricultural Ergonomics Laboratory, the interface shown in Figure 1 was developed for use with a tractor-air seeder simulator that was being developed for research purposes. At that time, minimal attention was given to the design of the symbols or pictorials chosen to represent the air seeder functions as the researchers were focused on development of a functioning simulator. Thus, the interface shown in  The individual elements that were compared in the experimental study by [6] are displayed in Figure 2. For the ease of presentation in this manuscript, the elements are displayed together.
To be able to achieve the objective of the current study, it was necessary to develop an integrated air seeder interface to compare against the baseline interface from the simulator (depicted in Figure 1). Individual elements from the study completed by [6] were used as the starting point. The first version of the interface designed according to user-centred design principles, hereafter referred to as UCD1, essentially consisted of the individual elements evaluated by [6], but with minor modifications to the elements used to display tool pressure, tool depth, and blockage ( Figure 3). Furthermore, an element was added in the centre of the interface to provide guidance information.
A second version of an interface, hereafter referred to as UCD2, was developed based on the results and feedback from the work reported by [6] ( Figure  4). The specific modifications are: 1) Tool Depth element: Some participants had difficulty making sense of a stationary tool with soil levels changing because this does not realistically reflect what would be happening with the machine. Participants also indicated a preference for the scale starting from the top to the bottom for the tool depth element. This feedback was used to develop an alternate element for depicting tool depth.
2) Blockage element: Most of the participants indicated a preference for green color over light blue color as an indication of the correct state (i.e. non-blockage state). Therefore, the color of the blockage elements was changed from blue to green. Figure 2. Air seeder monitoring elements that were evaluated in the study conducted by [6].   3) Tool Pressure element: Results from [6] indicated that the tool pressure element caused high mental workload-approximately 25% more than baseline conditions. Many study participants indicated difficulty in inferring information from the tool pressure element. Accordingly, changes have been made in the tool pressure element.
4) The coloring of the scales was modified to grayscale from red/green. 5) Marks on the scales were removed. 6) In the seed application rate, an additional animation showing the falling of the seeds was removed. 7) In fertilizer application rate, only one animation representing the falling fertilizer was displayed, instead of the four animations. 8) Another significant change regarding the placement of numeric readings was made; readings were moved from the bottom of each element to the middle of the scale.

Evaluation of Interfaces
Three interfaces (Old, UCD1, and UCD2) were compared using the metrics of situation awareness (levels 1 -3), mental workload, and response time in the lab environment. A simulation was developed in the Visual Basic programming language using Microsoft Visual Studio Express 2013. The simulation was constructed in such a way that values on the user interface fluctuated at random intervals while the participants monitored the values on the interface. When the simulation stopped, queries were presented to the participant on the screen to assess the participant's recall of the status of the various parameters and to determine the perceived level of mental workload. This study was completed in the Agricultural Ergonomics Laboratory in the Department of Biosystems Engineering at the University of Manitoba. The experimental protocol received human ethics approval from the University of Manitoba Education/Nursing Research Ethics Board. Participants were also asked to provide subjective feedback.
Thirty individuals (20 male, 10 female) were recruited to participate in the study. For convenience, recruitment was focused on the University of Manitoba campus, with the majority of the participants being University of Manitoba students. Ages ranged from 19 to 52 years with a mean age of 28.4 years. Only 10 participants had prior driving experience and most of the participants (29 out of 30) had no previous experience with agricultural machines. We did not screen participants using any eligibility criteria-all respondents were considered eligible to participate in the study.
The full experiment was divided into two sessions: low-level automation, and high-level automation sessions. Automation level and interface type were considered as independent variables while situation awareness, mental workload and response time were considered as dependent variables. In the low-level automation session, the user was responsible for both observing and correcting the situation. To correct the situation, the user was required to click on the incorrect (out-of-range) parameter on the interface (e.g. see the seed application rate, block-  Figure 3, and seed application rate, fan rpm and blockage in Figure 4). Response time is defined as the amount of time between the appearance of the error and correction of the error by the participant. During the high-level automation session, the user was only responsible for observing the situation; the user was not allowed to correct the situation by clicking on the interface. Half of the participants completed the low-level automation session first, and other half of the participants completed the high-level automation session first. After every simulation, study participants were asked questions related to situation awareness and mental workload (see Figure 5). The responses used to assess both situation awareness and mental workload were recorded by the simulation program during the experiment.
As recommended in the Situation Awareness Global Assessment Technique (SAGAT) [5], the questions asked were related to three levels of situation awareness. Responses received under "VALUE", "STATUS", and "FUTURE STATUS" headings were correlated to level 1 (perception), level 2 (comprehension) and level 3 (projection) situation awareness, respectively [5]. The degree of situation awareness attained by the participant was inferred based on the proportion of correct responses entered by the participant.
Participants reported their mental workload on a nine-point Integrated Workload Scale [16]. User's mental workload was inferred based on the user's selection on the nine-point integrated workload scale varying from "Not Demanding" to "Work Too Demanding". The numerical equivalent value of mental workload can vary from 1 to 9, where 1 represents "Not Demanding" and 9 represents "Work Too Demanding".
Subjective feedback regarding the three interfaces was collected after each experimental session using a paper form. Users were required to rate the three interfaces regarding the various criteria mentioned in the questionnaire (Table 1).  3 Rate the interfaces in terms of understanding or comprehension of the situation.

4
Rate the interfaces in terms of trend identification.

5
Rate the interfaces in terms of prediction of the future situation.

Research Hypothesis
As the factors being evaluated involved the respective levels of situation awareness, mental workload and reaction time for different interfaces, the hypotheses should be constructed with these parameters in mind. Consequently, the means of each respective interface form the most reasonable base for comparison, provided that potential random variations across factors for individuals is later acknowledged and accounted for during the analysis. As the dataset is heavily predicated on questionnaire responses from individual participants, accounting for the possibility of random variation was imperative. Linear mixed effect models were generated for each output parameter accounting for the input factors and the potential random variation per participant of each factor. Means were examined for each parameter in relation to their dominant factor using the emmeans() function in RMarkdown. Generated random effect models were used to predict values which were subsequently graphically and analytically compared to the originally measured values before being tested for the significance of random and fixed effects.

Mental Workload
The mental workload values for each respective interface were separated and sorted by interface, level of automation and steer type. The resulting measurements were arranged into a comparative boxplot, shown in Figure 10.
Examination of the mental workload shows that there is a reduction in mental workload for UCD2 when compared with the original interface and UCD1 (p = 0.000797) ( Table 3 & Table 4). It is interesting to note that there is an apparent and unexpected slight increase in mean mental workload when comparing UCD1 to the original interface.
Prediction of the values proved to be more reasonable, with a larger distribution of values found compared to the previous predictions for SA, as shown in Figure 11. R 2 was measured to be 0.823, indicating a reasonably significant approximation for the data.

Response Time
A significant decrease in mean reaction time is observed across interfaces as shown ( Table 5). Reaction times are only listed for low automation, as that was the only experimental scenario where a response was required from participants.
Analysis of the model indicated this difference was distinctively tied to the respective interface to a significant degree (p = 9.232e−08) ( Table 6).

Subjective Feedback
At the end of each session (high-automation or low-automation), subjective feedback was collected. Participants were asked to rate the three interfaces as best, average or worst based on their experience during the experiment and were required to evaluate the three interfaces using five questions related to the per-     . For design B in comparison to A, we found p = 0.000, Odds ratio = 2.844 and Coefficient = 1.04251. A positive coefficient and greater than 1 odds ratio indicate that B has been rated significantly higher than A. Odds of being rated higher for B are 2.8 more than that of A. Similarly, odds for being ranked high for C are 12.9 times more than A. The high odds ratio of C indicates that subjects have placed very high confidence in the UCD2 interface.

Conclusions
Based on the analysis performed on the data obtained, UCD2 was found to be superior to both UCD1 and the original interface across the assessed parameters. Situation awareness changes were marginal between interfaces, and were more closely correlated to the type of automation being used in the experiment than the interface design used. While the means for perception (level 1 SA), comprehension (level 2 SA) and projection (level 3 SA) were not found to have any significant distinction across interfaces, UCD2 was shown to have lower average levels of both participant mental workload and response time than either UCD1 or the original interface. The null hypothesis can be said to have been rejected for both mental workload and reaction time. The decrease in mental workload and response time, coupled with the relative parity of all interfaces for situation awareness provide sufficient grounds to state that the second alternate interface (i.e. UCD2) shows general improvement when situation awareness, mental workload and reaction time are the focus of assessment. There are two important limitations to be noted with this research. First, the research was conducted on a user interface for an agricultural air seeder. Although the results might be generalizable to other agricultural machines, we do not have any experimental evidence to confirm that these results can be applied to the user interface for other types of agricultural machines. Second, the research results are based on the perceptions of participants with limited experience using agricultural machines. It is unknown whether the results would differ if participants would have been recruited from the population of experienced air seeder users.