Fairness Assessment in Networked Games with Olfactory and Haptic Senses

In this paper, we carry out QoE (Quality of Experience) assessment of fairness between players in a networked balloon bursting game with olfactory and haptic senses. We also make a comparison of the fairness among different types of networked games with olfactory and haptic senses in the assessment. Then, we clarify the differences owing to usage of olfactory and haptic senses among the games. As a result, we illustrate that the fairness is largely affected by the delay difference, and we also demonstrate that the allowable range depends on the type of games, type of senses which are employed in the games, and the play methods.

However, in the games, when multiple media streams related to such sensations are transmitted over a network where QoS (Quality of Service) is not guaranteed like the Internet, media synchronization is disturbed and QoE (Quality of Experience) [11] of fairness among users may seriously be deteriorated owing to the network delay, delay jitter, and packet loss [5] [6] [7] [8].
In [5], the authors investigate the influence of network delay on the fairness among players in a networked real time game with haptic sense.In the game, each of two players lifts and moves the player's object and eliminates a target by using his/her haptic interface device.The two players compete with each other for the number of eliminated targets.They only deal with hard objects in the game.It is important to investigate the fairness in games which deal with soft object.
In [6] and [7], the authors deal with soft objects in a networked balloon bursting game with haptic sense.In the game, two players burst balloons (i.e., soft objects) in a 3D virtual space by using haptic interface devices and compete for the number of bursted balloons.In [6], the authors investigate the trade-off relationship between the operability of haptic interface device and the fairness between the two players by QoE assessment.In [7], they also examine the influences of network delay on QoE such as the operability and fairness.However, in the games, the authors do not deal with the olfactory sense.
In [8], the authors deal with a networked fruit harvesting game in a 3-D virtual space by using olfactory and haptic senses.In the game, a player (called the harvester) picks a fruit from a tree by a haptic interface device, and the player can perceive the smell of the fruit output by an olfactory display when the fruit approaches his/her view point.The other player (called the harvest impeder) tries to make the harvester perceive the smell of a different fruit by picking the fruit and moving it towards the harvester's viewpoint.They investigate the influence of the time it takes for a smell to reach a player on the fairness between the two players by QoE assessment.However, the authors do not take account of the influence of the network delay for haptic sense.Therefore, in this paper, we deal with the networked balloon bursting game [6] with olfactory and haptic senses and investigate the influence of network delay on the fairness (the fairness in the game means that the same condition is provided to all the players [12]) between players by QoE assessment.We also compare assessment results with those in the networked games with olfactory and haptic senses, and clarify the differences owing to usage of olfactory and haptic senses among the games.
The remainder of this paper is organized as follows.Section 2 describes the networked games.Section 3 explains the assessment environments and assessment results are presented in Section 4. Section 5 makes a comparison of results among games and Section 6 concludes the paper.

Networked Balloon Bursting Game with Olfactory and Haptic Senses
In this section, we introduce the networked balloon bursting game with olfactory and haptic senses.We have made the networked balloon bursting game with olfactory and haptic senses by adding a function handling the olfactory sense to the networked balloon bursting game with haptic sense [6].
Figure 1 shows the system configuration of the balloon bursting game with olfactory and haptic senses and displayed images of the virtual space.The system consists of two terminals (terminals 1 and 2), each of which has a PC, a haptic interface device (Geomagic Touch [13]), an olfactory display called SyP@D2 [14] (in this paper, the distance between each player and the olfactory display is set to about 0.3 m as in [8]), and a headset.In the game, a player at each terminal bursts balloons with his/her haptic interface device by moving a virtual stylus (a CG image of the stylus of the haptic interface device) in a 3D virtual space.
When the player touches the balloon with the tip of the stylus, the reaction force is perceived through the haptic interface device and he/she can feel the softness of the balloon.The reaction force applied to the haptic interface device is generated by the haptic rendering engine [15], which uses the object shape and material properties such as stiffness and friction for calculation of the reaction force.The player feels larger reaction force as the penetration depth of the stylus becomes larger.The penetration depth of the stylus is the distance from the surface of the balloon to the tip of the stylus.The force applied to a balloon when the player pushes the balloon with the stylus is equal to the reaction force against the player (for further details of calculation method of the reaction force, the reader is referred to [16]).The balloon is distorted when the player pushes the balloon with the stylus.If he/she pushes it strongly, the balloon is largely distorted, and it is burst and disappeared.Then, he/she hears a sound of bursting it via the headset.
There are four balloons in the virtual space.Two blue balloons are located on the left side and the other two pink balloons are located on the right side.In order to avoid the situation of trying to burst the same balloon simultaneously, in this paper, player 1 bursts two blue balloons on the left side and player 2 bursts two pink balloons on the right side for simplicity.A smell randomly-selected from among smells of grapefruit, banana, and apple is included in each balloon.When a balloon is burst by a player, the smell included in the balloon is generated by the olfactory display and the player can feel the smell.Since a terminal cannot output multiple smells at the same time, each terminal only outputs the smells included in the balloon burst by the local player, and each player does not affect each other.Then, the player answers what the small is by inputting the number (1: Grapefruit, 2: Banana, 3: Apple) with the keyboard.The two players compete with each other for the number of correct judgments.
In the game, the two players can burst their own balloons and judge what the smell is independently, and compete for the number of correct judgments.The players can also behave as one answerer and one impeder.As an answerer, he/she tries to answer the types of smell correctly, and as an impeder, he/she tries to impede the answer so that the answer cannot make a correct judgment.In this paper, we deal with the former case; for the latter, the reader is referred to [17].
In order to keep the consistency at the players' terminals, we employ the local lag control [18], which buffers the local information according to the network delay from the local terminal to the other terminal.

Assessment System
Our assessment system is shown in Figure 2. As shown in Figure 2, two termin als are connected to each other via a network emulator (NIST Net [19]).The network emulator generates an additional constant delay for each MU [20] (media units, each of which is the information unit for media synchronization) transmitted between the two terminals.The MU includes the timestamp, the position information of the cursor, and the information of balloons (ID number, position, and whether the balloon is displayed or not).We employ UDP as the transport protocol to transmit the MUs.We call the constant delay from terminal 1 to terminal 2 constant delay 1, and that from terminal 2 to terminal 1 constant delay 2 (see Figure 2).Also, the local lag at terminal 1 is called as local lag 1, and that at terminal 2 is called as local lag 2. Note that the local lags 1 and 2 are set to the same values as the constant delays 1 and 2, respectively.

Assessment Method
We carried out QoE assessment subjectively and objectively to investigate the influence of network delay on the fairness in the networked balloon bursting game with olfactory and haptic senses.
Before the assessment, each pair of subjects played the balloon bursting game with olfactory and haptic senses for three times to get used to the game on the condition that constant delays 1 and 2 were set to 0 ms; that is, the same condition was provided to the pair.By practicing, each subject knew how to burst a balloon by using a haptic interface device and how to feel and judge the smell.
We carried out the assessment by changing constant delay 1 from 0 ms to 500 ms and setting constant delay 2 to 0 ms, 100 ms, 300 ms, or 500 ms.Note that the constant delays were chosen in random order for each pair.In what follows, the assessments when constant delay 2 is set to 0, 100, 300, and 500 ms are referred to as assessments 1 through 4, respectively.
For subjective assessment, we enhanced the single-stimulus method of ITU-R BT. 500-12 [21].Before the assessment, each pair of subjects was asked to practice on the condition that there was no constant delay.The fairness (as described earlier, the fairness in the game means that the same condition is provided to all the players [12]) at this time was regarded as the standard (i.e., fair) in the assessment.After practice, they were asked to play the networked balloon bursting game with olfactory and haptic senses on the condition that there were constant delays.
After each stimulus, each subject was asked to base his/her judgment about the fairness based on the five-grade quality scale (see Table 1).The subject gave a score from 1 through 5 for each stimulus.We obtained the mean opinion score (MOS) [21] by averaging (simple arithmetic mean) scores of all the subjects.
We also carried out objective assessment at the same time as subjective as sessment.We employ the average difference in number of correct judgments as an objective assessment measure.The average difference in number of correct judgments is defined as the number of correct judgments of player 1 minus that of player 2. It took 30 seconds for each stimulus, and the number of subjects (males and females) whose ages were between 20 and 30 is 16.

Assessment Results
We show subjective and objective assessment results for assessments 1 through 4 in Figures 3-6, respectively.Each figure shows MOS of fairness and the average difference in number of correct judgments versus the delay difference (constant delay 1 minus constant delay 2).We also plot the 95% confidence intervals in the figures.
From Figure 3, we find that the MOS value decreases as the delay difference becomes larger.The average difference in number of correct judgment is positive and the average difference decreases as the delay difference becomes larger.This is because that subjects at terminal 2 can burst balloons more easily than those at terminal 1 since local lag 2 is 0 ms (equal to constant delay 2); as the delay difference increases (i.e., constant delay 1 increases), the local lag 1 also becomes larger, and it becomes more difficult to burst balloons and later to feel the smell for subjects at terminal 1.Therefore, the fairness is degraded.In the figure, we also see that when the delay difference is smaller than about 100 ms, the MOS value is higher than 3.5; this means that the deterioration in QoE is allowable for delay differences smaller than about 100 ms [22].
In Figure 4, we notice that when the delay difference is smaller than around 0 ms, the MOS value increases and the average difference in number of correct judgments decreases as the delay difference increases; when the delay difference is larger than around 0 ms, the MOS value decreases and the average difference in number of correct judgments is positive and decreases as the delay difference becomes larger.This is because when the delay difference is smaller than 0 ms (i.e., constant delay 1 is smaller than constant delay 2), subjects at terminal 1 can burst balloons more easily and feel the smell included in the balloons more quickly than subjects at terminal 2. When the delay difference is around 0 ms, the situations between two subjects are almost the same, the subjects feel fair, and the average difference in number of correct judgments is close to 0. However, when the delay difference is larger than about 0 ms, the tendency is almost the same as that in assessment 1.In the figure, we also find that when the delay difference is larger than about -75 ms and smaller than about 75 ms, the MOS value is higher than 3.5.
In Figure 5, we observe that when the delay difference is smaller than 0 ms, the MOS value increases and the average difference in number of correct judgments decreases as the delay difference increases; when the delay difference is larger than 0 ms, the MOS value decrease and the average difference in number  of correct judgments is positive and decreases as the delay difference becomes larger, which is the same as that in assessment 2. In the figure, we also find that when the delay difference is larger than about -75 ms and smaller than about 100 ms, the MOS value is higher than 3.5.
From Figure 6, we notice that the MOS value increases and the average difference in number of correct judgments decreases as the delay difference increases.This is because when the delay difference is smaller than 0 ms (i.e., constant delay 1 is larger than constant delay 2), subject at terminal 1 burst balloons more difficultly and feel the smell included in the balloons later, and the fairness is degraded.In the figure, we also see that when the delay difference is larger than about −200 ms, the MOS value is higher than 3.5.From the above results, we can say that in the networked balloon bursting game with olfactory and haptic senses (two players burst balloons and judge the smell without impeder), the MOS value of fairness decreases as the absolute delay difference increases.When the delay difference is smaller than about 0 ms, the MOS value increases as the delay difference increases; when the delay difference is larger than around 0 ms, the MOS value decreases as the delay difference increases.When the delay difference is larger than about -75 ms and smaller than about 75 ms, the MOS value is higher than 3.5, and the deterioration in fairness is allowable.That is in the game, the allowable range (in the range, players feel that the synchronization error is allowable) is from about -75 ms to 75 ms.If we define that when the MOS value is larger than 4, players hardly perce-ive synchronization error (i.e., the imperceptible range, in which players cannot perceive the synchronization error), we can see that the imperceptible range is from about -25 ms to 30 ms when constant delay 2 is 0 ms and 100 ms, when constant delay 2 is larger than 100 ms, imperceptible range does not exist.

Comparison of Results among Games
In this section, we make a comparison of assessment results of fairness among the following games: the networked real-time game with haptic sense [5], networked balloon bursting game [6] [7], and networked fruit harvesting game [8].

Assessment System
Since we will make a comparison of assessment results of fairness among different games, we briefly introduce the assessment environments of the games in this subsection.
In these games, two games deal with haptic media and other three games deal with olfactory and haptic media (please note that in the networked fruit harvesting game, the authors only take account of the influence of the network delay for olfactory sense).The authors change the constant delay generated for MUs transmitted to different terminal of players in the games.In the assessment of networked fruit harvesting game with olfactory and haptic senses, the authors only generate constant delay for the olfactory media, and the constant delays in the game is changed from 0 ms to larger values than those in the other games.
In the networked real-time game with haptic sense [5], networked balloon bursting game with haptic sense [6], and the game in this paper, the player play games without impeders; in the networked fruit harvesting game with olfactory and haptic senses [8] and networked balloon bursting game with olfactory and haptic senses [17], players behave as answerers and impeders, when the players (answerers) play the game, the impeders try to impede the answerers.
In the games, the authors assessed the fairness subjectively and objectively, and the single-stimulus method is employed for subjective assessments.In the networked real-time game with haptic sense [5], subjects were asked to give scores base on a three-grade scale (in order to compare with other results, we mapping the results to the Five-grade quality scale); in the other games in table 2, the subjects were asked to give scores base on a five-grade quality scale as described in the previous subsection.

Comparison of Results among Five Games
In order to clarify the differences owing to usage of olfactory and haptic senses among the games, we compare the results in this paper with assessment results in the networked real-time game with haptic sense [5], the networked balloon bursting game with haptic sense [6], the networked balloon bursting game with olfactory and haptic senses [17], in which one of two players behaves as an answerer and the other behaves as an impeder, and the networked fruit harvesting game with olfactory and haptic senses [8].We summarize the assessment results (delay differences) of fairness in the games in Table 2.We here assume that when the MOS value is higher than 3.5, the delay difference is allowable (i.e., the allowable ranges [22]), and when the MOS value is higher than 4.0, which we select as an example, subjects cannot perceive the delay difference (i.e., the imperceptible ranges).
First, we compare the assessment results of the networked balloon bursting game with olfactory and haptic senses with those of the networked real-time game with haptic [5] and networked balloon bursting game with haptic sense 2) From about -25 ms to 25 ms (delay of one side is (constant delay 1) 20 ms) 3) From about -10 ms to 10 ms (delay of one side (constant delay 1) is 40 ms) 1) From about -40 ms to 40 ms (delay of one side (constant delay 1) is 0 ms) 2) From about -20 ms to 20 ms (delay of one side (constant delay 1) is 20 ms) 3) From about -5 ms to 5 ms (delay of one side (constant delay 1) is 40 ms)

Players eliminate targets independently (without impeders)
Networked balloon bursting game with haptic sense [6] 1) Smaller than about 120 ms (constant delay 2 is 0 ms) 2) Smaller about 75 ms (constant delay 2 is 100 ms) 3) From about -75 ms to 75 ms (constant delay 2 is 300 ms) 4) Larger than about -120 ms (constant delay 2 is 500 ms) 1) Smaller than about 50 ms (constant delay 2 is 0 ms) 2) From about -25 ms to 50 ms (constant delay 2 is 100 ms) Players burst balloon independently (without impeders) Networked fruit harvesting game with olfactory and haptic senses [8] Olfactory 1) Smaller than about 680 ms (constant delay 2 is 0 ms) 2) Smaller than about 180 ms (constant delay 2 is 500 ms) 3) From about -180 ms to 220 ms (constant delay 2 is 700 ms) 4) Larger than about -180 ms (constant delay 2 is 1000 ms) 1) Smaller than about 600 ms (constant delay 2 is 0 ms) 2) Smaller than about 100 ms (constant delay 2 is 500 ms) 3)From about -20 ms to 20 ms (constant delay 2 is 700 ms) 4) Larger than about 380 ms (constant delay 2 is 1000 ms) Players judge fruits' smell with impeders Networked balloon bursting game with olfactory and haptic senses [17] Olfactory and Haptic 1) Smaller than about 175 ms (constant delay 2 is 0 ms) 2) Smaller than about 150 ms (constant delay 2 is 100 ms) 3) Larger than about -150 ms (constant delay 2 is 300 ms) 4) Larger than about -200 ms (constant delay 2 is 500 ms) 1) Smaller than about 75 ms (constant delay 2 is 0 ms) 2) From about -75 ms to 25 ms (constant delay 2 is 100 ms) 3) From about -50 ms to 50 ms (constant delay 2 is 300 ms) 4) Larger than about -25 ms (constant delay 2 is 500 ms) Players judge the smells included in burst balloons with impeders Networked balloon bursting game with olfactory and haptic senses in this paper 1) Smaller than about 100 ms (constant delay 2 is 0 ms) 2) From about -75 ms to 75 ms (constant delay 2 is 100 ms) 3) From about -75 ms to 100 ms (constant delay 2 is 300 ms) 4) Larger than about -200 ms (constant delay 2 is 500 ms) 1) Smaller than about 30 ms (constant delay 2 is 0 ms) 2) From about -25 ms to 30 ms (constant delay 2 is 100 ms) Players judge the smells included in burst balloons independently (without impeders) ※ : We only list media which are taken into account in the fairness assessment.[6].From Table 2, we notice that in the networked real-time game with haptic sense [5], when constant delay 1 is 0 ms, the allowable range is from about -45 ms to 45 ms and the imperceptible range is from about -40 ms to 40 ms; when constant delay 1 is 20 ms, the allowable range is from about -25 ms to 25 ms and the imperceptible range is from about -20 ms to 20 ms; when constant delay 1 is 40 ms, the allowable range is from about -10 ms to 10 ms and the imperceptible range is from about -5 ms to 5 ms.Therefore, we can say that in the networked real-time game with haptic sense [5], the allowable range is from about -10 ms to 10 ms and the imperceptible range is from about -5 ms to 5 ms.
In networked balloon bursting with haptic [6], we find that when constant delay 2 is 0 ms, the allowable range is smaller than about 120 ms and the imperceptible range is smaller than about 50 ms; when constant delay 2 is 100 ms, the allowable range is smaller than about 75 ms and the imperceptible range is from about -25 ms to 50 ms; when constant delay 2 is 300 ms, the allowable range is from about -75 ms to 75 ms; when constant delay 2 is 500 ms, the allowable range is larger than about -120 ms.However, when constant delay 2 is 300 ms and 500 ms, MOS values are smaller than 4 and the imperceptible range does not exist.Therefore, we can say that in the networked balloon bursting game with haptic sense [6], the allowable range is from about -75 ms to 75 ms.
By comparing with the games, we see that the MOS value of fairness decreases as the absolute value of delay difference increases, and there exists the allowable range of delay difference and the allowable range of delay difference depends on games contents.We also find that the allowable range of delay difference in the networked balloon bursting game with haptic sense is almost the same as that of the networked balloon bursting game with olfactory and haptic senses.That is, the allowable range is hardly affected by the olfactory sense in this case.
We also compare the results in the previous section with the games using olfactory and haptic senses [8] [17].From Table 2, we observe that in the networked fruit harvesting game with olfactory and haptic senses [8], when constant delay 2 is 0 ms, the allowable range is smaller than about 680 ms and the imperceptible range is smaller than about 600 ms; when constant delay 2 is 500 ms, the allowable range is smaller than about 180 ms and the imperceptible range is smaller than about 100 ms; when constant delay 2 is 700 ms, the allowable range is from about -180 ms to about 220 ms, and the imperceptible range is from about -20 ms to about 20 ms; when constant delay 2 is 1000 ms, the allowable range is larger than about -180 ms and the imperceptible range is larger than about 380 ms.From these results, we know that in the networked fruit harvesting game with olfactory and haptic senses, the allowable range is from about -180 ms to about 180 ms and the imperceptible range is from about -20 ms to about 20 ms.
In the networked balloon bursting game with olfactory and haptic senses [17], we find that when constant delay 2 is 0 ms, the allowable range is smaller than about 175 ms and the imperceptible range is smaller than about 75 ms; when constant delay 2 is 100 ms, the allowable range is smaller than about 150 ms and the imperceptible range is from about -75 ms to 25 ms; when constant delay 2 is 300 ms, the allowable range is larger than about -150 ms and the imperceptible range is from about -50 ms to 50 ms; when constant delay 2 is 500 ms, the allowable range is larger than about -200 ms and the imperceptible range is larger than about -25 ms.Therefore, we can say that in networked balloon bursting game with olfactory and haptic senses [17], the allowable range is from about -150 ms to 150 ms, and the imperceptible range is from about -25 ms to 25 ms.
By comparing with five games, we notice that the allowable range and imperceptible range of delay difference in [8] and [17] are larger than those in this paper.This is because the play method in [8] and [17] is different from that in this paper.That is, the allowable range in the game where answerers are impeded by the impeders is larger than that in the game without impeders.We also find that the allowable range and imperceptible range in [8] are different from those in [17].This is because only the olfactory sense is taken into account in the fairness assessment in [8] and the olfactory and haptic senses are taken into account in the assessment in [17].
From the above results, we can say the allowable range and imperceptible range of delay difference are dependent on games contents, media types which are employed in the games, and the play methods.To obtain more generalized conclusions, we further need to deal with various types of games.

Conclusions
In this paper, we carried out QoE assessment of fairness between players in networked balloon bursting game with olfactory and haptic senses (two players burst balloons and judge the smell without impeder), and compare the assessment results with those in the networked real-time game with haptic sense, the networked balloon bursting game with haptic sense, the networked balloon bursting game with olfactory and haptic senses (one of two players behaves as an answerer and the other behaves as an impeder), and the networked fruits harvesting game with olfactory and haptic senses.As a result, we found that the fairness is largely affected by the delay difference; that is, the MOS values decrease as the absolute value of the delay difference becomes larger.We also noticed that the allowable range depends on the type of games, type of senses which are employed in the games, and the play methods.Therefore, when we develop networked games using multiple senses or study QoS control for the games, it is necessary to take into account the playing methods and senses used in the games.Also, it is important to clarify the allowable and imperceptible ranges.
As the next step of our research, we need to carry out the assessment with other types of smells and game designs.Also, it is important to investigate the influences of network delay jitter and packet loss.Furthermore, it is important to study QoS control based on the results in this paper to maintain the fairness

Figure 1 .
Figure 1.System configuration of networked balloon bursting game.

Figure 2 .
Figure 2. Configuration of assessment system.

Table 2 .
Assessment environments of games.