Extracting Human Reaction Time from Observations in the Method of Constant Stimuli

Hongyun Wang^1*, Maryam Adamzadeh¹, Wesley A. Burgei², Shannon E. Foley², Hong Zhou³

¹Department of Applied Mathematics, University of California, Santa Cruz, CA, USA.
²U.S. Department of Defense, Joint Intermediate Force Capabilities Office, Quantico, VA, USA.
³Department of Applied Mathematics, Naval Postgraduate School, Monterey, CA, USA.

DOI: 10.4236/jamp.2022.1011220                

Abstract

We consider the psychophysical experiments in which the test subject’s binary reaction is determined by the prescribed exposure duration to a stimulus and a random variable subjective threshold. For example, when a subject is exposed to a millimeter wave beam for a prescribed duration, the occurrence of flight action is binary (yes or no). In experiments, in addition to the binary outcome, the actuation time of flight action is also recorded if it occurs; the delay from the initiation time to the actuation time of flight action is the human reaction time, which is not measurable. In this study, we model the random subjective threshold as a Weibull distribution and formulate an inference method for estimating the human reaction time, from data of prescribed exposure durations, binary outcomes and actuation times of flight action collected in a sequence of tests. Numerical simulations demonstrate that the inference of human reaction time based on the Weibull distribution converges to the correct value even when the underlying true model deviates from the inference model. This robustness of the inference method makes it applicable to real experimental data where the underlying true model is unknown.

Keywords

Method of Constant Stimuli, Psychophysical Experiments, Binary Outcome, Biovariability, Subjective Threshold, Human Reaction Time

1. Introduction

We consider a subclass of psychophysical experiments [1] [2] satisfying the properties: 1) the power of a stimulus source is fixed; the total amount of stimulus applied on the test subject is specified by the duration of exposure to the stimulus source; 2) the test subject’s outcome is binary: positive/null response; a positive response has an associated observable action while a null response has no action; 3) the subjective threshold on the exposure duration for producing a positive outcome is a random variable, fluctuating from one exposure test to another; and 4) if it occurs, a positive response is initiated at the time when the subjective threshold is reached; the time of observed actuation of positive response is delayed from the time of its initiation; this time delay is the human reaction time.

A possible situation for the type of psychophysical experiments described above is the occurrence of flight action when a test subject is exposed to a millimeter wave beam [3] [4] [5]. During the exposure, the electromagnetic energy is absorbed by the skin [6]. The absorbed energy goes into increasing the skin temperature. Thermal nociceptors in the skin are activated upon the local temperature reaching the activation temperature [7]. The electrical signal produced by nociceptors is proportional to the total number of nociceptors activated [8] [9]. When the pain signal received by the brain is strong enough, the brain issues instructions for the muscles to carry out a flight action (evading the beam and/or turning off the beam power) [3] [4]. For a millimeter wave beam of fixed specifications, both the number of nociceptors activated and the electrical signal produced by nociceptors increase with the exposure duration. As a result, there is a subjective threshold on the exposure duration for the pain signal to reach the threshold to initiate the flight action [5] [10]. This subjective threshold is a random variable, reflecting the underlying biovariability. It fluctuates from one test subject to another, from one exposure test to another on the same subject [11]. The observed actuation time of flight action, if it occurs, is delayed from the time when the pain signal reaches the threshold for initiating the flight action: it takes time for the pain signal to propagate from the exposed skin to the brain, for the instruction signal to propagate from the brain to muscles, and for muscles to actuate the flight action [12] [13]. This time delay is the human reaction time. These time quantities are illustrated in Figure 1.

In the method of limits (MoL) [14] [15], the beam is kept on until the observed occurrence of flight action. In each MoL test, flight action occurs eventually and the time of observed flight action is recorded in the data set. The time of observed flight action is the sum of 1) the random subjective threshold on the

Figure 1. Flight action is induced by an exposure duration exceeding the fluctuating subjective threshold. The time of observed flight action is delayed from its initiation at the subjective threshold by the human reaction time.

exposure duration for inducing flight action; and 2) the human reaction time, both of which are unknown. If we know one of these two quantities, we can calculate the other from the data. When both of these two quantities are unknown, the data set of observed flight action times from MoL tests does not allow us to extract the subjective threshold and the human reaction time simultaneously.

In the method of constant stimuli (MoCS), the exposure duration is prescribed before the start of beam power [14] [16]. The exposure may or may not lead to flight action, depending on whether the realized sample of subjective threshold in that test (a random sample) is lower or higher than the prescribed exposure duration. The binary outcome regarding the occurrence of flight action is recorded in the data set. In addition, if flight action occurs, the observed time of its actuation is recorded in the data set. We will see that this data set from MoCS tests allows us to extract both the median subjective threshold and the human reaction time. First, we notice that the data set of binary outcomes is independent of the human reaction time; an adaptive Bayesian method has been proposed to design the MoCS tests and to infer the median subjective threshold from the observed binary outcomes [17]. This has been done in our previous study of millimeter wave exposure tests [11]. In the current study, we focus on extracting the human reaction time based on the data of observed flight action times collected in a sequence of MoCS tests designed by the adaptive Bayesian method [17].

The rest of this paper is organized as follows. In Section 2, we introduce mathematical notations and models for describing the process of exposure tests and the associated observations. In Section 3, we lay out the inference formulations and methods. In Section 4, we carry out Monte Carlo simulations to generate artificial data sets to test the performance of the inference method proposed. In particular, we test the situation where the underlying true model for data generation is different from the inference model. In Section 5, we discuss the main results and conclusions obtained.

2. Models of Exposure Tests and Observations

2.1. Method of Constant Stimuli (MoCS)

We consider the situation where a test subject is exposed to a millimeter wave beam in experiments. During exposure, a fraction of the beam power is absorbed into the skin and it heats up the tissue underneath the skin surface. When the beam power is relatively high, the relevant thermal effect of beam exposure takes place within a short time period. Over that short time period, the effect of surface radiation cooling and surface convection cooling is negligible; the effect of blood flow cooling is also negligible. Here the relevant thermal effect refers to heating the skin tissue to activate the thermal nociceptors. Under these assumptions, when exposed to a beam of constant (time-invariant) high power, within the relevant time frame, the skin temperature increases monotonically with time [18] [19].

Thermal nociceptors are activated in the region where the local temperature is increased above the activation temperature, leading to a heat sensation. As the activated volume gets larger, the heat sensation becomes stronger. Eventually, when the heat sensation exceeds the test subject’s tolerance, the subject quickly moves away to avoid the beam and/or quickly turns off the beam power (we shall call this the flight action of the test subject).

In the method of limits (MoL), the beam power is kept on and steady until flight action is materialized/observed, upon which the beam power is turned off either by the experimenter or by the test subject. The actual duration of exposure (from the start to the end of beam power) varies with the test subject’s personal tolerance on heat sensation, which fluctuates from one test subject to another, from one exposure test to another on the same subject.

In the method of constant stimuli (MoCS), in each test, the exposure duration is predetermined before the start of beam power. After the exposure of prescribed duration, flight action may or may not occur. In each individual test, the binary outcome regarding the occurrence of flight action is recorded. When it does occur, the time of observed flight action is also recorded, which contains the human reaction time.

2.2. The Occurrence of Flight Action

We view a millimeter wave exposure as a stimulus response process. The binary response is the occurrence of flight action. In the general situation, the stimulus is described by several variables: the power density of the beam used, the beam spot size, and the duration of exposure. When the power density and beam spot size are fixed, the stimulus is solely described by the exposure duration. Corresponding to the personal tolerance on heat sensation, there is a subjective threshold on exposure duration for inducing flight action. Similar to the case with the personal tolerance on heat sensation, the subjective threshold on exposure duration is a random variable, fluctuating from one test subject to another (lateral biovariability), from one exposure test to another on the same test subject (longitudinal biovariability). In a test, if the prescribed exposure duration exceeds the realized sample value of subjective threshold in that particular test, flight action occurs. If it occurs, the time of observed flight action is delayed from the time of its initiation at the time of subjective threshold. At the time of subjective threshold, the exposure has activated enough thermal nociceptors and generated a sufficiently large electrical signal that will eventually result in flight action even if the beam power is turned off right at that time instance. The time delay from the initiation to the actuation of flight action is caused by that the pain signals travel from nociceptors via neuron fibers to brain; the brain generates instruction signals; the instruction signals travel from brain to muscles; and finally the muscles carry out the flight action. This time delay is referred to as the human reaction time.

To facilitate the discussion, we list mathematical notations for the quantities and random variables introduced above.

· $t_{\text{E}}$ : prescribed exposure duration, a predetermined quantity in each test

· ${\eta }_j\equiv {\left(t_{\text{E}}\right)}_j$ : prescribed exposure duration in test j.

· $t_{\text{c}}\equiv t_{\text{c}}\left(\omega \right)$ : subjective threshold on exposure duration for inducing flight action, a random variable. Here $\omega$ represents the randomness due to biovariabilities.

· $m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\left(\omega \right)\right)$ : median of the subjective threshold for flight action.

· $s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\left(\omega \right)\right)$ : standard deviation of the subjective threshold for flight action.

· ${\left(t_{\text{c}}\right)}_j$ : the realized sample value of random variable $t_{\text{c}}\left(\omega \right)$ in test j.

· $t_{\text{R}}\equiv t_{\text{R}}\left(\omega \right)$ : human reaction time, a random variable independent of $t_{\text{c}}\left(\omega \right)$.

· $m_{\text{R}}\equiv \text{median}\left(t_{\text{R}}\left(\omega \right)\right)$ : median of the human reaction time. In this study, we focus on the case of $t_{\text{R}}\left(\omega \right)$ = deterministic = $m_{\text{R}}$ (i.e., the human reaction time is an unknown but deterministic quantity).

· $I\equiv I\left(\omega \right)$ : binary outcome regarding the occurrence of flight action, a binary random variable.

· $I=1$ : occurrence; $I=0$ : no occurrence.

· $I_j$ : the realized sample value of $I\left(\omega \right)$ in test j.

· $t_{\text{F}}\equiv t_{\text{F}}\left(\omega \right)$ : time of observed flight action, a conditional random variable. $t_{\text{F}}\left(\omega \right)$ is defined only if $I\left(\omega \right)=1$ (i.e., if flight action occurs).

· $t_j\equiv {\left(t_{\text{F}}\right)}_j$ : the realized sample value of $t_{\text{F}}\left(\omega \right)$ in test j with $I_j=1$.

The time quantities described above are illustrated in Figure 1. In the framework of stimulus response, the binary outcome $I\left(\omega \right)$ is completely determined by the prescribed exposure duration $t_{\text{E}}$ and the random subjective threshold $t_{\text{c}}\left(\omega \right)$.

$I\left(\omega \right)=\{\begin{array}{ll}1\hfill & \text{if}{t}_{\text{c}}\left(\omega \right)<t_{\text{E}}\hfill \\ 0\hfill & \text{if}{t}_{\text{c}}\left(\omega \right)>t_{\text{E}}\hfill \end{array}$ (1)

Mathematically, binary random variable $I\left(\omega \right)$ is derived from random variable $t_{\text{c}}\left(\omega \right)$ with the prescribed exposure duration $t_{\text{E}}$ as a parameter. The stimulus response function describes the probability of flight action vs the prescribed exposure duration.

$P\left(t_{\text{E}}\right)\equiv \text{Pr}\left(I\left(\omega \right)=1\right)=\text{Pr}\left(t_{\text{c}}\left(\omega \right)<t_{\text{E}}\right)$ (2)

The probability of flight action, $P\left(t_{\text{E}}\right)$, is an increasing function of $t_{\text{E}}$. When $t_{\text{E}}$ is right at $m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\left(\omega \right)\right)$, we have ${P\left(t_{\text{E}}\right)|}_{t_{\text{E}}=m_{\text{c}}}=\mathrm{50\; <mi>\; \%\; </mi>}$ : it is equally likely for $t_{\text{E}}=m_{\text{c}}$ to induce or not induce flight action.

In Figure 2, we illustrate four representatives of possible realizations for an exposure test. They are not exhaustive. The binary outcome $I_j$ is determined by the relation between ${\left(t_{\text{c}}\right)}_j$ and ${\left(t_{\text{E}}\right)}_j$. Flight action occurs when the subjective threshold ${\left(t_{\text{c}}\right)}_j$ is below the prescribed exposure duration ${\left(t_{\text{E}}\right)}_j$. Since $t_{\text{c}}$ is a random variable, $\left({\left(t_{\text{c}}\right)}_j<{\left(t_{\text{E}}\right)}_j\right)$ is possible and uncertain for any given ${\left(t_{\text{E}}\right)}_j$ in the neighborhood of median subjective threshold $m_{\text{c}}$. A lower value of

Figure 2. Four representative realizations of an exposure test. The binary outcome (I) is determined by the relation between the subjective threshold ( $t_{\text{c}}$ ) and the prescribed exposure duration ( $t_{\text{E}}$ ). When flight action occurs, its actuation is at time $t_{\text{F}}=t_{\text{c}}+t_{\text{R}}$.

prescribed $t_{\text{E}}$ may induce flight action in some tests while a higher value of prescribed $t_{\text{E}}$ may fail to induce flight action in some other tests. Of course, the probability of $\left({\left(t_{\text{c}}\right)}_j<{\left(t_{\text{E}}\right)}_j\right)$ increases with the prescribed ${\left(t_{\text{E}}\right)}_j$. In the case of $\left({\left(t_{\text{c}}\right)}_j<{\left(t_{\text{E}}\right)}_j\right)$, flight action occurs and is irreversibly initiated at the time of subjective threshold ${\left(t_{\text{c}}\right)}_j$ ; the flight action is actuated/observed at time ${\left(t_{\text{F}}\right)}_j>{\left(t_{\text{c}}\right)}_j$ ; the time elapsed between the initiation and the actuation, ${\left(t_{\text{R}}\right)}_j={\left(t_{\text{F}}\right)}_j-{\left(t_{\text{c}}\right)}_j$, is the human reaction time.

2.3. The Time of Actuation/Observation of Flight Action

If flight action occurs, the time of its actuation/observation, $t_{\text{F}}\left(\omega \right)$, is the sum of $t_{\text{c}}\left(\omega \right)$ and $t_{\text{R}}=m_{\text{R}}$ (a deterministic quantity in the current study). The time of observed flight action $t_{\text{F}}\left(\omega \right)$ is defined only when $I\left(\omega \right)=1$.

$t_{\text{F}}\left(\omega \right)=\{\begin{array}{ll}{t}_{\text{c}}\left(\omega \right)+m_{\text{R}}\hfill & \text{if}I\left(\omega \right)=\text{1}\hfill \\ \text{undefined}\hfill & \text{if}I\left(\omega \right)=0\hfill \end{array}$ (3)

Note that the probability of $I\left(\omega \right)=1$ depends on the prescribed exposure duration $t_{\text{E}}$. Consequently, a description of conditional statistics of $t_{\text{F}}\left(\omega \right)$ is incomplete unless the value of $t_{\text{E}}$ is provided. For conciseness, in notations for the conditional statistics of $t_{\text{F}}\left(\omega \right)$, we shall omit $I\left(\omega \right)=1$ since the condition $I\left(\omega \right)=1$ is always implied when discussing $t_{\text{F}}\left(\omega \right)$. Instead we focus on specifying $t_{\text{E}}$. For example, we use the notation

$\underset{\text{shortnotation}}{\underset{︸}{{\rho }_{\left(t_{\text{F}}\mathrm{<mo>\; |\; </mo>}\eta \right)}\left(t\right)}}\equiv \underset{\text{fullnotation}}{\underset{︸}{{\rho }_{\left(t_{\text{F}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{E}}=\eta \mathrm{,}I\left(\omega \right)=1\right)}\left(t\right)}}={\rho }_{\left(t_{\text{c}}+m_{\text{R}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)={\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t-m_{\text{R}}\right)$ (4)

The conditional density ${\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)$ is heavily influenced by the prescribed exposure duration $\eta \equiv t_{\text{E}}$. Figure 3 compares ${\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)$ for two values of $\eta$. It is clear that the conditional density varies with $\eta$. In particular, $E\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)$ is

Figure 3. Schematic graphs of conditional density ${\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)$ for ${\eta }_1<m_{\text{c}}$ (bottom) and for ${\eta }_2>m_{\text{c}}$ (top). It is clear that $E\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)$ is an increasing function of $\eta$.

an increasing function of $\eta$, which demonstrates that a description of $t_{\text{F}}$ is incomplete unless $\eta$ is specified.

In test j, neither the subjective threshold for flight action ${\left(t_{\text{c}}\right)}_j$ nor the human reaction time $m_{\text{R}}$ is directly measurable. Instead, the observed binary outcome $I_j$ along with the prescribed exposure duration ${\eta }_j\equiv {\left(t_{\text{E}}\right)}_j$ provides a binary description of ${\left(t_{\text{c}}\right)}_j$.

$\text{binarydescriptionof}{\left(t_c\right)}_j:\{\begin{array}{ll}{\left(t_c\right)}_j<{\eta }_j\hfill & \text{if}{I}_j=1\hfill \\ {\left(t_c\right)}_j>{\eta }_j\hfill & \text{if}{I}_j=0\hfill \end{array}$ (5)

In the case of $I_j=1$ (flight action occurs), the time of observed flight action ${\left(t_{\text{F}}\right)}_j$ gives the sum of ${\left(t_{\text{c}}\right)}_j$ and $m_{\text{R}}$, both of which are unknown in our study.

$t_j\equiv {\left(t_{\text{F}}\right)}_j=\{\begin{array}{ll}{\left(t_{\text{c}}\right)}_j+m_{\text{R}}\hfill & \text{if}{I}_j=\text{1}\hfill \\ \text{undefined}\hfill & \text{if}{I}_j=\text{0}\hfill \end{array}$ (6)

In this study, we aim at inferring the statistics of subjective threshold $t_{\text{c}}$ and the human reaction time $m_{\text{R}}$, from data measured in the method of constant stimuli (MoCS). First, we infer $m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)$ from the data set of prescribed exposure durations and the corresponding binary outcomes $\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\right\}$, collected in a sequence of exposure tests. As described in our previous study [11], the exposure tests are designed sequentially and adaptively in a Bayesian framework, based on the test results already collected. The inference of $m_{\text{c}}$ is based on the maximum likelihood estimation (MLE) formulation using a Weibull inference model. We will study the robustness of inferring median subjective threshold $m_{\text{c}}$ when the underlying true model governing the data generation is different from the assumed Weibull inference model.

Once $m_{\text{c}}$ is obtained, we infer $m_{\text{R}}$ and $s_{\text{c}}$ from the data set of observed flight action times $\left\{\left({\eta }_j\mathrm{,}{t}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \text{with}{I}_j=1\right\}$. The inference of $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ is also based on the maximum likelihood estimation (MLE) formulation using a Weibull inference model. The primary objective is to infer the human reaction time $m_{\text{R}}$ whereas $s_{\text{c}}$ is an unknown variable in the inference formulation. Again, we will study the robustness of inferring $m_{\text{R}}$ when the underlying true model governing the data generation is different from the assumed Weibull inference model.

3. Inference Formulations and Methods

3.1. Data Measured in the Method of Constant Stimuli

In the method of constant stimuli, in testj, the exposure duration ${\eta }_j\equiv {\left(t_{\text{E}}\right)}_j$ is prescribed; after the exposure, the observed binary outcome $I_j$ regarding the occurrence of flight action is recorded. If $I_j=1$ (flight action occurs), the time of observed flight action $t_j\equiv {\left(t_{\text{F}}\right)}_j$ is stored in data; if $I_j=0$ (flight action does not occur), data entry $t_j$ is undefined (empty). The statistics of $t_{\text{F}}$ is meaningful only when conditional on $I=1$. The data set, measured in the method of constant stimuli, has the form

$\text{D}=\left\{\left({\eta }_j\mathrm{,}{I}_j\mathrm{,}{t}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}N\right\}\mathrm{,}{\eta }_j\equiv {\left(t_{\text{E}}\right)}_j\mathrm{,}{t}_j\equiv {\left(t_{\text{F}}\right)}_j$ (7)

where N is the number of exposure tests. Again, the data entry $t_j$ is defined only in those tests with $I_j=1$. Below we infer, from data set (7), the statistics of subjective threshold $t_{\text{c}}$ for flight action and infer the deterministic value of the human reaction time $m_{\text{R}}$. It is worthwhile to point out that neither ${\left(t_{\text{c}}\right)}_j$ nor $m_{\text{R}}$ is directly measured in the data. In test j, the observed binary outcome $I_j$ provides a binary description of sample ${\left(t_{\text{c}}\right)}_j$ in that particular test, as described in (5). In the case of $I_j=1$, the time of observed flight action $t_j\equiv {\left(t_{\text{F}}\right)}_j$ gives the sum of ${\left(t_{\text{c}}\right)}_j$ and $m_{\text{R}}$ in that particular test, as given in (6).

We infer $m_{\text{c}}$ and $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ in two steps as described in Figure 4. The inference of $m_{\text{c}}$ uses only the data set of binary outcomes $\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}N\right\}$ which is independent of the human reaction time $m_{\text{R}}$. The subsequent inference of

Figure 4. Two inference steps for estimating $m_{\text{c}}$ and $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$.

$\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ utilizes the estimated $m_{\text{c}}$ from step 1 and the data set of observed flight action times $\left\{\left({\eta }_j\mathrm{,}{t}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}N\text{with}{I}_j=1\right\}$.

3.2. Inference Model for t_c

In our inference formulation, we use the Weibull distribution to model $t_{\text{c}}\left(\omega \right)$, the random subjective threshold for flight action. The PDF and CDF of the Weibull distribution are

$t_{\text{c}}\left(\omega \right)~\text{Weibull}\left(k\mathrm{,}\lambda \right)$

${\rho }_W\left(t\right)=\frac{k}{\lambda }{\left(\frac{t}{\lambda }\right)}^{k-1}\exp \left[-{\left(\frac{t}{\lambda }\right)}^k\right]\mathrm{,}t>0$ (8)

$F_W\left(t\right)=1-\exp \left[-{\left(\frac{t}{\lambda }\right)}^k\right],t>0$ (9)

where k is the shape parameter and $\lambda$ the scale parameter in the Weibull distribution. It is important to point out that the underlying true distribution of $t_{\text{c}}\left(\omega \right)$ is unknown. We simply use the Weibull distribution as the inference model in our formulation. In Monte Carlo simulations, we will study the performance of this Weibull-based inference when the underlying true model of $t_{\text{c}}\left(\omega \right)$ deviates from the Weibull distribution.

The mean, median, and standard deviation of $t_{\text{c}}$ are

$E\left(t_{\text{c}}\right)=\lambda \Gamma \left(1+1/k\right)$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=\lambda {\left(\ln 2\right)}^{1/k}$ (10)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\lambda \sqrt{\Gamma \left(1+2/k\right)-\Gamma {\left(1+1/k\right)}^2}$ (11)

where $\Gamma \left(z\right)$ is the gamma function defined as

$\Gamma \left(s\right)={\displaystyle {\int }_0^{\infty }}{u}^{\left(s-1\right)}{\text{e}}^{-u}\text{d}u$ (12)

The mapping from $\left(\lambda \mathrm{,}k\right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is invertible.

$k=\frac{1}{S^{-1}\left(s_{\text{c}}^2/m_{\text{c}}^2\right)}\mathrm{,}S\left(u\right)\equiv \frac{\Gamma \left(1+2u\right)-\Gamma {\left(1+u\right)}^2}{{\left(\ln 2\right)}^{2u}}$

$\lambda =\frac{m_{\text{c}}}{{\left(\ln 2\right)}^{1/k}}$ (13)

Numerical computation shows that $S\left(u\right)$ defined in (13) is an increasing function of u and thus is invertible. The inverse mapping allows us to specify the Weibull distribution using the median and the standard deviation $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$. This parameterization is necessary when we test the inference performance on artificial data sets generated from 6 distributions (including the Weibull distribution) as the underlying true model of $t_{\text{c}}$. In numerical tests, data sets generated from all distributions are constrained by a given median and a given standard deviation.

The conditional density of $t_{\text{c}}$ given $t_{\text{c}}<\eta$ is

${\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)=\frac{1}{F_W\left(\eta \right)}{\rho }_W\left(t\right)\mathrm{,}\text{for}0<t<h\text{only}$ (14)

The conditional density of $t_{\text{F}}\equiv t_{\text{c}}+m_{\text{R}}$ given $t_{\text{c}}<\eta$ is

$\begin{array}{c}{\rho }_{\left(t_{\text{F}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\right)={\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t-m_{\text{R}}\right)\\ =\frac{1}{F_W\left(\eta \right)}{\rho }_W\left(t-m_{\text{R}}\right)\mathrm{,}\text{for}{m}_{\text{R}}<t<m_{\text{R}}+h\text{only}\end{array}$ (15)

The conditional density (15) provides a proper mathematical framework for connecting 1) the observed flight action time $t_j\equiv {\left(t_{\text{F}}\right)}_j$ in a test with flight action $I_j=1$, 2) the prescribed exposure duration ${\eta }_j$, and 3) the unknown deterministic human reaction time $m_{\text{R}}$. It serves as a key component for inferring $m_{\text{R}}$ from data $\left\{\left({\eta }_j\mathrm{,}{t}_j\right)\right\}$.

3.3. Inference of m_c

As illustrated in the top row of Figure 4, we use the method of maximum likelihood estimation (MLE) to infer $m_{\text{c}}$ from a date set $D_{\text{bin}}=\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}N\right\}$ where ${\eta }_j$ is the prescribed exposure duration and $I_j$ the binary outcome in test j. Under the Weibull inference model, given ${\eta }_j$, the probability of observing binary outcome $I_j$ is given by

$\text{Pr}\left(I\left(\omega \right)=I_j\right)=F_W{\left({\eta }_j\right)}^{I_j}{\left[1-F_W\left({\eta }_j\right)\right]}^{\left(1-I_j\right)}$ (16)

where $F_W\left(t\right)$ is the CDF of the Weibull distribution given in (9).

$F_W\left(t\right)$ contains two parameters: $\left(\lambda \mathrm{,}k\right)$. In the inference of $m_{\text{c}}$, we fix the shape parameter at $k=2.5$ and focus on estimating the scale parameter $\lambda$. As we discussed in our previous study [11], fixing k at a wrong value, especially at a smaller value (corresponding to a wider Weibull distribution), will not affect the inference accuracy of the median $m_{\text{c}}$.

To formulate the MLE method, we include inference variable $\lambda$ explicitly in the notation of Weibull CDF: $F_W\left(t\mathrm{<mo>\; ;\; </mo>}\lambda \right)$. Given data set $D_{\text{bin}}$ of binary observations from N exposure tests, the log-likelihood as a function of $\lambda$ is

$\mathcal{l}\left(\lambda \mathrm{<mo>\; ;\; </mo>}{D}_{\text{bin}}\right)={\displaystyle \underset{j=1}{\overset{N}{\sum }}}\left\{I_j\ln F_W\left({\eta }_j\mathrm{<mo>\; ;\; </mo>}\lambda \right)+\left(1-I_j\right)\ln \left[1-F_W\left({\eta }_j\mathrm{<mo>\; ;\; </mo>}\lambda \right)\right]\right\}$ (17)

The maximum likelihood estimate for $\lambda$ is

$\hat{\lambda }=\text{arg}\underset{\lambda }{\max }\mathcal{l}\left(\lambda \mathrm{<mo>\; ;\; </mo>}{D}_{\text{bin}}\right)$ (18)

Once $\hat{\lambda }$ is obtained, $m_{\text{c}}$ is calculated according to (10)

$m_{\text{c}}=\hat{\lambda }{\left(\ln 2\right)}^{1/k}\mathrm{,}k=2.5$ (19)

3.4. Inference of (m_R, s_c)

After $m_{\text{c}}$ is estimated, we use the method of maximum likelihood estimation (MLE) to infer $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$, the deterministic human reaction time and the standard deviation of $t_{\text{c}}$. The inference is based on a date set $D_{\text{time}}=\left\{\left({\eta }_j\mathrm{,}{t}_j\right)\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}N\text{with}{I}_j=1\right\}$ where ${\eta }_j$ is the prescribed exposure duration and $t_j$ the observed flight action time in those tests with $I_j=1$. The flowchart from the input of data set and $m_{\text{c}}$ to the inference output of $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ is illustrated in the bottom row of Figure 4.

We construct the formulation for inferring $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$. Since $m_{\text{c}}$ is now known (inferred in subsection 3.3), we write the PDF and CDF of the Weibull distribution as parameterized in $\left(m_{\text{c}}\mathrm{,}k\right)$ with k as the unknown parameter, instead of parameterized in $\left(\lambda \mathrm{,}k\right)$.

${\rho }_W\left(t\mathrm{<mo>\; ;\; </mo>}k\right)=\frac{\left(\ln 2\right)k}{m_{\text{c}}}{\left(\frac{t}{m_{\text{c}}}\right)}^{k-1}\exp \left[-\left(\ln 2\right){\left(\frac{t}{m_{\text{c}}}\right)}^k\right]\mathrm{,}t>0$ (20)

$F_W\left(t\mathrm{<mo>\; ;\; </mo>}k\right)=1-\exp \left[-\left(\ln 2\right){\left(\frac{t}{m_{\text{c}}}\right)}^k\right]\mathrm{,}t>0$ (21)

When the prescribed exposure duration is $\eta$, the time of observed flight action (in those tests with occurrence of flight action) has the conditional density given in (15). The conditional density depends on parameters $\left(m_{\text{R}}\mathrm{,}k\right)$, which are the inference variables in our formulation. For clarity, we include $\left(m_{\text{R}}\mathrm{,}k\right)$ explicitly in the notation of conditional density.

$\begin{array}{c}{\rho }_{\left(t_{\text{F}}\mathrm{<mo>\; |\; </mo>}\eta \right)}\left(t\mathrm{<mo>\; ;\; </mo>}\left(m_{\text{R}}\mathrm{,}k\right)\right)={\rho }_{\left(t_{\text{c}}+m_{\text{R}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t\mathrm{<mo>\; ;\; </mo>}\left(m_{\text{R}}\mathrm{,}k\right)\right)={\rho }_{\left(t_{\text{c}}\mathrm{<mo>\; |\; </mo>}{t}_{\text{c}}<\eta \right)}\left(t-m_{\text{R}}\mathrm{<mo>\; ;\; </mo>}k\right)\\ =\{\begin{array}{ll}\frac{{\rho }_W\left(t-m_{\text{R}}\mathrm{<mo>\; ;\; </mo>}k\right)}{F_W\left(\eta \mathrm{<mo>\; ;\; </mo>}k\right)}\mathrm{,}\hfill & \text{for}{m}_{\text{R}}<t<m_{\text{R}}+h\text{only}\hfill \\ \mathrm{0,}\hfill & \text{otherwise}\hfill \end{array}\end{array}$ (22)

Given data set $D_{\text{time}}$ of observed flight action times from those tests with the occurrence of flight action ( $I_j=1$ ), the log-likelihood as a function of $\left(m_{\text{R}}\mathrm{,}k\right)$ is

$\mathcal{l}\left(m_{\text{R}}\mathrm{,}k\mathrm{<mo>\; ;\; </mo>}{D}_{\text{time}}\right)=\{\begin{array}{ll}{\displaystyle \underset{I_j=1}{\sum }}\ln \frac{{\rho }_W\left(t_j-m_{\text{R}}\mathrm{<mo>\; ;\; </mo>}k\right)}{F_W\left({\eta }_j\mathrm{<mo>\; ;\; </mo>}k\right)}\mathrm{,}\hfill & \underset{I_j=1}{\max }\left(t_j-{\eta }_j\right)<m_{\text{R}}<\underset{I_j=1}{\min }\left(t_j\right)\hfill \\ -\infty \mathrm{,}\hfill & \text{otherwise}\hfill \end{array}$ (23)

The maximum likelihood estimate for $\left(m_{\text{R}}\mathrm{,}k\right)$ is

$\left({\hat{m}}_{\text{R}}\mathrm{,}\hat{k}\right)=\text{arg}\underset{\left(m_{\text{R}}\mathrm{,}k\right)}{\max }\mathcal{l}\left(m_{\text{R}}\mathrm{,}k\mathrm{<mo>\; ;\; </mo>}{D}_{\text{time}}\right)$ (24)

Once $\hat{k}$ is available, the standard deviation $s_{\text{c}}$ is calculated according to (11)

$s_{\text{c}}=\frac{m_{\text{c}}}{{\left(\ln 2\right)}^{1/\hat{k}}}\sqrt{\Gamma \left(1+2/\hat{k}\right)-\Gamma {\left(1+1/\hat{k}\right)}^2}$ (25)

In (23), the correct domain of the log-likelihood in variable $m_{\text{R}}$ is

$\underset{I_j=1}{\max }\left(t_j-{\eta }_j\right)<m_{\text{R}}<\underset{I_j=1}{\min }\left(t_j\right)$ (26)

which is affected by both the observed flight action time $t_j$ and the prescribed exposure duration ${\eta }_j$. The maximization of log-likelihood in (24) is subject to constraint (26). Note that constraint (26) is not naturally implied in the analytical expression of ${\rho }_W\left(t-m_{\text{R}}\mathrm{<mo>\; ;\; </mo>}k\right)$ given in (20). Therefore, in a correct numerical implementation of MLE, constraint (26) needs to be enforced explicitly.

It is worthwhile to point out that for a data set of N exposure tests, the summation in (23) contains less than N terms since not all tests of MoCS lead to occurrence of flight action. The fraction of tests with occurrence of flight action ( $I_j=1$ ) varies with the distribution of prescribed exposure duration $\left\{{\eta }_j\right\}$. In a sequence of tests designed using the adaptive Bayesian method [11] [17], $\eta$ is selected sequentially based on the current posterior to maximize the inference accuracy for $m_{\text{c}}$ given the total number of tests allocated. The sequence $\left\{{\eta }_j\right\}$ produced by the Bayesian method is a random walk of $\eta$ toward and then around the true value of median subjective threshold $m_{\text{c}}$. With this Bayesian experimental design, the rate of flight action occurrence is approximately 50% (i.e. about 50% of exposure tests lead to $I_j=1$ ).

4. Numerical Tests

4.1. Generating Artificial Data Sets

Recall that in our model described in section 2, the subjective threshold on exposure duration for flight action ( $t_{\text{c}}$ ) is a random variable while the human reaction time ( $m_{\text{R}}$ ) is a deterministic quantity. In each individual test based on the method of constant stimuli (MoSC), given the prescribed exposure duration ${\eta }_j$, the binary outcome regarding the occurrence of flight action ( $I_j$ ) and the time of observed flight action if it occurs ( $t_j$ ) are completely determined by the realized sample of ${\left(t_{\text{c}}\right)}_j$ in that particular test.

$I_j=\{\begin{array}{ll}1\hfill & \text{if}{\left(t_c\right)}_j<h_j\hfill \\ 0\hfill & \text{if}{\left(t_c\right)}_j>h_j\hfill \end{array}$ (27)

$t_j={\left(t_{\text{c}}\right)}_j+m_{\text{R}}\text{if}{I}_j=1$ (28)

An artificial data set $D=\left\{\left({\eta }_j\mathrm{,}{I}_j\mathrm{,}{t}_j\right)\right\}$ consisting of N exposure tests is generated by drawing N independent samples of random variable $t_{\text{c}}$.

In our numerical tests, we examine the performance of the inference method described in section 3 on artificial data sets generated from 6 distribution types for $t_{\text{c}}$.

· Weibull distribution

· Log-normal distribution

· Truncated normal distribution

· Gamma distribution

· Triangular distribution

· Uniform distribution

4.2. 6 Distribution Types for t_c Used in Data Generation

4.2.1. Weibull Distribution

The Weibull distribution has been described in subsection 3.2. In particular, we established that the mapping from $\left(\lambda \mathrm{,}k\right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is numerically invertible.

4.2.2. Log-Normal Distribution

The PDF and CDF of the log-normal distribution are

$t_{\text{c}}\left(\omega \right)~\text{Log-normal}\left(\mu \mathrm{,}{\sigma }^2\right)\mathrm{,}\ln \left(t_{\text{c}}\left(\omega \right)\right)~N\left(\mu \mathrm{,}{\sigma }^2\right)$ (29)

${\rho }_{t_{\text{c}}}\left(t\right)=\frac{1}{\sigma t}\varphi \left(\frac{\ln t-\mu }{\sigma }\right),t>0$ (30)

$F_{t_{\text{c}}}\left(t\right)=\Phi \left(\frac{\ln t-\mu }{\sigma }\right),t>0$ (31)

where $\mu$ and ${\sigma }^2$ are respectively the mean and the variance of $\ln \left(t_{\text{c}}\left(\omega \right)\right)$. Functions $\varphi \left(s\right)$ and $\Phi \left(s\right)$ are the PDF and CDF of the standard normal distribution.

$\varphi \mathrm{<mo\; stretchy="false">\; (\; </mo>}\left(s\right)=\frac{1}{\sqrt{2\pi }}{\text{e}}^{-s^2/2}$ (32)

$\Phi \left(s\right)={\displaystyle {\int }_{-\infty }^s}\varphi \left(v\right)\text{d}v=\frac{1}{2}\left(\text{erf}\left(\frac{s}{\sqrt{2}}\right)+1\right)$ (33)

The standard normal CDF $\Phi \left(s\right)$ is expressed in terms of the error function $\text{erf}\left(s\right)$ defined as

$\text{erf}\left(s\right)\equiv \frac{2}{\sqrt{\pi }}{\displaystyle {\int }_0^s}{\text{e}}^{-u^2}\text{d}u$

The mean, median, and standard deviation of $t_{\text{c}}$ are

$E\left(t_{\text{c}}\right)=\exp \left(\mu +\frac{{\sigma }^2}{2}\right)$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=\exp \left(\mu \right)$ (34)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\exp \left(\mu +\frac{{\sigma }^2}{2}\right)\sqrt{\exp \left({\sigma }^2\right)-1}$ (35)

The mapping from $\left(\mu \mathrm{,}\sigma \right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is invertible. The inverse mapping is

$\mu =\ln (\; m\; c\; )$

$\sigma =\sqrt{\ln \left(1+\sqrt{1+4{\left(s_{\text{c}}/m_{\text{c}}\right)}^2}\right)-\ln (\; 2\; )}$

4.2.3. Truncated Normal Distribution

The PDF and CDF of the truncated normal distribution are

$t_{\text{c}}\left(\omega \right)~N_{\in \left(\mathrm{0,}+\infty \right)}\left(\mu \mathrm{,}{\sigma }^2\right)$ (36)

${\rho }_{t_{\text{c}}}\left(t\right)=\frac{1}{Z\sigma }\varphi \left(\frac{t-\mu }{\sigma }\right)I_{\left(\mathrm{0,}+\infty \right)}\left(t\right)$ (37)

$F_{t_{\text{c}}}\left(t\right)=\frac{1}{Z}\left[\Phi \left(\frac{t-\mu }{\sigma }\right)-\Phi \left(\alpha \right)\right]I_{\left(\mathrm{0,}+\infty \right)}\left(t\right)$ (38)

where $\mu$ and ${\sigma }^2$ are respectively the mean and the variance of the untruncated normal distribution. $\varphi \left(s\right)$ and $\Phi \left(s\right)$ are the standard normal PDF and CDF given in (32) and (33). $I_{\left(\mathrm{0,}+\infty \right)}\left(t\right)$ is the indicator function of interval $\left(\text{0,}+\infty \right)$ defined as

$I_{\left(\mathrm{0,}+\infty \right)}\left(t\right)\equiv \{\begin{array}{ll}1\hfill & \text{if}t\in \left(\text{0,}+\infty \right)\hfill \\ 0\hfill & \text{otherwise}\hfill \end{array}$ (39)

Quantities Z and $\alpha$ are defined in terms of $\mu$ and $\sigma$ as

$Z\equiv 1-\Phi \left(\alpha \right)\mathrm{,}\alpha \equiv \frac{-\mu }{\sigma }$ (40)

The mean, median, and standard deviation of $t_{\text{c}}$ are

$E\left(t_{\text{c}}\right)=\mu +\sigma \frac{\varphi \left(\alpha \right)}{Z}\mathrm{,}$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=\mu +\sigma {\Phi }^{-1}\left(\frac{\Phi \left(\alpha \right)+1}{2}\right)$ (41)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\sigma \sqrt{1+\frac{\alpha \varphi \left(\alpha \right)}{Z}-{\left(\frac{\varphi \left(\alpha \right)}{Z}\right)}^2}$ (42)

where ${\Phi }^{-1}\left(p\right)$ is the inverse function of $\Phi \left(s\right)$, and from (33), it has the expression

${\Phi }^{-1}\left(p\right)=\sqrt{2}\text{erfinv}\left(2p-1\right)$ (43)

The mapping from $\left(\mu \mathrm{,}\sigma \right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is numerically invertible.

4.2.4. Gamma Distribution

The PDF and CDF of the gamma distribution are

$t_{\text{c}}\left(\omega \right)~\text{gamma}\left(k\mathrm{,}\lambda \right)$ (44)

${\rho }_{t_{\text{c}}}\left(t\right)=\frac{1}{\Gamma \left(k\right)\lambda }{\left(\frac{t}{\lambda }\right)}^{k-1}\exp \left(\frac{-t}{\lambda }\right)\mathrm{,}t>0$ (45)

$F_{t_{\text{c}}}\left(t\right)=\frac{1}{\Gamma \left(k\right)}\gamma \left(k\mathrm{,}\frac{t}{\lambda }\right)\mathrm{,}t>0$ (46)

where k is the shape parameter and $\lambda$ the scale parameter of the gamma distribution. $\gamma \left(k\mathrm{,}s\right)$ is the lower incomplete gamma function defined as

$\gamma \left(k\mathrm{,}s\right)\equiv {\displaystyle {\int }_0^s}{u}^{k-1}{\text{e}}^{-u}\text{d}u$ (47)

The mean, median, and standard deviation of $t_{\text{c}}$ are

$E\left(t_{\text{c}}\right)=k\lambda \mathrm{,}$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=\lambda {\gamma }^{-1}\left(k\mathrm{,}\frac{1}{2\Gamma \left(k\right)}\right)$ (48)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\sqrt{k}\lambda \mathrm{,}$ (49)

where ${\gamma }^{-1}\left(k\mathrm{,}z\right)$ is the inverse function of $\gamma \left(k\mathrm{,}s\right)$ with respect to variable s. It satisfies

$\gamma \left(k\mathrm{,}{\gamma }^{-1}\left(k\mathrm{,}z\right)\right)=z\mathrm{,}{\gamma }^{-1}\left(k\mathrm{,}\gamma \left(k\mathrm{,}s\right)\right)=s$ (50)

The mapping from $\left(\lambda \mathrm{,}k\right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is numerically invertible.

4.2.5. Triangular Distribution

The PDF and CDF of the triangular distribution are

$t_{\text{c}}\left(\omega \right)~\text{triangular}\left(a\mathrm{,}b\mathrm{,}c\right)$ (51)

${\rho }_{t_{\text{c}}}\left(t\right)=\frac{2\left(x-a\right)}{\left(b-a\right)\left(c-a\right)}{I}_{\left[a,c\right]}\left(t\right)+\frac{2\left(b-x\right)}{\left(b-a\right)\left(b-c\right)}{I}_{\left(c,b\right]}\left(t\right)$ (52)

$F_{t_{\text{c}}}\left(t\right)=\frac{{\left(x-a\right)}^2}{\left(b-a\right)\left(c-a\right)}{I}_{\left[a,c\right]}\left(t\right)+\left(1-\frac{{\left(b-x\right)}^2}{\left(b-a\right)\left(b-c\right)}\right)I_{\left(c,b\right]}\left(t\right)+I_{\left(b,+\infty \right)}\left(t\right)$ (53)

where a is the lower limit, $b\in \left(a\mathrm{,}+\infty \right)$ the upper limit, and $c\in \left(a\mathrm{,}b\right)$ the mode of the triangular distribution. We consider the case of $c\le \left(a+b\right)/2$ (i.e., the distribution is symmetric or right-skewed). The mean, median, and standard deviation of $t_{\text{c}}$ have the expressions below.

$E\left(t_{\text{c}}\right)=\frac{1}{3}\left(a+b+c\right)\mathrm{,}$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=b-\sqrt{\frac{\left(b-a\right)\left(b-c\right)}{2}}$ (54)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\frac{1}{3\sqrt{2}}\sqrt{a^2+b^2+c^2-ab-ac-bc}\mathrm{,}$ (55)

The general triangular distribution has 3 parameters $a\le c\le b$. In order to map $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ back to parameters, we reduce the number of parameters to 2. We select $\left(b\mathrm{,}c\right)$ as independent parameters and set parameter a as a function of $\left(b\mathrm{,}c\right)$ by enforcing a symmetric triangular distribution or a right-skewed distribution subject to constraint $a\ge 0$.

$a\left(b\mathrm{,}c\right)\mathrm{<mo>\; =\; </mo>}\{\begin{array}{ll}\left(2c-b\right)\hfill & \text{if}\left(2c-b\right)\ge \text{0}\hfill \\ 0\hfill & \text{otherwise}\hfill \end{array}$

In this subclass of triangular distributions parameterized by $\left(b\mathrm{,}c\right)$, the mapping from $\left(b\mathrm{,}c\right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is numerically invertible.

4.2.6. Uniform Distribution

The PDF and CDF of the uniform distribution are

$t_{\text{c}}\left(\omega \right)~\text{uniform}\left(a\mathrm{,}b\right)$ (56)

${\rho }_{t_{\text{c}}}\left(t\right)=\frac{1}{b-a}{I}_{\left[a,b\right]}\left(t\right)$ (57)

$F_{t_{\text{c}}}\left(t\right)=\left(\frac{t-a}{b-a}\right)I_{\left[a,b\right]}\left(t\right)+I_{\left(b,+\infty \right)}\left(t\right)$ (58)

where a is the lower limit and $b\in \left(a\mathrm{,}+\infty \right)$ the upper limit of the uniform distribution. The mean, median, and standard deviation of $t_{\text{c}}$ are

$E\left(t_{\text{c}}\right)=\frac{1}{2}\left(a+b\right)\mathrm{,}$

$m_{\text{c}}\equiv \text{median}\left(t_{\text{c}}\right)=\frac{1}{2}\left(a+b\right)$ (59)

$s_{\text{c}}\equiv \text{std}\left(t_{\text{c}}\right)=\frac{1}{2\sqrt{3}}\left(b-a\right)$ (60)

The mapping from $\left(a\mathrm{,}b\right)$ to $\left(m_{\text{c}}\mathrm{,}{s}_{\text{c}}\right)$ is invertible. The inverse mapping is

$a=m_{\text{c}}-\sqrt{3}{s}_{\text{c}}\mathrm{,}$

$b=m_{\text{c}}+\sqrt{3}{s}_{\text{c}}$

For $s_{\text{c}}/m_{\text{c}}>1/\sqrt{3}$, however, the inverse mapping yields a uniform distribution with $a<0$, which is invalid for describing $t_{\text{c}}$ since the exposure duration and the subjective threshold on the exposure duration are always positive.

4.2.7. Graphs of the 6 Distribution Types

In our numerical simulations below, we test the inference method on data sets generated from the 6 distribution types described above. In each distribution type, we explore 4 cases of different relative distribution widths as measured by $s_{\text{c}}/m_{\text{c}}$. Since the problem is invariant up to a constant scaling, in the data generation, we fix $m_{\text{c}}=2$ and examine the cases of $s_{\text{c}}=0.25$, 0.5, 1, and 1.5. In the case of $s_{\text{c}}=1.5$, we use only 5 distributions, excluding the uniform distribution since it gives an invalid distribution protruding to $t<0$. The graphs of PDF ${\rho }_{t_{\text{c}}}$ are shown in the 6 panels of Figure 5, one panel for each of the 6 distribution types. Each panel contains 4 graphs of ${\rho }_{t_{\text{c}}}$, one for each value of $s_{\text{c}}$, except the bottom right panel (the uniform distribution), which contains only 3 graphs.

4.3. Monte Carlo Simulations and Results

4.3.1. Simulation Parameters and Procedures

We generate artificial data sets from the 6 distribution types listed above. In data generation, the true non-dimensional values of $m_{\text{c}}$, $s_{\text{c}}$ and $m_{\text{R}}$ are set to

$m_{\text{c}}^{\left(\text{e}\right)}=\mathrm{2,}{s}_{\text{c}}^{\left(\text{e}\right)}=\left\{\mathrm{1.5,1,0.5,0.25}\right\}\mathrm{,}{m}_{\text{R}}^{\left(\text{e}\right)}=1$

This set of non-dimensional values is consistent with the physical situation where the subjective threshold is $t_{\text{c}}~0.4\text{s}$, the human reaction time $m_{\text{R}}~0.2\text{s}$ and the time scale for non-dimensionalization $t_{\text{scale}}~0.2\text{s}$. It is important to emphasize that these true values are used only in data generation and in evaluating the performance of inference method (by comparing the inferred values to these true values). These true values are unknown in the inference process, which uses only the artificial data sets generated.

As described in our previous study [11], a set of N exposure tests is designed sequentially using the adaptive Bayesian method. Specifically, the exposure duration for the next test ${\eta }_j$ is calculated based on the current posterior of the unknown $m_{\text{c}}$ (which is viewed as a random variable in the Bayesian framework).

Figure 5. Six distribution types combined with four values of $s_{\text{c}}$ yield 23 distributions for $t_{\text{c}}$. $s_{\text{c}}=1.5$ is invalid for the uniform distribution. $m_{\text{c}}=2$ is fixed in all distributions.

Once ${\eta }_j$ is determined and the true distribution for $t_{\text{c}}$ is selected (out of the 6 distributions), we sample random variable $t_{\text{c}}$ and generate the binary outcome and the observed flight action time $\left(I_j\mathrm{,}{t}_j\right)$ according to (27) and (28).

The inference of $m_{\text{c}}$ and $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ is carried out in two steps (see the schematic illustration in Figure 4). In the inference of $m_{\text{c}}$, only the data set of binary outcomes $\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\right\}$ is used. The subsequent inference of $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ utilizes the data set of observed flight action times $\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\right\}$ as well as the already estimated $m_{\text{c}}$. Each Monte Carlo run consists of generating one data set of N tests and carrying out inference on the data set to produce one sample of each inferred parameter. Histograms, scatter plots and root-mean-square (RMS) errors of the inferred values are calculated based on $M=1000$ Monte Carlo repeats.

4.3.2. Inference Results of m_c

$m_{\text{c}}$ is inferred from data of binary observations $\left\{\left({\eta }_j\mathrm{,}{I}_j\right)\right\}$.

We first exhibit in two figures the histograms of inferred $m_{\text{c}}$ using data sets generated from the 6 distributions with $s_{\text{c}}^{\left(\text{e}\right)}=1$. Figure 6 shows the results on data from i) Weibull (top row), ii) log-normal (middle row) and iii) truncated normal distribution (bottom row); Figure 7 displays the results on data from iv) gamma (top row), v) triangular (middle row) and vi) uniform distribution (bottom row). Each histogram is based on $M=1000$ Monte Carlo repeats (i.e., based on 1000 samples of inferred $m_{\text{c}}$, one sample from each Monte Carlo repeat). In all situations, the inference formulation is based on the Weibull distribution as described in section 3. In Figure 6 and Figure 7, the left column is $N=80$ (number of exposure tests in the data set of each Monte Carlo repeat), the right column $N=320$. A comparison of the two columns indicates that the inferred $m_{\text{c}}$ converges to the true value as the sample size is increased.

Next, we examine the RMS error of the inferred $m_{\text{c}}$, defined below and calculated in simulations based on $M=1000$ Monte Carlo repeats.

${\text{err}}_{m_{\text{c}}}\equiv \sqrt{\frac{1}{M}{\displaystyle \underset{i=1}{\overset{M}{\sum }}}{\left({\left({\hat{m}}_{\text{c}}\right)}_i-m_{\text{c}}^{\left(\text{e}\right)}\right)}^2}$ (61)

The RMS errors of inferred $m_{\text{c}}$ vs N are plotted in the 6 panel of Figure 8. Each panel is labeled with the distribution used in data generation, and contains 4 curves corresponding to the 4 values of $s_{\text{c}}^{\left(\text{e}\right)}$ used in data generation. For all data sets, the RMS error decreases as the sample size N is increased, and at a fixed N, the RMS error is higher on a data set generated using a larger value of $s_{\text{c}}^{\left(\text{e}\right)}$. That is, when the uncertainty in subjective threshold $t_{\text{c}}$ is larger in the underlying data, the inference error is higher.

In Figure 9, we compare directly the RMS errors of inferred $m_{\text{c}}$ vs N on data from the 6 distributions. The inference errors on data from the uniform distribution are somewhat larger than those from other 5 distributions. This is expected since the uniform distribution is the farthest from the inference model (see graphs of PDFs in Figure 5). Figure 8 and Figure 9 demonstrate that the inference of median subjective threshold $m_{\text{c}}$ based on the Weibull distribution converges to the correct value even when the underlying true model of the data deviates from the inference model.

Figure 6. Histograms of inferred $m_{\text{c}}$ on data from 1) Weibull (top row), 2) log-normal (middle row) and 3) truncated normal distribution (bottom row). Left column: $N=80$ ; right column: $N=320$. Each histogram is based on 1000 Monte Carlo repeats.

Figure 7. Histograms of inferred $m_{\text{c}}$ on data from 4) gamma (top row), 5) triangular (middle row) and vi) uniform distribution (bottom row). Left column: $N=80$ ; right column: $N=320$. Each histogram is based on 1000 Monte Carlo repeats.

Figure 8. The RMS errors of inferred $m_{\text{c}}$ vs N on data generated from the 6 distributions (one panel for each distribution) with the 4 values of $s_{\text{c}}^{\left(\text{e}\right)}$.

Figure 9. Comparison of RMS errors in inferred $m_{\text{c}}$ vs N on data from the 6 distributions. Left: results on data with $s_{\text{c}}^{\left(\text{e}\right)}=1$ ; right: $s_{\text{c}}^{\left(\text{e}\right)}=0.5$.

4.3.3. Inference Results of (m_R, s_c)

$\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ is inferred from data of observed flight action times $\left\{\left({\eta }_j\mathrm{,}{t}_j\right)\text{with}{I}_j=1\right\}$. We explore the scatter plots of inferred $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ using data sets generated from the 6 distributions with $s_{\text{c}}^{\left(\text{e}\right)}=1$. Figure 10 presents the results on data from 3 of the 6 distributions: 1) Weibull (top row), 2) log-normal (middle row) and 6) truncated normal distribution (bottom row); Figure 11 displays the results on data from the other 3 distributions: 4) gamma (top row), 5) triangular (middle row) and 6) uniform distribution (bottom row).

Each scatter plot is based on $M=1000$ Monte Carlo repeats. In all situations, the inference formulation is based on the Weibull distribution as described in section 3. In Figure 10 and Figure 11, the left column is $N=80$, the right column $N=320$. Note that N is the number of exposure tests in the data set of each Monte Carlo repeat. In the Bayesian experimental design intended for optimizing the inference of $m_{\text{c}}$, about half of tests produce no flight action. Only those tests with the occurrence of flight action are relevant in the inference of $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$. As a result, the effective sample size for inferring $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ is about N/2. A comparison of the two columns indicates that as the sample size is increased, the inferred $m_{\text{R}}$ (the human reaction time) converges to the true value while the inferred $s_{\text{c}}$ (standard deviation of the subjective threshold) converges to an incorrect value. For example, for data from the log-normal distribution, the inferred $s_{\text{c}}$ converges to a value lower than the true $s_{\text{c}}^{\left(\text{e}\right)}=1$.

To investigate the convergence of inferred $m_{\text{R}}$, we plot RMS errors of inferred $m_{\text{R}}$ vs N in the 6 panels of Figure 12. Each panel is labeled with the distribution used in data generation, and contains 4 curves corresponding to the 4 values of $s_{\text{c}}^{\left(\text{e}\right)}$ used in data generation. For all data sets, the RMS error of inferred $m_{\text{R}}$ decreases as the sample size N is increased, and at a fixed N, the RMS error of inferred $m_{\text{R}}$ is higher on a data set generated using a larger value of $s_{\text{c}}^{\left(\text{e}\right)}$. That is, when the uncertainty in subjective threshold $t_{\text{c}}$ is larger in the

Figure 10. Scatter plots of inferred $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ on data from 1) Weibull (top row), 2) log-normal (middle row) and 3) truncated normal distribution (bottom row). Left column: $N=80$ ; right column: $N=320$. Each scatter plot is based on 1000 Monte Carlo repeats.

Figure 11. Scatter plots of inferred $\left(m_{\text{R}}\mathrm{,}{s}_{\text{c}}\right)$ on data from 4) gamma (top row), 5) triangular (middle row) and 6) uniform distribution (bottom row). Left column: $N=80$ ; right column: $N=320$. Each scatter plot is based on 1000 Monte Carlo repeats.

Figure 12. The RMS errors of inferred $m_{\text{R}}$ vs N on data generated from the 6 distributions (one panel for each distribution) with the 4 values of $s_{\text{c}}^{\left(\text{e}\right)}$.

Figure 13. Comparison of RMS errors in inferred $m_{\text{R}}$ vs N on data from the 6 distributions. Left: results on data with $s_{\text{c}}^{\left(\text{e}\right)}=1$ ; right: $s_{\text{c}}^{\left(\text{e}\right)}=0.5$.

underlying data, the inference error of human reaction time $m_{\text{R}}$ is higher.

In Figure 13, we compare directly the RMS errors of inferred $m_{\text{R}}$ vs N on data from the 6 distributions. The inference errors are comparable on data from all 6 distributions, with data from the uniform distribution yielding slightly larger inference errors. Figure 12 and Figure 13 demonstrate that the inference of human reaction time $m_{\text{R}}$ based on the Weibull distribution converges to the correct value even when the underlying true model of the data deviates from the inference model.

5. Concluding Remarks

In this study, we consider the psychophysical experiments in which a test subject is exposed to a stimulus source for a prescribed duration and the test subject’s response is binary. For example, the occurrence of flight action is binary when a subject is exposed to a millimeter wave beam for a prescribed duration. The binary outcome is determined by the prescribed exposure duration and a subjective threshold of the test subject on exposure duration. To include the biovariability, we view the subjective threshold as a random variable. In an exposure test, if the realized sample value of subjective threshold is lower than the prescribed duration, flight action is initiated at the time of subjective threshold, which is not measurable. The actuation of flight action is observed at a later time, which is recorded in experiments. The delay from the initiation time to the actuation time of flight action is the human reaction time, which is not measurable. Here, we view the human reaction time as an unknown deterministic quantity. The case of a random variable human reaction time will be investigated in a subsequent study. We consider the experiments based on the method of constant stimuli (MoCS) in which the exposure duration is prescribed before each test. This is different from the experiments based on the method of limits (MoL) in which the beam stimulus is kept on until the observed actuation of flight action. In a test of MoL, flight action always occurs; in the observed actuation time of flight action, the subjective threshold and the human reaction are tangled together, making it difficult to infer either one. In contrast, in a test based on MoCS, flight action may or may not occur; the observed binary outcome is independent of human reaction time and it provides a binary description of the realized sample of subjective threshold in that test, making it possible to estimate the median subjective threshold from a sequence of properly designed tests.

In our modeling framework, the human reaction time and the distribution of random subjective threshold are unknown. The primary objective of the current study is to estimate the human reaction time from data of prescribed exposure durations, binary outcomes and actuation times of flight action (if it occurs), measured in a sequence of tests. We model the random subjective threshold $t_{\text{c}}$ as a Weibull distribution. We estimate the human reaction time $m_{\text{R}}$ from data in two inference steps. In step one, we use only the data of binary outcomes, which excludes the effect of human reaction time. We fix the shape parameter of Weibull distribution at $k=2.5$ and use the maximum likelihood estimation (MLE) to infer the median subjective threshold $m_{\text{c}}$ from data of binary outcomes. In method of constant stimuli, a critical part of experimental design is to specify the exposure duration in each test. In our previous study [11], we used an adaptive Bayesian method to select the exposure duration for the next test based on the current posterior of the unknown median $m_{\text{c}}$. This experimental design produces the optimal inference on the median subjective threshold $m_{\text{c}}$ and has been used in real exposure tests. In the current study, we adopt this experimental design. Once $m_{\text{c}}$ is estimated, in step two of the inference, we estimate the human reaction time $m_{\text{R}}$ from data of observed actuation times of flight action. We parameterize the Weibull distribution by the known median $m_{\text{c}}$ and the unknown standard deviation $s_{\text{c}}$ ; we use the maximum likelihood estimation (MLE) to infer the two unknowns: $m_{\text{R}}$ and $s_{\text{c}}$, both of which affect the distribution of observed actuation times of flight action.

We run Monte Carlo simulations to test the performance of the proposed inference method. Recall that the inference formulation is based on the Weibull distribution. To explore its robustness, we test the inference method on data generated from 6 different distribution types as the underlying true model of random subjective threshold. To assess the effect of randomness of the subjective threshold, we test the inference method on data generated with 4 values of relative distribution width of subjective threshold: $s_{\text{c}}/m_{\text{c}}=\left\{\mathrm{0.125,0.25,0.5,0.75}\right\}$ in each model. In each case, to study the convergence of the inference, we run the inference method on data with sample size ranging from $N=80$ to $N=2560$. For each set of (model distribution, simulation parameters), we carry out 1000 Monte Carlo repeats to generate 1000 samples of each estimated parameter. The convergence is examined by plotting the RMS inference error vs sample size (N). Based on the results of extensive numerical tests, we draw several comments, observations and conclusions on the inference method.

· The inference of median subjective threshold ( $m_{\text{c}}$ ) uses only the data of binary outcomes in the method of constant stimuli. The inferred $m_{\text{c}}$ converges to the correct value as the sample size increases, even when the underlying true model of random subjective threshold deviates from the inference model (Weibull distribution).

· The inference of human reaction time ( $m_{\text{R}}$ ) utilizes the data of observed actuation times of flight action (if it occurs). The inferred $m_{\text{R}}$ converges to the correct value as the sample size increases, even when the underlying true model of random subjective threshold deviates from the inference model (Weibull distribution).

· The standard deviation of subjective threshold ( $s_{\text{c}}$ ) is a by-product in the inference of $m_{\text{R}}$. However, when the underlying true model of random subjective threshold deviates from the inference model, the inferred $s_{\text{c}}$ converges to an incorrect value.

· The robustness of inferring both the median subjective threshold $m_{\text{c}}$ and the human reaction time $m_{\text{R}}$ makes the inference method applicable to real experimental data in which the underlying true model of the random subject threshold is unknown. In contrast, the inference of standard deviation of subjective threshold ( $s_{\text{c}}$ ) depends on knowing the underlying true model, which makes it impractical.

· The inference errors of $m_{\text{c}}$ and $m_{\text{R}}$ decrease as the standard deviation of subjective threshold $s_{\text{c}}$ is reduced. That is, for a hypothetical system with a lower uncertainty in the subjective threshold, the inference errors would be smaller.

The inference discussed above utilizes the data collected in a sequence of exposure tests designed with the objective of optimizing the inference of only the median subjective threshold $m_{\text{c}}$. We extend the usage of this same data set to infer the human reaction time $m_{\text{R}}$ in a two-step procedure. This approach is practical and operationally important since data sets of this type already exist from real experiments. Now suppose we revise the experimental design with the objective of optimizing the inference of both $m_{\text{c}}$ and $m_{\text{R}}$. Will the resulting sequence of tests be significantly different from the current one in terms of the distribution of prescribed exposure durations? Will the new experimental design notably improve the inference accuracy of $m_{\text{c}}$ and $m_{\text{R}}$ over the current one? Will the inference accuracy improve if we infer $m_{\text{c}}$ and $m_{\text{R}}$ simultaneously instead of in a two-step procedure? These questions will be explored in a future study.

Acknowledgements and Disclaimer

The authors acknowledge the Joint Intermediate Force Capabilities Office of U.S. Department of Defense and the Naval Postgraduate School for supporting this work. The views expressed in this document are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Kuroda, T. and Hasuo, E. (2014) The Very First Step to Start Psychophysical Experiments. Acoustical Science and Technology, 35, 1-9. https://doi.org/10.1250/ast.35.1
[2]	Waskom, M., Okazawa, G. and Kiani, R. (2019) Designing and Interpreting Psychophysical Investigations of Cognition. Neuron, 104, 100-112. https://doi.org/10.1016/j.neuron.2019.09.016
[3]	Parker, J.E., Nelson, E.J. and Beason, C.W. (2017) Thermal and Behavioral Effects of Exposure to 30-kW, 95-GHz Millimeter Wave Energy. Technical Report, AFRL-RH-FS-TR-2017-0016.
[4]	Parker, J.E., Nelson, E.J., Beason, C.W. and Cook, M.C. (2017) Effects of Variable Spot Size on Human Exposure to 95-GHz Millimeter Wave Energy. Technical Report, AFRL-RH-FS-TR-2017-0017.
[5]	Cazares, S.M., Snyder, J.A., Belanich, J., Biddle, J.C., Buytendyk, A.M., Teng, S.H.M. and O’Connor, K. (2019) Active Denial Technology Computational Human Effects End-to-End Hypermodel for Effectiveness (ADT CHEETEH-E). Human Factors and Mechanical Engineering for Defense and Safety, 3, Article No. 13. https://doi.org/10.1007/s41314-019-0023-7
[6]	Zhadobov, M., Chahat, N., Sauleau, R., Le Quement, C. and Le Drean, Y. (2011) Millimeter-Wave Interactions with the Human Body: State of Knowledge and Recent Advances. International Journal of Microwave and Wireless Technologies, 3, 237-247. https://doi.org/10.1017/S1759078711000122
[7]	Walters, T.J., Blick, D.W., Johnson, L.R., Adair, E.R. and Foster, K.R. (2000) Heating and Pain Sensation Produced in Human Skin by Millimeter Waves: Comparison to a Simple Thermal Model. Health Physics, 78, 259-267. https://doi.org/10.1097/00004032-200003000-00003
[8]	Douglass, D.K., Carstens, E. and Watkins, L.R. (1992) Spatial Summation in Human Thermal Pain Perception: Comparison within and between Dermatomes. Pain, 50, 197-202. https://doi.org/10.1016/0304-3959(92)90161-4
[9]	Defrin, R. and Urca, G. (1996) Spatial Summation of Heat Pain: A Reassessment. Pain, 66, 23-29. https://doi.org/10.1016/0304-3959(96)02991-0
[10]	Wang, H., Burgei, W.A. and Zhou, H. (2020) A Concise Model and Analysis for Heat-Induced Withdrawal Reflex Caused by Millimeter Wave Radiation. American Journal of Operations Research, 10, 31-81. https://doi.org/10.4236/ajor.2020.102004
[11]	Wang, H., Adamzadeh, M., Burgei, W.A., Foley, S.E. and Zhou, H. (2021) Inference of Median Subjective Threshold in Psychophysical Experiments. Journal of Applied Mathematics and Physics, 9, 982-1002. https://doi.org/10.4236/jamp.2021.95068
[12]	Campbell, J.N. and LaMotte, R.H. (1983) Latency to Detection of First Pain. Brain Research, 266, 203-208. https://doi.org/10.1016/0006-8993(83)90650-9
[13]	Welford, A.T. (1980) Choice Reaction Time: Basic Concepts. In: Welford, A.T., Ed., Reaction Times, Academic Press, New York, 73-128.
[14]	Herrick, R.M. (1967) Psychophysical Methodology: VI. Random Method of Limits. Perceptual & Motor Skills, 24, 915-922. https://doi.org/10.2466/pms.1967.24.3.915
[15]	Herrick, R.M. (1973) Psychophysical Methodology: Comparison of Thresholds of the Method of Limits and of the Method of Constant Stimuli. Perception & Psychophysics, 13, 548-554. https://doi.org/10.3758/BF03205818
[16]	(2009) Method of Constant Stimuli. In: Binder, M.D., Hirokawa, N. and Windhorst, U., Eds., Encyclopedia of Neuroscience, Springer, Berlin, 2347.
[17]	Watson, A.B. and Pelli, D.G. (1983) Quest: A Bayesian Adaptive Psychometric Method. Perception & Psychophysics, 33, 113-120. https://doi.org/10.3758/BF03202828
[18]	Wang, H., Burgei, W.A. and Zhou, H. (2020) Non-Dimensional Analysis of Thermal Effect on Skin Exposure to an Electromagnetic Beam. American Journal of Operations Research, 10, 147-162. https://doi.org/10.4236/ajor.2020.105011
[19]	Wang, H., Burgei, W.A. and Zhou, H. (2020) Analytical Solution of One-Dimensional Pennes’ Bioheat Equation. Open Physics, 18, 1084-1092. https://doi.org/10.1515/phys-2020-0197