Abstract

We present here an alternative definition of the P-value for statistical hypothesis test of a real-valued parameter for a continuous random variable X. Our approach uses neither the notion of Type I error nor the assumption that null hypothesis is true. Instead, the new P-value involves the maximum likelihood estimator, which is usually available for a parameter such as the mean μ or standard deviation σ of a random variable X with a common distribution.

Keywords

Hypothesis Testing, P-Value, Evidence, Statistics

Share and Cite:

1. Introduction

A principal goal of statistics is to obtain evidence from data for comparing alternative decisions. For example, statistical evidence may allow one to decide that a population mean $\mu$ satisfies $\mu \le {\mu }_0$ as opposed to $\mu >{\mu }_0$ for some specified ${\mu }_0$ . Unfortunately, evidence is an ambiguous concept in statistics, though [1]-[7] among others have attempted to define it. Currently, the P-value [8] [9] is perhaps the most frequently applied measure of evidence used to reject or fail to reject a hypothesis. In Section 2, we review the standard P-value and the notion of a maximum likelihood estimator (MLE). We then propose the new MLE P-value involving the MLE of some parameter of interest. The MLE P-value is not related to significance levels and not defined under the assumption that the null hypothesis is true. Both the standard and new MLE P-values may be considered measures of evidence, but the latter is more intuitive. In Section 3, we present four examples and compare the two approaches.

2. The Standard and MLE P-Values

The notion of P-value is a fundamental tool in statistical inference and has been widely used for reporting outcomes of hypothesis tests. Yet in practice, the P-value is often misinterpreted, misused, or miscommunicated. Moreover, it does not unequivocally reflect the available evidence for the null hypothesis since H₀ is assumed to hold. In this section, we propose the new MLE P-value that may give different values in some cases than do existing definitions. The MLE P-value provides a simple and intuitive interpretation of P-value. Our definition appears applicable to a wide range of hypothesis testing problems and yields an interpretation of P-value as both a cardinal and ordinal measure of the evidence. We restrict our development to standard one-sided hypothesis testing, but Example 3 illustrates that the approach here can also be used for two-sided hypothesis testing.

We first summarize the two standard ways of defining P-value for the general hypothesis test $H_0:\theta \in {\Theta }_0$ vs $H_1:\theta \notin {\Theta }_0$ with a parameter space ${\Theta }_0$ . Let $T\left(X\right)$ be a test statistic for a random sample $X=\left(X_1,\cdots ,X_n\right)$ from a random variable $X$ with a single scalar parameter $\theta$ . Denote the observed data for $X$ as $x=\left(x_1,\cdots ,x_n\right)$ .

Definition 1 (Definitions of P-value from [10] and [11]). Under the assumption that $H_0:\theta \in {\Theta }_0$ is true, the P-value associated with the observed data $x$ is defined as either

$\text{PV}1\left(x|{\Theta }_0\right)=\underset{\theta \in {\Theta }_0}{\sup }P\left\{T\left(X\right)\ge T\left(x\right)|\theta \right\}$ (1)

or

$\text{PV}2\left(x|{\Theta }_0\right)=\inf \left\{\alpha :T\left(x\right)\in R_{\alpha }\right\},$ (2)

where $R_{\alpha }$ is the rejection region for a level of significance $\alpha$ .

Equation (1) is usually interpreted as follows. Under the assumption that H₀ is true, P-value is the probability that $T\left(X\right)$ is as least as extreme as its observed value $T\left(x\right)$ . This interpretation can lead to the common misunderstanding that this definition of P-value is the probability that H₀ is true. On the other hand, under the assumption that H₀ is true, Equation (2) is based on significance levels (Type I Error probabilities) and can lead to the misunderstanding that P-value is only a measure of Type I Error and not related to the likelihood that H₀ is true.

Both definitions lead to the question: how can the assumption that H₀ is true produce evidence that H₀ is true? The answer is that H₀ is rejected when there is a probability less than or equal to 0.05 (for example) that H₀ is true when it is assumed true. In other words, a contradiction involving probability is reached. For other issues with these definitions, see [10]-[12], for example. To address such issues, we utilize the well-known maximum likelihood estimator (MLE) [13] [14] and define a P-value utilizing the maximum likelihood estimator for the parameter under consideration.

Definition 2 (Likelihood Function and MLE). Let $x=\left(x_1,\cdots ,x_n\right)$ be sample data from a random sample $X=\left(X_1,\cdots ,X_n\right)$ from a random variable $X$ with sample space $S$ and real-valued parameter $\theta$ . For the joint pdf $f\left(x|\theta \right)$ of the random sample $X$ . For any sample data $x$ , the likelihood function of $\theta$ is defined as

$L\left(\theta |x\right)=f\left(x|\theta \right),$ (3)

where $L\left(\theta |x\right)$ in (3) is a function of the variable $\theta$ for given data $x$ . The MLE $\widehat{\theta }\left(x\right)$ maximizes $L\left(\theta |x\right)$ for the data sample $x$ ; i.e., $\widehat{\theta }\left(x\right)=\underset{ \theta }{\arg \max }L\left(\theta |x\right)$ in terms of $x$ .

Definition 3 (MLE P-value). Let $x=\left(x_1,\cdots ,x_n\right)$ be observed random sample data for a random sample $X=\left(X_1,\cdots ,X_n\right)$ from a continuous random variable $X$ with a single real-valued parameter $\theta$ and pdf $f_X\left(x|\theta \right)$ . Let $Y=\widehat{\theta }\left(X\right)$ denote the MLE for $\theta$ with pdf $f_Y\left(y|\theta \right)$ . For the hypothesis test $H_0:\theta \in {\Theta }_0$ vs $H_1:\theta \notin {\Theta }_0$ , the MLE P-value (MPV) at the sample data $x$ for the null hypothesis $H_0:\theta \in {\Theta }_0$ is defined as

$\text{MPV}\left(x|{\Theta }_0\right)={\displaystyle {\int }_{{\Theta }_0}{f}_Y}\left(y|\theta =\widehat{\theta }\left(x\right)\right)\text{d}y,$(4)

where the integration is over the scalar values $y\in {\Theta }_0$ of $Y=\widehat{\theta }\left(X\right)$ .

In the integration of (4), $\theta$ is estimated by the number MLE $\widehat{\theta }\left(x\right)$ from the sample data $x$ . Then the pdf $f_Y\left(y|\theta =\widehat{\theta }\left(x\right)\right)$ of the MLE $Y=\widehat{\theta }\left(X\right)$ is integrated over the values $\theta$ in the null hypothesis $H_0:\theta \in {\Theta }_0$ . For example, in the one-tailed hypothesis test $H_0:\theta \le {\theta }_0$ vs $H_1:\theta >{\theta }_0$ , the set ${\Theta }_0$ would be$\left(-\infty ,{\theta }_0\right]$ . In this case (4), $H_0:\theta \in {\Theta }_0$ approximates the frequentist probability that $Y=\widehat{\theta }\left(X\right)\in {\Theta }_0$ , and so $\text{MPV}\left(x|{\Theta }_0\right)$ approximates the likelihood that is true. Any inaccuracies are due (i) to integrating over $Y=\widehat{\theta }\left(X\right)$ since $\theta$ has no prior or posterior distribution and (ii) to setting $\theta =\widehat{\theta }\left(x\right)$ . Both result from using the MLE $\widehat{\theta }\left(x\right)$ as a surrogate for the unknown parameter $\theta$ . Doing so, however, utilizes the fact that distributions of MLEs for a parameter $\theta$ are often known for a continuous random variable $X$ with a common distribution such as the normal or exponential. In such cases, (4) may be analytically integrable. If not, a numerical integration could possibly be employed. However, only analytical integration is considered here.

3. Examples

Four examples are now presented to illustrate the MLE P-value approach. The first two involve the normal random variable, and the third involves an exponential random variable. The fourth demonstrates a possible limitation of the method.

Example 1. For a random sample of size $n$ , consider the hypothesis test $H_0:\mu \le {\mu }_0$ vs $H_1:\mu >{\mu }_0$ for the parameter $\mu$ of a normal random variable $X~N\left(\mu ,{\sigma }^2\right)$ . In this case, we rewrite $\text{MPV}\left(x|{\Theta }_0\right)$ as $\text{MPV}\left(x|{\mu }_0\right)$ . We also show that (4) gives the standard P-value for $\theta =\mu$ .

Result 1. Let $X_1,\cdots ,X_n$ be a random sample from a random variable $X~N\left(\mu ,{\sigma }^2\right)$ with unknown $\mu$ , and consider the one-sided hypothesis test $H_0:\mu \le {\mu }_0$ vs $H_1:\mu >{\mu }_0$ . Let $\overline{x}$ be the sample mean for sample values $x_1,\cdots ,x_n$ . When ${\sigma }^2$ is known,

$\text{MPV}\left(x|{\mu }_0\right)=\varphi \left(\frac{{\mu }_0-\overline{x}}{\sigma /\sqrt{n}}\right);$(5)

and when ${\sigma }^2$ is unknown,

$\text{MPV}\left(x|{\mu }_0\right)=F_{n-1}\left(\frac{{\mu }_0-\overline{x}}{s/\sqrt{n}}\right),$(6)

where $\varphi \left(z\right)$ is the cdf for the standard normal distribution and $F_{n-1}\left(t\right)$ is the cdf for the student t distribution with n − 1 degrees of freedom.

Proof. When ${\sigma }^2$ is known, to prove (5) we use the fact [9] that the MLE $\widehat{\mu }\left(X\right)$ for $\mu$ is the sample mean$\overline{X}~N\left(\mu ,{\sigma }^2/n\right)$ . The integral of (4) therefore becomes

$P\left[\overline{X}\le {\mu }_0\right]=\varphi \left(\frac{{\mu }_0-\mu }{\sigma /\sqrt{n}}\right).$ (7)

Substituting $\overline{x}$ for $\mu$ in (7) gives (5). When ${\sigma }^2$ is unknown, Equation (6) follows from the definition of the student t distribution [9]. ■

Numerically, when $n=9$ , ${\mu }_0=12$ , $\sigma =2$ , and $\overline{x}=13$ , the MLE P-value of (5) for $H_0:\mu \le {\mu }_0$ vs $H_1:\mu >{\mu }_0$ is $\varphi \left(\frac{12-13}{2/3}\right)\simeq 0.067$ from the standard normal table [9]. If $n=25$ instead, the MLE P-value of (5) is $\varphi \left(\frac{12-13}{2/5}\right)\simeq 0.006$ , and it is much less likely that $\mu \le {\mu }_0$ . It should be noted that the right sides of (5) and (6) are the standard P-values for the hypothesis test $H_0:\mu \le {\mu }_0$ vs $H_1:\mu >{\mu }_0$ using the definition of (2). In particular, the usual test statistic for this case is $z=\frac{\overline{x}-{\mu }_0 }{\sigma /\sqrt{n}}$ with a critical region of $z>z_{\alpha }$ [9], where α is the level of significance, i.e., the probability of committing a Type I error. The P-value is then the lowest level of α for which the observed z is significant. But this lowest level of α in standard hypothesis testing is simply $1-\varphi \left(\frac{\overline{x}-{\mu }_0 }{\sigma /\sqrt{n}}\right)=\varphi \left(\frac{{\mu }_0-\overline{x}}{\sigma /\sqrt{n}}\right)$ , which is the right side of (5).

Example 2. For a random sample of size $n$ , consider the hypothesis test $H_0:{\sigma }^2\le {\sigma }_0^2$ vs $H_1:{\sigma }^2>{\sigma }_0^2$ for the variance ${\sigma }^2$ of a normal random variable $X~N\left(\mu ,{\sigma }^2\right)$ . From [9], the MLE for the variance of a normal distribution is ${\widehat{\sigma }}^2\left(X\right)=\frac{{{\displaystyle \sum }}_{i=1}^n{\left(X_i-\overline{X}\right)}^2}{n}$ , so $\frac{n{\widehat{\sigma }}^2\left(X\right)}{{\sigma }^2}~{\chi }^2\left(n-1\right)$ , from which we have the following.

Result 2. Let $X_1,\cdots ,X_n$ be a random sample from a random variable $X~N\left(\mu ,{\sigma }^2\right)$ with unknown ${\sigma }^2$ , and consider the one-sided hypothesis test $H_0:{\sigma }^2\le {\sigma }_0^2$ vs $H_1:{\sigma }^2>{\sigma }_0^2$ . Then the integral of (4) becomes

$\text{NPV}\left(x|{\sigma }_0^2\right)=F_{{\chi }^2\left(n-1\right)}\left(\frac{n{\sigma }_0^2}{{\widehat{\sigma }}^2\left(x\right)}\right).$(8)

Proof. From [9], the MLE ${\widehat{\sigma }}^2\left(X\right)$ for ${\sigma }^2$ of a normal distribution is $\frac{{{\displaystyle \sum }}_{i=1}^n{\left(X_i-\overline{X}\right)}^2}{n}=\frac{\left(n-1\right)S^2}{n}$ and so $\frac{n{\widehat{\sigma }}^2\left(X\right)}{{\sigma }^2}~{\chi }^2\left(n-1\right)$ . Hence

$P\left[{\widehat{\sigma }}^2\left(X\right)\le {\sigma }_0^2\right]=P\left[\frac{n{\widehat{\sigma }}^2\left(X\right)}{{\sigma }^2}\le \frac{n{\sigma }_0^2}{{\sigma }^2}\right]=F_{{\chi }^2\left(n-1\right)}\left(\frac{n{\sigma }_0^2}{{\sigma }^2}\right).$ (9)

Substituting ${\widehat{\sigma }}^2\left(x\right)$ for ${\sigma }^2$ in $F_{{\chi }^2\left(n-1\right)}\left(\frac{n{\sigma }_0^2}{{\sigma }^2}\right)$ in (9) yields (8). ■

Numerically, when $n=10$ , ${\sigma }_0^2=4$ , and $s^2=9$ , the MLE P-value of (8) for $H_0:{\sigma }^2\le {\sigma }_0^2$ vs $H_1:{\sigma }^2>{\sigma }_0^2$ becomes $F_{{\chi }^2\left(9\right)}\left(4.94\right)\simeq 0.16$ .

Example 3. In this example, we illustrate that the MLE approach is applicable in two-sided hypothesis testing. For a random sample of size $n$ , consider the hypothesis test $H_0:\mu ={\mu }_0$ vs $H_1:\mu \ne {\mu }_0$ for the parameter $\mu$ of $X~N\left(\mu ,{\sigma }^2\right)$ with $\sigma$ known. To be more realistic, we modify this test to $H_0:\mu \in \left[{\mu }_0-\delta ,{\mu }_0+\delta \right]$ vs $H_1:\mu \notin \left[{\mu }_0-\delta ,{\mu }_0+\delta \right]$ , where $\delta$ is an acceptable tolerance level, i.e., an acceptable deviation from ${\mu }_0$ . In this case, (4) becomes

$\text{MPV}\left(x|{\mu }_0\right)=\varphi \left(\frac{{\mu }_0+\delta -\overline{x}}{\sigma /\sqrt{n}}\right)-\varphi \left(\frac{{\mu }_0-\delta -\overline{x}}{\sigma /\sqrt{n}}\right).$(10)

Numerically, when $n=25$ , ${\mu }_0=12$ , ${\sigma }^2=16$ , $\overline{x}=12.3$ , and $\delta =0.5$ , the MLE P-value of (10) becomes $\varphi \left(0.25\right)-\varphi \left(-1.00\right)\simeq 0.4400$ .

Example 4. Let the random variable $X$ have an exponential distribution with pdf $f_X\left(x|\lambda \right)=\lambda {\text{e}}^{-\lambda x}$ , $x>0$ , with parameter $\lambda >0$ . For a random sample of size $n$ , consider the hypothesis test $H_0:\lambda \le {\lambda }_0$ vs $H_1:\lambda >{\lambda }_0$ . From [15], the MLE for $\lambda$ is $Y=1/\overline{X}$ . It can be shown [16] that $\overline{X}$ for the exponential random variable $X$ follows the gamma distribution $\Gamma \left(n,n\lambda \right)$ . Hence $Y$ follows an inverse gamma distribution [16], so the right side of (4) becomes the regularized gamma function $Q\left(n,\frac{n}{{\lambda }_0\overline{X}}\right)$ , which is available in various software packages such as MATLAB.

Example 5. To illustrate the difficulty of finding the pdf $f_Y\left(y|\theta =\widehat{\theta }\left(x\right)\right)$ for MLE$Y=\widehat{\theta }\left(X\right)$ in the integral of (4), consider the random variable $X$ with pdf $f_X\left(x|\theta \right)=\theta x^{-\left(\theta +1\right)}$ for $x>1$ and the parameter $\theta >0$ . Then it can be shown [9] that $\widehat{\theta }\left(x\right)=\frac{n}{{{\displaystyle \sum }}_{i=1}^n\ln \left(X_i\right)}$ , whose pdf is difficult to determine.

4. Conclusions

The MLE P-value defined here gives a value that estimates the probability that a one-sided null hypothesis on a single parameter is true. Two-sided hypotheses can be similarly treated using tolerance levels as in Example 3. In the approach of this paper, the MLE $Y=\widehat{\theta }\left(X\right)$ may be thought of as a new type of test statistic that is integrated over ${\Theta }_0$ , while its numerical value $\widehat{\theta }\left(x\right)$ from the data $x$ is used to estimate $\theta$ in this integration. Obtaining such an approximation of the probability that the null hypothesis is true is, in fact, the ultimate goal of hypothesis testing. Certainty is not possible. But given the MLE P-value, a decision maker would need to decide if this value is sufficiently large to accept the null hypothesis. Different decision makers might judge a given MLE P-value probability differently, but a metric for the decision has now been provided.

The principal limitation of the analytical approach here is that the pdf of the maximum likelihood estimator needs to be both known and reasonable to integrate. Future work should be directed at the numerical integration of these integrals. Tables could be developed for certain parameters of particular random variables $X.$ Although MLEs have been used here as the surrogate for single parameters, other estimators could also be used. The advantage of MLEs is that they are well-studied and often immediately available. Finally, an approach similar to that of this paper could be applied to hypothesis testing for parameters of discrete random variables.

Acknowledgements

The author thanks his former student Maryham Moghimi for her discussion about statistics and the P-value.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

References