A New Maximum Test via the Dependent Samples t-Test and the Wilcoxon Signed-Ranks Test

A maximum test in lieu of forcing a choice between the two dependent samples t-test and Wilcoxon signed-ranks test is proposed. The maximum test, which requires a new table of critical values, maintains nominal α while guaranteeing the maximum power of the two constituent tests. Critical values, obtained via Monte Carlo methods, are uniformly smaller than the Bonferroni-Dunn adjustment, giving it power superiority when testing for treatment alternatives of shift in location parameter when data are sampled from non-normal distributions.


Introduction
The process of selection of a test can be complicated, confusing, and in some cases disappointing.In choosing a particular test, consideration must be given to its robustness properties with respect to Type I errors for departures from population normality and that it has favorable comparative statistical power [1].When parametric conditions are violated one test or another may be found to be more powerful than its competitor analogue under a given set of circumstances [1].Therefore, the choice is often left to intuitive selection or guesswork [2].
In the context of two samples when testing for a treatment modeled as a shift in location parameter, both the parametric t-test and nonparametric Wilcoxon signed-ranks (WSR) test are possibilities.Under normality, the t-test is the uniformly most powerful unbiased test.However, that distinction is lost when data are sampled from non-normal distributions [3].Blair and Higgins [3] found no instance where the t-test held more than a modest power advantage over the Wilcoxon test, whereas the latter held a clear power advantage, as much as 0.895, over the t-test for data obtained from skewed distributions.See also references [4][5][6][7][8].
Although they disagreed with the practice, Sawilowsky and Fahoome [9] noted transforming data to fit parametric assumptions which is commonly recommended when confronting with nonnormally distributed data.This practice (a) suffers from not knowing a priori which transformation is to use, and (b) conducts the hypothesis test on a metric that is often irrelevant to the research context [9].Other solutions, such as conducting both the t-test and the Wilcoxon signed-ranks test and accepting the latter if it is significant, or conducting a preliminary test for normality prior to a test of effects, merely serve to increase the Experiment-Wise Type I error rate [9].

Maximum Test
Consider the two dependent samples layout.A resolution may be obtained in the form of a maximum test, based on taking the more significant result of the dependent sample t (DT) and Wilcoxon signed-ranks (WSR) tests.In order to avoid inflation of Type I errors, the larger of the two obtained critical values must then be compared with the appropriate critical value obtained from the joint sampling distribution of the two tests.
A maximum test has useful diagnostic properties with extreme component statistics that is sensitive to any statistical departures from normal null hypothesis [2].The advantage of the maximum test was "the fact that the test is automatically adaptive to the weight in the tail of the population from which the data were sampled" [2, p. 17].Type I error rates are not inflated, because the critical value of a maximum test is a single test based on the joint sampling distribution of the component tests.Other important advantages of the maximum test are (1) avoidance of the task to choose one test over another or others without running the risk of power losses and (2) avoidance of the task to conduct post hoc alpha adjustments such as the Bonferroni-Dunn.

Statement of the Problem
The three-fold purpose of this study is to 1) create a maximum test using the parametric dependent samples t-test and the non-parametric Wilcoxon sign rank test 2) obtain critical values via Monte Carlo methods using sample deviates obtained randomly and with replacement from a contaminated (mixed) normal distribution.Critical values will be derived for nominal alpha levels of 0.05, 0.025, 0.01 and 0.005 for sample sizes 8 -30, 45, 60, 90, and 120, and 3) to demonstrate the critical values compare favorably to the Bonferroni-Dunn adjustment.

Model
Algina, Blair and Coombs [10, p. 28] defined the maximum test as a statistic "for a particular data set, two or more statistics and test the same hypothesis and selecting as the test statistic the one with the smallest p value" and in the event that "each statistic has the same critical value the maximum statistic is simply the most extreme of the calculated statistics".
Cox [2, p. 50] described a significance test as a procedure for measuring the consistency of data with a null hypothesis H o having the form where "an observed vector, y, of response variables, or sometimes written as y obs , and null hypothesis H o according to which y is the observed value of a random variable Y, with sampling space S y , and having a probability density where p obs is the observed value of a random variable.Cox [2] provided the following form of the a maximum test which Cox [2] credited to Tippet [11], ( ) where p j is the significance level in the j th test and small values of q are evidence against H o .The required level of significance and the allowance for selection was noted as ( ( ) If component tests are independent and continuous (3) becomes , and that an upper bound for (3) is in any case kq obs .
Hence the maximum test via the dependent sample t-test and the Wilcoxon signed-rank test was defined as ; ; Both the DT and WSR were computed in the same metric as the DT.The variable t WSR refers to the probability of the acquired Z from the WSR test that was converted to an obtained t associated with 1 2 2 n n + − degrees of freedom.In the event scores are tied WSR DT t t = then either DT or the WSR was used.

Assumptions
The maximum test does not create any new assumptions.Although the critical values are derived from a specific mixed normal distribution, they are useful for general mixed normal distributions and other non-normal shapes.

Method
Monte Carlo simulation methods were used to obtain critical values.A FORTRAN program employing various subroutines of the International Mathematical and Statistical Libraries [12] was developed in order to create the test and to obtain critical values.Deviates were randomly sampled from a contaminated (mixed normal) distribution based on two populations which differ in respective means and/or variances.It was formed by sampling with a probability of 0.95 from a normal distribution with mean of 0 and a standard deviation of 1, and with a probability of 0.05 from a normal distribution with a mean equal to 22 and a standard deviation of 10.The contaminated (mixed normal) distribution was chosen for this study because is familiar to many readers, it's commonly used in robustness studies and they are important population models across a variety of disciplines [13].This maximum test could generally be used for any model of mixed normal populations, and to a lesser extent any non-normal model.
Critical values were obtained as follows.Random deviates were assigned to two groups.Both the dependent samples t-test and the Wilcoxon signed-ranks test were computed.Subsequently, the probability of the obtained Z from the Wilcoxon test was converted to an obtained t that would be associated with the degrees of freedom using the IMSL tin(p, df) subroutine [12].Then, the two obtained t values were compared, and whichever was the higher was recorded.This process was repeated 200,000 times and the results were stored in an array, which was subsequently sorted from low to high.Then, the value corresponding to the percentile associated with the desired alpha level was selected.For example, the value at the 95 th percentile represents the critical value for the nominal alpha = 0.05 level.

Critical Values
Critical values were selected from the ordered array to represent values at the 0.05, 0.025, 0.01 and 0.005 significance levels.This process was repeated for sample sizes n = 8 through 30, 45, 60, 90 and 120.The critical values are presented in Table 1.
As expected, the tabled critical values moved inversely with sample size, meaning that as the sample size increased the tabled values decreased.In some instances, critical values reversed direction, and then returned to the descending pattern as n increased.In a few instances, the critical values repeated at different samples sizes and alpha levels.Both anomalies are attributed to the computational nature of the maximum test, and were previously noted by other workers on the maximum test [9,14].Hence, although these two phenomena appear antithetical, they are expected and are not disconcerting.

Inspection of Maximum Test Critical Values and Comparison with Bonferroni-Dunn Critical Values
As noted in Tables 2 and 3 on the following page, the critical values for the maximum test were systematically larger than critical values for the t-test.This behavior controls the inflation of Type I errors.
The fallback plan in avoiding Type I error inflation is to use a Bonferroni-Dunn adjustment when conducting multiple statistical tests.Bonferroni-Dunn attempts to control the probability of rejecting at least one true hypothesis at some specified level a by testing each of the hypotheses of interest at level of significance α.It is used "when conducting multiple tests of significance to set an upper bound on the overall significance level α" [15].Simes [15, p. 751]  ."Usually, the adjustment is too conservative and lacks requisite power to reject an individual hypothesis as the number of tests increases, thereby having the effect of missing real differences [16].
Critical values for the maximum test are compared with values obtained via Bonferroni-Dunn adjustment, NT α , where NT refers to the number of tests being conducted.Note that in Tables 2 and 3 above, the maximum test's critical values are systematically lower than the Bonferroni-Dunn test, and hence, it will be a more powerful test [17].See [17, p. 259] on comparing statistical tests at specific α levels when one test cannot be conducted at that precise level due to the test's discrete sampling distribution.

Example of Use of the Maximum Test
Suppose a test of difference in average performance of a treatment versus a control group was conducted with n 1 = n 2 = 20, for a two-sided test with α = 0.05.The first step is to conduct both the dependent t-test and Wilcoxon signed-ranks test.The second step is to select whichever obtained statistic is higher in magnitude (i.e., select the statistic whose absolute value is greater).The third step is to enter Table 1 with n = 20, α = 0.025, and retrieve the critical value of ±2.189228.If the obtained maximum statistic is either greater than 2.189228, or if it is less than −2.189228, reject the null hypothesis in favor of the alternative.

Discussion and Conclusions
The maximum test's critical values were systematically lower than the Bonferroni-Dunn test and therefore the more powerful test, and were systematically larger than critical values for the t-test which controls the inflation of Experiment-Wise Type I errors when conducting both the dependent samples t and Wilcoxon signed-ranks test.The maximum test eliminates the need to make a forced choice between the dependent sample t-test and the WSR test when the distribution from which samples are drawn remain unknown, or are known to be non-normally distributed.The test permits the safe application of both the classical and non-parametric tests with the maximum of the two referred to the new table of critical values that are designed to maintain the Type I error rate to nominal α while guaranteeing the maximum power of the two tests.The maximum test also renders the Bonferroni-Dunn adjustment method unnecessary.
Thus the observed level of significance and the allowance for selection is, ( ) Y f y in some family H o ."The function ( ) t t y = of the observations, or a test statistic.The corresponding random variable is represented by T.