Modified p-Value of Two-Sided Test for Normal Distribution with Restricted Parameter Space

This article proposes a modified p-value for the two-sided test of the location of the normal distribution when the parameter space is restricted. A commonly used test for the two-sided test of the normal distribution is the uniformly most powerful unbiased (UMPU) test, which is also the likelihood ratio test. The p-value of the test is used as evidence against the null hypothesis. Note that the usual p-value does not depend on the parameter space but only on the observation and the assumption of the null hypothesis. When the parameter space is known to be restricted, the usual p-value cannot sufficiently utilize this information to make a more accurate decision. In this paper, a modified p-value (also called the rp-value) dependent on the parameter space is proposed, and the test derived from the modified p-value is also shown to be the UMPU test.     

Asymptotics of goodness-of-fit tests based on minimum p-value statistics

This paper provides some new results on the asymptotics of goodness-of-fit (GOF) tests based on minimum p-value statistics. In connection with detectability of sparse signals in high-dimensional data, various tests were proposed and investigated during the last decade, especially with respect to asymptotic properties. Minimum p-value GOF statistics were already investigated as minimum level attained statistics by Berk and Jones with respect to Bahadur efficiency. The distribution of minimum p-value GOF statistics is closely related to the distribution of higher criticism statistics, the distribution of the supremum of a normalized Brownian bridge, and the supremum of an Ornstein–Uhlenbeck process.     

The p-value Function and Statistical Inference

ABSTRACT

This article has two objectives. The first and narrower is to formalize the p-value function, which records all possible p-values, each corresponding to a value for whatever the scalar parameter of interest is for the problem at hand, and to show how this p-value function directly provides full inference information for any corresponding user or scientist. The p-value function provides familiar inference objects: significance levels, confidence intervals, critical values for fixed-level tests, and the power function at all values of the parameter of interest. It thus gives an immediate accurate and visual summary of inference information for the parameter of interest. We show that the p-value function of the key scalar interest parameter records the statistical position of the observed data relative to that parameter, and we then describe an accurate approximation to that p-value function which is readily constructed.     

A note on multiple testing for composite null hypotheses

Multiple hypothesis testing literature has recently experienced a growing development with particular attention to the control of the false discovery rate (FDR) based on p-values. While these are not the only methods to deal with multiplicity, inference with small samples and large sets of hypotheses depends on the specific choice of the p-value used to control the FDR in the presence of nuisance parameters. In this paper we propose to use the partial posterior predictive p-value [Bayarri, M.J., Berger, J.O., 2000. p-values for composite null models. J. Amer. Statist. Assoc. 95, 1127–1142] that overcomes this difficulty. This choice is motivated by theoretical considerations and examples. Finally, an application to a controlled microarray experiment is presented.     

Fuzzy p-value in testing fuzzy hypotheses with crisp data

In testing statistical hypotheses, as in other statistical problems, we may be confronted with fuzzy concepts. This paper deals with the problem of testing hypotheses, when the hypotheses are fuzzy and the data are crisp. We first introduce the notion of fuzzy p-value, by applying the extension principle and then present an approach for testing fuzzy hypotheses by comparing a fuzzy p-value and a fuzzy significance level, based on a comparison of two fuzzy sets. Numerical examples are also provided to illustrate the approach.     

A new generalized p-value approach for testing equality of coefficients of variation in k normal populations

A new generalized p-value method is proposed for testing the equality of coefficients of variation in k normal populations. Simulation studies show that the type I error probabilities are close to the nominal level. The proposed test is also compared with likelihood ratio test, modified Bennett's test and score test through Monte Carlo simulation, the results demonstrate that the generalized p-value method has satisfactory performance in terms of sizes and powers.     

The Minimaxity of the Mid P-value under Linear and Squared Loss Functions

The mid-p is defined as the sum of the probabilities of all outcomes more extreme than an observed value, plus half of the probabilities of all outcomes exactly as extreme. On the one hand, it offers greater power than the standard p-value, but on the other, tests based on the mid-p statistic may have greater Type I error than their nominal level. This article investigates the mid p-value's properties under the estimated truth paradigm, which views p-values as estimators of the truth. The mid-p is shown to minimize the maximum risk for one-sided and two-sided tests.     

Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests

Sequential designs can be used to save computation time in implementing Monte Carlo hypothesis tests. The motivation is to stop resampling if the early resamples provide enough information on the significance of the p-value of the original Monte Carlo test. In this paper, we consider a sequential design called the B-value design proposed by Lan and Wittes and construct the sequential design bounding the resampling risk, the probability that the accept/reject decision is different from the decision from complete enumeration. For the B-value design whose exact implementation can be done by using the algorithm proposed in Fay, Kim and Hachey, we first compare the expected resample size for different designs with comparable resampling risk. We show that the B-value design has considerable savings in expected resample size compared to a fixed resample or simple curtailed design, and comparable expected resample size to the iterative push out design of Fay and Follmann. The B-value design is more practical than the iterative push out design in that it is tractable even for small values of resampling risk, which was a challenge with the iterative push out design. We also propose an approximate B-value design that can be constructed without using a specially developed software and provides analytic insights on the choice of parameter values in constructing the exact B-value design.     

Assessing occupational exposure via the one-way random effects model with unbalanced data

This article considers a one-way random effects model for assessing the proportion of workers whose mean exposures exceed the occupational exposure limit based on exposure measurements from a random sample of workers. Hypothesis testing and interval estimation for the relevant parameter of interest are proposed when the exposure data are unbalanced. The methods are based on the generalized p-value approach, and simplify to the ones in Krishnamoorthy and Mathews (J. Agri. Biol. Environ. Statist. 7 (2002) 440) when the data are balanced. The sizes and powers of the test are evaluated numerically. The numerical studies show that the proposed inferential procedures are satisfactory even for small samples. The results are illustrated using practical examples.     

The adaptive calibration of testing p-values

Abstract

In statistical hypothesis testing, a p-value is expected to be distributed as the uniform distribution on the interval (0, 1) under the null hypothesis. However, some p-values, such as the generalized p-value and the posterior predictive p-value, cannot be assured of this property. In this paper, we propose an adaptive p-value calibration approach, and show that the calibrated p-value is asymptotically distributed as the uniform distribution. For Behrens–Fisher problem and goodness-of-fit test under a normal model, the calibrated p-values are constructed and their behavior is evaluated numerically. Simulations show that the calibrated p-values are superior than original ones.     

A p-value for testing the equivalence of the variances of a bivariate normal distribution

A p-value is developed for testing the equivalence of the variances of a bivariate normal distribution. The unknown correlation coefficient is a nuisance parameter in the problem. If the correlation is known, the proposed p-value provides an exact test. For large samples, the p-value can be computed by replacing the unknown correlation by the sample correlation, and the resulting test is quite satisfactory. For small samples, it is proposed to compute the p-value by replacing the unknown correlation by a scalar multiple of the sample correlation. However, a single scalar is not satisfactory, and it is proposed to use different scalars depending on the magnitude of the sample correlation coefficient. In order to implement this approach, tables are obtained providing sub-intervals for the sample correlation coefficient, and the scalars to be used if the sample correlation coefficient belongs to a particular sub-interval. Once such tables are available, the proposed p-value is quite easy to compute since it has an explicit analytic expression. Numerical results on the type I error probability and power are reported on the performance of such a test, and the proposed p-value test is also compared to another test based on a rejection region. The results are illustrated with two examples: an example dealing with the comparability of two measuring devices, and an example dealing with the assessment of bioequivalence.     

Two-Tailed p-Values and Coherent Measures of Evidence

ABSTRACT

In a test of significance, it is common practice to report the p-value as one way of summarizing the incompatibility between a set of data and a proposed model for the data constructed under a set of assumptions together with a null hypothesis. However, the p-value does have some flaws, one being in general its definition for two-sided tests and a related serious logical one of incoherence, in its interpretation as a statistical measure of evidence for its respective null hypothesis. We shall address these two issues in this article.     

Parametric and permutation testing for multivariate monotonic alternatives

We are firstly interested in testing the homogeneity of k mean vectors against two-sided restricted alternatives separately in multivariate normal distributions. This problem is a multivariate extension of Bartholomew (in Biometrica 46:328–335, 1959b) and an extension of Sasabuchi et al. (in Biometrica 70:465–472, 1983) and Kulatunga and Sasabuchi (in Mem. Fac. Sci., Kyushu Univ. Ser. A: Mathematica 38:151–161, 1984) to two-sided ordered hypotheses. We examine the problem of testing under two separate cases. One case is that covariance matrices are known, the other one is that covariance matrices are unknown but common. For the general case that covariance matrices are known the test statistic is obtained using the likelihood ratio method. When the known covariance matrices are common and diagonal, the null distribution of test statistic is derived and its critical values are computed at different significance levels. A Monte Carlo study is also presented to estimate the power of the test. A test statistic is proposed for the case when the common covariance matrices are unknown. Since it is difficult to compute the exact p-value for this problem of testing with the classical method when the covariance matrices are completely unknown, we first present a reformulation of the test statistic based on the orthogonal projections on the closed convex cones and then determine the upper bounds for its p-values. Also we provide a general nonparametric solution based on the permutation approach and nonparametric combination of dependent tests.     

Inferences on the Coefficients of Variation in a Multivariate Normal Population

In this article, the problem of testing the equality of coefficients of variation in a multivariate normal population is considered, and an asymptotic approach and a generalized p-value approach based on the concepts of generalized test variable are proposed. Monte Carlo simulation studies show that the proposed generalized p-value test has good empirical sizes, and it is better than the asymptotic approach. In addition, the problem of hypothesis testing and confidence interval for the common coefficient variation of a multivariate normal population are considered, and a generalized p-value and a generalized confidence interval are proposed. Using Monte Carlo simulation, we find that the coverage probabilities and expected lengths of this generalized confidence interval are satisfactory, and the empirical sizes of the generalized p-value are close to nominal level. We illustrate our approaches using a real data.     

Efficient weighted bootstraps for the mean

The weighted bootstrap due to Mason and Newton (1992. Ann. Statist. 20, 1611–1624.) is applied to Studentized statistics in view of deriving efficient confidence intervals for the mean. First, we give conditions on the moments of the weights to ensure that the weighted bootstrap approximation leads to uniformly correct two-sided confidence intervals up to the rate O(n^−3/2). Then, we discuss the practical choice of the random weights in order to construct one-sided confidence intervals accurate up to O(n^−3/2) and two-sided confidence intervals up to higher orders. Simulations are given to illustrate the practical efficiency of our approach.     

Estimating the p-values of robust tests for the linear model

In this paper, we study the estimation of p-values for robust tests for the linear regression model. The asymptotic distribution of these tests has only been studied under the restrictive assumption of errors with known scale or symmetric distribution. Since these robust tests are based on robust regression estimates, Efron's bootstrap (1979) presents a number of problems. In particular, it is computationally very expensive, and it is not resistant to outliers in the data. In other words, the tails of the bootstrap distribution estimates obtained by re-sampling the data may be severely affected by outliers.We show how to adapt the Robust Bootstrap (Ann. Statist 30 (2002) 556; Bootstrapping MM-estimators for linear regression with fixed designs, http://mathstat.carleton.ca/~matias/pubs.html) to this problem. This method is very fast to compute, resistant to outliers in the data, and asymptotically correct under weak regularity assumptions. In this paper, we show that the Robust Bootstrap can be used to obtain asymptotically correct, computationally simple p-value estimates. A simulation study indicates that the tests whose p-values are estimated with the Robust Bootstrap have better finite sample significance levels than those obtained from the asymptotic theory based on the symmetry assumption.Although this paper is focussed on robust scores-type tests (in: Directions in Robust Statistics and Diagnostics, Part I, Springer, New York), our approach can be applied to other robust tests (for example, Wald- and dispersion-type also discussed in Markatou et al., 1991).     

Two Wilson–Hilferty type approximations for the null distribution of the Blum,Kiefer and Rosenblatt test of bivariate independence

The Blum et al. (Ann. Math. Statist. 32 (1961) 485) test of bivariate independence, an asymptotic equivalent of Hoeffding's (Ann. Math. Statist. 19 (1948) 546) test, is consistent against all dependence alternatives. A concise tabulation of a well-considered approximation for the asymptotic percentiles of its null distribution is given in Blum et al. and a more complete selection of Monte Carlo percentiles, for samples of size 5 and larger, appears in Mudholkar and Wilding (J. Roy. Statist. Soc. 52 (2003) 1). However, neither tabulation is adequate for estimating p-values of the test. In this note we use a moment based analogue of the classical Wilson–Hilferty transformation to obtain two transformations of type T_n=(nB_n)^h_n. The transformations T_n are then used to construct and compare a Gaussian and a scaled chi-square approximation for the null distribution of nB_n. Both approximations have excellent accuracy, but the Gaussian approximation is more convenient because of its portability.     

Implementation of Kolmogorov–Smirnov P-value computation in Visual Basic®: implication for Microsoft Excel® library function

This paper investigates methodologies for evaluating the probabilistic value (P-value) of the Kolmogorov–Smirnov (K–S) goodness-of-fit test using algorithmic program development implemented in Microsoft® Visual Basic® (VB). Six methods were examined for the one-sided one-sample and two methods for the two-sided one-sample cumulative sampling distributions in the investigative software implementation that was based on machine-precision arithmetic. For sample sizes n≤2000 considered, results from the Smirnov iterative method found optimal accuracy for K–S P-values≥0.02, while those from the SmirnovD were more accurate for lower P-values for the one-sided one-sample distribution statistics. Also, the Durbin matrix method sustained better P-value results than the Durbin recursion method for the two-sided one-sample tests up to n≤700 sample sizes. Based on these results, an algorithm for Microsoft Excel® function was proposed from which a model function was developed and its implementation was used to test the performance of engineering students in a general engineering course across seven departments.     

Sample size determination in step-down and step-up multiple tests for comparing treatments with a control

We address the problem of sample size determination in multiple comparisons of k treatments with a control for step-down and step-up testing, assuming normal data and homogeneous variances. We define power as the probability of correctly rejecting all hypotheses for which the treatment vs. control difference exceeds a specified value. Our paper supplements papers by Hayter and Tamhane (J. Statist. Plann. Inference 27 (1991) 271–290) who solved the problem for one-sided comparisons using the step-down procedure and by Liu (J. Statist. Plann. Inference 62 (1997b) 255–261) who considered the two-sided case using the single-step method. We provide expressions that allow computer evaluation of the power and necessary sample sizes for one- and two-sided tests using either step-down or step-up procedures. Tables are given from which sample sizes to guarantee a specified power can be determined.