Closed testing procedures for all pairwise comparisons in a randomized block design

We consider multiple comparison test procedures among treatment effects in a randomized block design. We propose closed testing procedures based on maximum values of some two-sample t test statistics and based on F test statistics. It is shown that the proposed procedures are more powerful than single-step procedures and the REGW (Ryan/Einot–Gabriel/Welsch)-type tests. Next, we consider the randomized block design under simple ordered restrictions of treatment effects. We propose closed testing procedures based on maximum values of two-sample one-sided t test statistics and based on Batholomew’s statistics for all pairwise comparisons of treatment effects. Although single-step multiple comparison procedures are utilized in general, the power of these procedures is low for a large number of groups. The closed testing procedures stated in the present article are more powerful than the single-step procedures. Simulation studies are performed under the null hypothesis and some alternative hypotheses. In this studies, the proposed procedures show a good performance.     

Extended tests for non-zero between-study variance

The ANOVA F-test, James tests and generalized F-test are extended to test hypotheses on the between-study variance for values greater than zero. Using simulations, we compare the performance of extended test procedures with respect to the actual attained type I error rate. Examples are provided to demonstrate the application of the procedures in ANOVA models and meta-analysis.     

Testing for monotonic trend in time series based on resampling methods

In this paper, we propose several tests for monotonic trend based on the Brillinger's test statistic (1989, Biometrika, 76, 23–30). When there are highly correlated residuals or short record lengths, Brillinger's test procedure tends to have significance level much higher than the nominal level. It is found that this could be related to the discrepancy between the empirical distribution of the test statistic and the asymptotic normal distribution. Hence, in this paper, we propose three bootstrap-based procedures based on the Brillinger's test statistic to test for monotonic trend. The performance of the proposed test procedures is evaluated through an extensive Monte Carlo simulation study, and is compared to other trend test procedures in the literature. It is shown that the proposed bootstrap-based Brillinger test procedures can well control the significance levels and provide satisfactory power performance in testing the monotonic trend under different scenarios.     

Exact unconditional testing procedures for comparing two independent Poisson rates

We consider seven exact unconditional testing procedures for comparing adjusted incidence rates between two groups from a Poisson process. Exact tests are always preferable due to the guarantee of test size in small to medium sample settings. Han [Comparing two independent incidence rates using conditional and unconditional exact tests. Pharm Stat. 2008;7(3):195–201] compared the performance of partial maximization p-values based on the Wald test statistic, the likelihood ratio test statistic, the score test statistic, and the conditional p-value. These four testing procedures do not perform consistently, as the results depend on the choice of test statistics for general alternatives. We consider the approach based on estimation and partial maximization, and compare these to the ones studied by Han (2008) for testing superiority. The procedures are compared with regard to the actual type I error rate and power under various conditions. An example from a biomedical research study is provided to illustrate the testing procedures. The approach based on partial maximization using the score test is recommended due to the comparable performance and computational advantage in large sample settings. Additionally, the approach based on estimation and partial maximization performs consistently for all the three test statistics, and is also recommended for use in practice.     

Using principal components to test normality of high-dimensional data

Many multivariate statistical procedures are based on the assumption of normality and different approaches have been proposed for testing this assumption. The vast majority of these tests, however, are exclusively designed for cases when the sample size n is larger than the dimension of the variable p, and the null distributions of their test statistics are usually derived under the asymptotic case when p is fixed and n increases. In this article, a test that utilizes principal components to test for nonnormality is proposed for cases when p/n → c. The power and size of the test are examined through Monte Carlo simulations, and it is argued that the test remains well behaved and consistent against most nonnormal distributions under this type of asymptotics.     

Invariant tests based on M-estimators,estimating functions,and the generalized method of moments

We study the invariance properties of various test criteria which have been proposed for hypothesis testing in the context of incompletely specified models, such as models which are formulated in terms of estimating functions (Godambe, 1960) or moment conditions and are estimated by generalized method of moments (GMM) procedures (Hansen, 1982), and models estimated by pseudo-likelihood (Gouriéroux, Monfort, and Trognon, 1984b,c) and M-estimation methods. The invariance properties considered include invariance to (possibly nonlinear) hypothesis reformulations and reparameterizations. The test statistics examined include Wald-type, LR-type, LM-type, score-type, and C(α)?type criteria. Extending the approach used in Dagenais and Dufour (1991), we show first that all these test statistics except the Wald-type ones are invariant to equivalent hypothesis reformulations (under usual regularity conditions), but all five of them are not generally invariant to model reparameterizations, including measurement unit changes in nonlinear models. In other words, testing two equivalent hypotheses in the context of equivalent models may lead to completely different inferences. For example, this may occur after an apparently innocuous rescaling of some model variables. Then, in view of avoiding such undesirable properties, we study restrictions that can be imposed on the objective functions used for pseudo-likelihood (or M-estimation) as well as the structure of the test criteria used with estimating functions and generalized method of moments (GMM) procedures to obtain invariant tests. In particular, we show that using linear exponential pseudo-likelihood functions allows one to obtain invariant score-type and C(α)?type test criteria, while in the context of estimating function (or GMM) procedures it is possible to modify a LR-type statistic proposed by Newey and West (1987) to obtain a test statistic that is invariant to general reparameterizations. The invariance associated with linear exponential pseudo-likelihood functions is interpreted as a strong argument for using such pseudo-likelihood functions in empirical work.     

One-sided range test for testing against an ordered alternative under heteroscedasticity

In a one-way fixed effects analysis of variance model, when normal variances are unknown and possibly unequal, a one-sided range test for testing the null hypothesis H ₀ : μ ₁ = … = μ_k against an ordered alternative H_a : μ ₁ ≤ … ≤ μ_k by a single-stage and a two-stage procedure, respectively, is proposed. The critical values under H ₀ and the power under a specific alternative are calculated. Relation between the one-stage and the two-stage test procedures is discussed. A numerical example to illustrate these procedures is given.     

Paradoxes in nonparametric tests

This paper is a continuation of one (1992) in which the author studied the paradoxes that can arise when a nonparametric statistical test is used to give an ordering of k samples and the subsets of those samples. This article characterizes the projection paradoxes that can occur when using contingency tables, complete block designs, and tests of dichotomous behaviour of several samples. This is done by examining the “dictionaries” of possible orderings of each of these procedures. Specifically, it is shown that contingency tables and complete block designs, like the Kruskal-Wallis nonparametric test on k samples, minimize the number and kinds of projection paradoxes that can occur; however, using a test of dichotomous behaviour of several samples does not. An analysis is given of two procedures used to determine the ordering of a pair of samples from a set of k samples. It is shown that these two procedures may not have anything in common.     

Comparisons of several Pareto distributions based on record values

In order to avoid wrong conclusions in any further analysis, it is of importance to conduct a formal comparison for characteristic quantities of the distributions. These characteristic quantities we are familiar with include mean, quantity and reliability function, and so on. In this paper, we consider two tests aiming at the comparisons for function of parameters in Pareto distribution based on record values. They are generalized p-value-based test and parametric bootstrap-based test, respectively. The resulting procedures are easy to compute and are applicable to small samples. A simulation study is conducted to investigate and compare the performance of the proposed tests. A phenomenon we note is that generalized p-value-based test almost uniformly outperforms the parametric bootstrap-based test.     

Simulation-Based Tests that Can Use Any Number of Simulations

Conventional procedures for Monte Carlo and bootstrap tests require that B, the number of simulations, satisfy a specific relationship with the level of the test. Otherwise, a test that would instead be exact will either overreject or underreject for finite B. We present expressions for the rejection frequencies associated with existing procedures and propose a new procedure that yields exact Monte Carlo tests for any positive value of B. This procedure, which can also be used for bootstrap tests, is likely to be most useful when simulation is expensive.     

Comparing distributions of time to onset of disease in animal tumorigenicity experiments

The cause-of-death test of Peto et al.(1980)pools information from a Hoel-Walburg test on incidental tumors with information from a logrank test on fatal tumors in order to compare the tumor rate of a group of rodents exposed to a carcinogen against the tumor rate of a group of unexposed animals. The cause-of-death test, which can arise as a partial likelihood score test from a model that assumes proportional odds for tumor prevalence and proportional hazards for tumor mortality, is not, in general, a direct test for equality of tumor onset distributions for occult tumors that are observed in both fatal and incidental contexts. This paper develops a direct cause-of-death test for comparing distributions of time to onset of occultumors. The test is derived as a partial likelihood score test under an assumed proportional hazards model for tumor onset distributions. The size and power of the proposed test are compared in a Monte Carlo simulation study to the size and power of competitive procedures, including procedures that do not require cause-of-death information.     

Data Dependent Cells Chi‐Square Test With Recurrent Events

We consider a recurrent event wherein the inter‐event times are independent and identically distributed with a common absolutely continuous distribution function F. In this article, interest is in the problem of testing the null hypothesis that F belongs to some parametric family where the q‐dimensional parameter is unknown. We propose a general Chi‐squared test in which cell boundaries are data dependent. An estimator of the parameter obtained by minimizing a quadratic form resulting from a properly scaled vector of differences between Observed and Expected frequencies is used to construct the test. This estimator is known as the minimum chi‐square estimator. Large sample properties of the proposed test statistic are established using empirical processes tools. A simulation study is conducted to assess the performance of the test under parameter misspecification, and our procedures are applied to a fleet of Boeing 720 jet planes' air conditioning system failures.     

Testing for normality in linear regression models

The importance of the normal distribution for fitting continuous data is well known. However, in many practical situations data distribution departs from normality. For example, the sample skewness and the sample kurtosis are far away from 0 and 3, respectively, which are nice properties of normal distributions. So, it is important to have formal tests of normality against any alternative. D'Agostino et al. [A suggestion for using powerful and informative tests of normality, Am. Statist. 44 (1990), pp. 316–321] review four procedures Z ²(g ₁), Z ²(g ₂), D and K ² for testing departure from normality. The first two of these procedures are tests of normality against departure due to skewness and kurtosis, respectively. The other two tests are omnibus tests. An alternative to the normal distribution is a class of skew-normal distributions (see [A. Azzalini, A class of distributions which includes the normal ones, Scand. J. Statist. 12 (1985), pp. 171–178]). In this paper, we obtain a score test (W) and a likelihood ratio test (LR) of goodness of fit of the normal regression model against the skew-normal family of regression models. It turns out that the score test is based on the sample skewness and is of very simple form. The performance of these six procedures, in terms of size and power, are compared using simulations. The level properties of the three statistics LR, W and Z ²(g ₁) are similar and close to the nominal level for moderate to large sample sizes. Also, their power properties are similar for small departure from normality due to skewness (γ₁≤0.4). Of these, the score test statistic has a very simple form and computationally much simpler than the other two statistics. The LR statistic, in general, has highest power, although it is computationally much complex as it requires estimates of the parameters under the normal model as well as those under the skew-normal model. So, the score test may be used to test for normality against small departure from normality due to skewness. Otherwise, the likelihood ratio statistic LR should be used as it detects general departure from normality (due to both skewness and kurtosis) with, in general, largest power.     

Multiple Comparison Procedures: The Practical Solution

A practicing statistician looks at the multiple comparison controversy and related issues through the eyes of the users. The concept of consistency is introduced and discussed in relation to five of the more common multiple comparison procedures. All of the procedures are found to be inconsistent except the simplest procedure, the unrestricted least significant difference (LSD) procedure (or multiple t test). For this and other reasons the unrestricted LSD procedure is recommended for general use, with the proviso that it should be viewed as a hypothesis generator rather than as a method for simultaneous hypothesis generation and testing. The implications for Scheffé's test for general contrasts are also discussed, and a new recommendation is made.     

Evaluation of different statistical methods using SAS software: an in silico approach for analysis of real-time PCR data

Real-time polymerase chain reaction (PCR) is reliable quantitative technique in gene expression studies. The statistical analysis of real-time PCR data is quite crucial for results analysis and explanation. The statistical procedures of analyzing real-time PCR data try to determine the slope of regression line and calculate the reaction efficiency. Applications of mathematical functions have been used to calculate the target gene relative to the reference gene(s). Moreover, these statistical techniques compare C_t (threshold cycle) numbers between control and treatments group. There are many different procedures in SAS for real-time PCR data evaluation. In this study, the efficiency of calibrated model and delta delta C_t model have been statistically tested and explained. Several methods were tested to compare control with treatment means of C_t. The methods tested included t-test (parametric test), Wilcoxon test (non-parametric test) and multiple regression. Results showed that applied methods led to similar results and no significant difference was observed between results of gene expression measurement by the relative method.     

Goodness-of-fit test for Gaussian regression with block correlated errors

We propose a procedure to test that the expectation of a Gaussian vector is linear against a nonparametric alternative. We consider the case where the covariance matrix of the observations has a block diagonal structure. This framework encompasses regression models with autocorrelated errors, heteroscedastic regression models, mixed-effects models and growth curves. Our procedure does not depend on any prior information about the alternative. We prove that the test is asymptotically of the nominal level and consistent. We characterize the set of vectors on which the test is powerful and prove the classical √log log (n)/n convergence rate over directional alternatives. We propose a bootstrap version of the test as an alternative to the initial one and provide a simulation study in order to evaluate both procedures for small sample sizes when the purpose is to test goodness of fit in a Gaussian mixed-effects model. Finally, we illustrate the procedures using a real data set.     

Rank tests in the two-sample scale problem with unequal and unknown locations

The two-sample scale problem is studied in the case of unequal and unknown location parameters. The method proposed is based on the idea of Moses (1963) and it is distribution-free. The two samples are separated into random subgroups of the same sizek. It is proposed to choosek=4 and to apply the Wilconxon test or the Savage test to the ranges or sample variances of the subgroups. The asymptotic power functions of the tests are compared. For small and moderate sample sizes simulations are carried out. Relations to some other procedures, especially to the method of Compagnone and Denker (1996) are briefly discussed.     

Testing Variability in the Two-Sample Case

A number of robust methods for testing variability have been reported in previous literature. An examination of these procedures for a wide variety of populations confirms their general robustness. Shoemaker's improvement of the F test extends that test use to a realistic variety of population shapes. However, a combination of the Brown–Forsythe and O'Brien methods based on testing kurtosis is shown to be conservative for a wide range of sample sizes and population distributions. The composite test is also shown to be more powerful in most conditions than other conservative procedures.     

Choosing the best pairwise comparisons of means from non-normal populations,with unequal variances,but equal sample sizes

A Monte Carlo simulation evaluated five pairwise multiple comparison procedures for controlling Type I error rates, any-pair power, and all-pairs power. Realistic conditions of non-normality were based on a previous survey. Variance ratios were varied from 1:1 to 64:1. Procedures evaluated included Tukey's honestly significant difference (HSD) preceded by an F test, the Hayter–Fisher, the Games–Howell preceded by an F test, the Pertiz with F tests, and the Peritz with Alexander–Govern tests. Tukey's procedure shows the greatest robustness in Type I error control. Any-pair power is generally best with one of the Peritz procedures. All-pairs power is best with the Pertiz F test procedure. However, Tukey's HSD preceded by the Alexander–Govern F test may provide the best combination for controlling Type I and power rates in a variety of conditions of non-normality and variance heterogeneity.     

Change-point detection in a shape-restricted regression model

A regression model with a possible structural change and with a small number of measurements is considered. A priori information about the shape of the regression function is used to formulate the model as a linear regression model with inequality constraints and a likelihood ratio test for the presence of a change-point is constructed. The exact null distribution of the test statistic is given. Consistency of the test is proved when the noise level goes to zero. Numerical approximations to the powers against various alternatives are given and compared with the powers of the k-linear-r-ahead recursive residuals tests and CUSUM tests. Performance of four different estimators of the change-point is studied in a Monte Carlo experiment. An application of the procedures to some real data is also presented.