Monte carlo sampling approach to testing nonnested hypothesis: monte carlo results

Alternative ways of using Monte Carlo methods to implement a Cox-type test for separate families of hypotheses are considered. Monte Carlo experiments are designed to compare the finite sample performances of Pesaran and Pesaran's test, a RESET test, and two Monte Carlo hypothesis test procedures. One of the Monte Carlo tests is based on the distribution of the log-likelihood ratio and the other is based on an asymptotically pivotal statistic. The Monte Carlo results provide strong evidence that the size of the Pesaran and Pesaran test is generally incorrect, except for very large sample sizes. The RESET test has lower power than the other tests. The two Monte Carlo tests perform equally well for all sample sizes and are both clearly preferred to the Pesaran and Pesaran test, even in large samples. Since the Monte Carlo test based on the log-likelihood ratio is the simplest to calculate, we recommend using it.     

Manipulating Statistical Hypotheses and Tests as Boolean Functions

Statistical hypotheses and test statistics are Boolean functions that can be manipulated using the tools of Boolean algebra. These tools are particularly useful for exploring multiple comparisons or simultaneous inference theory, in which multiparameter hypotheses or multiparameter test statistics may be decomposed into combinations of uniparameter hypotheses or uniparameter tests. These concepts are illustrated with both finite and infinite decompositions of familiar multiparameter hypotheses and tests. The corresponding decompositions of acceptance regions and rejection regions are also shown. Finally, the close relationship between hypothesis and test decompositions and Roy's union—intersection principle is demonstrated by a derivation of the union—intersection test of the univariate general linear hypothesis.     

Evidence From Marginally Significant t Statistics

ABSTRACT

This article examines the evidence contained in t statistics that are marginally significant in 5% tests. The bases for evaluating evidence are likelihood ratios and integrated likelihood ratios, computed under a variety of assumptions regarding the alternative hypotheses in null hypothesis significance tests. Likelihood ratios and integrated likelihood ratios provide a useful measure of the evidence in favor of competing hypotheses because they can be interpreted as representing the ratio of the probabilities that each hypothesis assigns to observed data. When they are either very large or very small, they suggest that one hypothesis is much better than the other in predicting observed data. If they are close to 1.0, then both hypotheses provide approximately equally valid explanations for observed data. I find that p-values that are close to 0.05 (i.e., that are “marginally significant”) correspond to integrated likelihood ratios that are bounded by approximately 7 in two-sided tests, and by approximately 4 in one-sided tests.

The modest magnitude of integrated likelihood ratios corresponding to p-values close to 0.05 clearly suggests that higher standards of evidence are needed to support claims of novel discoveries and new effects.     

Testing Time Series for Nonlinearity

The technique of surrogate data analysis may be employed to test the hypothesis that an observed data set was generated by one of several specific classes of dynamical system. Current algorithms for surrogate data analysis enable one, in a generic way, to test for membership of the following three classes of dynamical system: (0) independent and identically distributed noise, (1) linearly filtered noise, and (2) a monotonic nonlinear transformation of linearly filtered noise.We show that one may apply statistics from nonlinear dynamical systems theory, in particular those derived from the correlation integral, as test statistics for the hypothesis that an observed time series is consistent with each of these three linear classes of dynamical system. Using statistics based on the correlation integral we show that it is also possible to test much broader (and not necessarily linear) hypotheses.We illustrate these methods with radial basis models and an algorithm to estimate the correlation dimension. By exploiting some special properties of this correlation dimension estimation algorithm we are able to test very specific hypotheses. Using these techniques we demonstrate the respiratory control of human infants exhibits a quasi-periodic orbit (the obvious inspiratory/expiratory cycle) together with cyclic amplitude modulation. This cyclic amplitude modulation manifests as a stable focus in the first return map (equivalently, the sequence of successive peaks).     

A Method for Designing Three-Hypothesis Test Problems and Sequential Schemes

In applications, a two-sided hypothesis test problem sometimes needs to be changed to a three-hypothesis one with the two alternative hypotheses properly selected. In this article, we obtain the hypothesis design and the three-hypothesis sequential test scheme under the Koopman–Darmois distribution by solving a system of equations that meet requirements on the error rates and average sample number. This method provides a useful guide for practitioners to design hypotheses in multihypothesis test problems with controlled error rates and sampling cost. Formulas of the scheme's error rates and average sample number are obtained using numerical quadrature for the discrete-time situation.     

Analysis of ordered covariate effects among groups with repeated measurements

Testing procedures for ordered covariate effects are developed in the repeated measures experiment. The maximum likelihood estimators of covariate effects under the ordered hypothesis are approximated by the isotonic regression of their unconstrained estimators. The asymptotic null distributions of the test statistics are chi-bar-square distributions which are mixtures of chi-square distributions. A Monte-Carlo simulation reveals that the proposed test for ordered covariate effects is seriously more powerful than the usual chi-square test that neglects the information on the order restriction. These testing methods are applied for analyzing the effect of vitamin E diet supplement on growth rate of animals.     

RAO'S SCORE TESTS IN SURVIVAL ANALYSIS: EXAMPLES AND BARTLETT TYPE ADJUSTMENTS

Score method in hypothesis testing is one of Professor C. R. Rao's great contributions to statistics. It provides a simple and unified way to test some simple and composite hypotheses in many statistical problems. Some popular tests in statistical practice derived with the help of intuitions can be shown as score tests under some statistical models. The subject-years test and log-rank test in survival analysis are two of the examples. In this paper, we first introduce these two examples. After formulating these two tests as score tests, we then review some recent results on the Bartlett type adjustments for these tests.     

The new robust two-sample test for randomly right-censored data

Using a minimum p-value principle, a new two-sample test MIN3 is proposed in the paper. The cumulative distribution function of the MIN3 test statistic is studied and approximated by the Beta distribution of the third kind. Lower percentage points of the distribution of the new test statistic under the null hypothesis are computed. Also the test power for a lot of types of alternative hypotheses (with 0, 1 and 2 point(-s) of the intersection(-s) of survival functions) is studied and we found that the usage of the MIN3 test is a preferred strategy by the Wald and Savage decision-making criteria under risk and uncertainty. The results of application of the MIN3 test are shown for two examples from lifetime data analysis.     

Comparison of correlated proportions based on paired binary data from clustered samples

In this article, we explore hypothesis testing problems related to correlated proportions from clustered matched-pair binary data. Null hypotheses of equality in proportions, homogeneity, and non-inferiority of one to another are similar testing problems of linear contrasts of correlated proportions with suitable transformation. The covariance estimators of the test statistics are based on moment estimation under the null hypotheses. We present a general framework for testing linear contrasts of the correlated proportions from clustered matched-pair data based upon a class of unbiased estimators of the proportions. The corresponding testing procedures do not impose structure assumptions on the correlation matrix and are easy to use. Simulation results suggest that the proposed method is more likely to maintain the proper significance level and to improve power than the test proposed by Obuchowski.     

Use of baseline measurements in the two-period crossover design

It is proposed that baseline measurements be obtained prior to each period in a two-period crossover design. These measurements are used in a preliminary test for determining the validity of a test for treatment comparison and also for testing the hypothesis of equal treatment effects. The null hypothesis in this preliminary test consists of the following three hypotheses: that there is no difference in disease conditions prior to the two periods, no difference in residual effects of the drugs, and no treatment × period interaction.. A numerical example is given and the efficiencies of several methods are computed.     

A combined p-value test for multiple hypothesis testing

Tests that combine p-values, such as Fisher's product test, are popular to test the global null hypothesis H₀ that each of n component null hypotheses, H₁,…,H_n, is true versus the alternative that at least one of H₁,…,H_n is false, since they are more powerful than classical multiple tests such as the Bonferroni test and the Simes tests. Recent modifications of Fisher's product test, popular in the analysis of large scale genetic studies include the truncated product method (TPM) of Zaykin et al. (2002), the rank truncated product (RTP) test of Dudbridge and Koeleman (2003) and more recently, a permutation based test—the adaptive rank truncated product (ARTP) method of Yu et al. (2009). The TPM and RTP methods require users' specification of a truncation point. The ARTP method improves the performance of the RTP method by optimizing selection of the truncation point over a set of pre-specified candidate points. In this paper we extend the ARTP by proposing to use all the possible truncation points {1,…,n} as the candidate truncation points. Furthermore, we derive the theoretical probability distribution of the test statistic under the global null hypothesis H₀. Simulations are conducted to compare the performance of the proposed test with the Bonferroni test, the Simes test, the RTP test, and Fisher's product test. The simulation results show that the proposed test has higher power than the Bonferroni test and the Simes test, as well as the RTP method. It is also significantly more powerful than Fisher's product test when the number of truly false hypotheses is small relative to the total number of hypotheses, and has comparable power to Fisher's product test otherwise.     

Testing non-inferiority and superiority for two endpoints for several treatments with a control

Some multiple comparison procedures are described for multiple armed studies. The procedures are appropriate for testing all hypotheses for comparing two endpoints and multiple test arms to a single control group, for example three different fixed doses compared to a placebo. The procedure assumes that among the two endpoints, one is designated as a primary endpoint such that for a given treatment arm, no hypothesis for the secondary endpoint can be rejected unless the hypothesis for the primary endpoint was rejected. The procedures described control the family-wise error rate in the strong sense at a specified level α.     

Testing for two states in a hidden Markov model

The authors consider hidden Markov models (HMMs) whose latent process has m ≥ 2 states and whose state‐dependent distributions arise from a general one‐parameter family. They propose a test of the hypothesis m = 2. Their procedure is an extension to HMMs of the modified likelihood ratio statistic proposed by Chen, Chen & Kalbfleisch (2004) for testing two states in a finite mixture. The authors determine the asymptotic distribution of their test under the hypothesis m = 2 and investigate its finite‐sample properties in a simulation study. Their test is based on inference for the marginal mixture distribution of the HMM. In order to illustrate the additional difficulties due to the dependence structure of the HMM, they show how to test general regular hypotheses on the marginal mixture of HMMs via a quasi‐modified likelihood ratio. They also discuss two applications.     

On the sample breakdown robustness of some nonparametric tests

In this paper, relying on the sample breakdown points, we investigate the sample breakdown properties of some nonparametric tests. It is shown that the sample breakdown points of the sign test asymptotically dominate those of the Wilcoxon test for one–sided hypotheses, However, the different conclusion is derived in the case of testing some shrinking neighborhood hypotheses. The breakdown behaviors of the Kolmogorov test and X²–test are also explored. These studies unify or refine some existing breakdown analyses of tests.     

Invariant tests based on M-estimators,estimating functions,and the generalized method of moments

We study the invariance properties of various test criteria which have been proposed for hypothesis testing in the context of incompletely specified models, such as models which are formulated in terms of estimating functions (Godambe, 1960) or moment conditions and are estimated by generalized method of moments (GMM) procedures (Hansen, 1982), and models estimated by pseudo-likelihood (Gouriéroux, Monfort, and Trognon, 1984b,c) and M-estimation methods. The invariance properties considered include invariance to (possibly nonlinear) hypothesis reformulations and reparameterizations. The test statistics examined include Wald-type, LR-type, LM-type, score-type, and C(α)?type criteria. Extending the approach used in Dagenais and Dufour (1991), we show first that all these test statistics except the Wald-type ones are invariant to equivalent hypothesis reformulations (under usual regularity conditions), but all five of them are not generally invariant to model reparameterizations, including measurement unit changes in nonlinear models. In other words, testing two equivalent hypotheses in the context of equivalent models may lead to completely different inferences. For example, this may occur after an apparently innocuous rescaling of some model variables. Then, in view of avoiding such undesirable properties, we study restrictions that can be imposed on the objective functions used for pseudo-likelihood (or M-estimation) as well as the structure of the test criteria used with estimating functions and generalized method of moments (GMM) procedures to obtain invariant tests. In particular, we show that using linear exponential pseudo-likelihood functions allows one to obtain invariant score-type and C(α)?type test criteria, while in the context of estimating function (or GMM) procedures it is possible to modify a LR-type statistic proposed by Newey and West (1987) to obtain a test statistic that is invariant to general reparameterizations. The invariance associated with linear exponential pseudo-likelihood functions is interpreted as a strong argument for using such pseudo-likelihood functions in empirical work.     

Wavelet goodness-of-fit test for dependent data

We introduce a new goodness-of-fit test which can be applied to hypothesis testing about the marginal distribution of dependent data. We derive a new test for the equivalent hypothesis in the space of wavelet coefficients. Such properties of the wavelet transform as orthogonality, localisation and sparsity make the hypothesis testing in wavelet domain easier than in the domain of distribution functions. We propose to test the null hypothesis separately at each wavelet decomposition level to overcome the problem of bi-dimensionality of wavelet indices and to be able to find the frequency where the empirical distribution function differs from the null in case the null hypothesis is rejected. We suggest a test statistic and state its asymptotic distribution under the null and under some of the alternative hypotheses.     

A simple nonparametric test for bivariate symmetry about a line

Over the years many researchers have dealt with testing the hypotheses of symmetry in univariate and multivariate distributions in the parametric and nonparametric setup. In a multivariate setup, there are several formulations of symmetry, for example, symmetry about an axis, joint symmetry, marginal symmetry, radial symmetry, symmetry about a known point, spherical symmetry, and elliptical symmetry among others. In this paper, for the bivariate case, we formulate a concept of symmetry about a straight line passing through the origin in a plane and accordingly develop a simple nonparametric test for testing the hypothesis of symmetry about a straight line. The proposed test is based on a measure of deviance between observed counts of bivariate samples in suitably defined pairs of sets. The exact null distribution and non-null distribution, for specified classes of alternatives, of the test statistics are obtained. The null distribution is tabulated for sample size from n=5 up to n=30. The null mean, null variance and the asymptotic null distributions of the proposed test statistics are also obtained. The empirical power of the proposed test is evaluated by simulating samples from the suitable class of bivariate distributions. The empirical findings suggest that the test performs reasonably well against various classes of asymmetric bivariate distributions. Further, it is advocated that the basic idea developed in this work can be easily adopted to test the hypotheses of exchangeability of bivariate random variables and also bivariate symmetry about a given axis which have been considered by several authors in the past.     

A note on homogeneity tests when combining 2×2 tables

This paper examines the use of homogeneity tests prior to tests of overall association among g 2 x 2 tables. When placed in the context of a one-way analysis of variance, hypotheses of overall association and homogeneity can be viewed as hypotheses regarding mean and treatment effects, respectively. In this context, the need for homogeneity tests is presented. What constitutes a relevant test of homogeneity is also examined. The conclusion is that some of the difficulties raised in the literature regarding tests of homogeneity stem from differences in the hypothesis of association being examined.     

Extensions of empirical likelihood and chi-squared-based tests for ordered alternatives

Several methods for comparing k populations have been proposed in the literature. These methods assess the same null hypothesis of equal distributions but differ in the alternative hypothesis they consider. We focus on two important alternative hypotheses: monotone and umbrella ordering. Two new families of test statistics are proposed, including two known tests, as well as two new powerful tests under monotone ordering. Furthermore, these families are adapted for testing umbrella ordering. We compare some members of the families with respect to power and Type I errors under different simulation scenarios. Finally, the methods are illustrated in several applications to real data.     

Restricted most powerful Bayesian tests for linear models

Uniformly most powerful Bayesian tests (UMPBTs) are a new class of Bayesian tests in which null hypotheses are rejected if their Bayes factor exceeds a specified threshold. The alternative hypotheses in UMPBTs are defined to maximize the probability that the null hypothesis is rejected. Here, we generalize the notion of UMPBTs by restricting the class of alternative hypotheses over which this maximization is performed, resulting in restricted most powerful Bayesian tests (RMPBTs). We then derive RMPBTs for linear models by restricting alternative hypotheses to g priors. For linear models, the rejection regions of RMPBTs coincide with those of usual frequentist F‐tests, provided that the evidence thresholds for the RMPBTs are appropriately matched to the size of the classical tests. This correspondence supplies default Bayes factors for many common tests of linear hypotheses. We illustrate the use of RMPBTs for ANOVA tests and t‐tests and compare their performance in numerical studies.