Some Properties of Non Inferiority Tests for Two Independent Probabilities

For exact tests of non inferiority for two independent binomial probabilities, in 1999 Röhmel and Mansmann proved that if a rejection region from an exact test fulfills the Barnard convexity condition, then the corresponding significance level can be computed as the maximum in a subset of the null space boundary. This is particularly important because computing time of significance levels is greatly reduced. Later, in 2000, Frick extended the Röhmel and Mansmann theorem to more general critical regions also corresponding to exact tests. In this article, we generalize Frick's theorem to both exact and asymptotic tests. Like the two theorems mentioned, in this article the resulting theorem also includes, as particular cases, non inferiority hypotheses for parameters such as difference between proportions, proportions ratio, and odds ratio for two independent binomial probabilities. Moreover, proof of this result follows a different line of reasoning than that followed by Frick and is much simpler. In addition, some applications of the main result are provided.     

Power and sample size when multiple endpoints are considered

A common approach to analysing clinical trials with multiple outcomes is to control the probability for the trial as a whole of making at least one incorrect positive finding under any configuration of true and false null hypotheses. Popular approaches are to use Bonferroni corrections or structured approaches such as, for example, closed-test procedures. As is well known, such strategies, which control the family-wise error rate, typically reduce the type I error for some or all the tests of the various null hypotheses to below the nominal level. In consequence, there is generally a loss of power for individual tests. What is less well appreciated, perhaps, is that depending on approach and circumstances, the test-wise loss of power does not necessarily lead to a family wise loss of power. In fact, it may be possible to increase the overall power of a trial by carrying out tests on multiple outcomes without increasing the probability of making at least one type I error when all null hypotheses are true. We examine two types of problems to illustrate this. Unstructured testing problems arise typically (but not exclusively) when many outcomes are being measured. We consider the case of more than two hypotheses when a Bonferroni approach is being applied while for illustration we assume compound symmetry to hold for the correlation of all variables. Using the device of a latent variable it is easy to show that power is not reduced as the number of variables tested increases, provided that the common correlation coefficient is not too high (say less than 0.75). Afterwards, we will consider structured testing problems. Here, multiplicity problems arising from the comparison of more than two treatments, as opposed to more than one measurement, are typical. We conduct a numerical study and conclude again that power is not reduced as the number of tested variables increases.     

Bridging the gap: A likelihood function approach for the analysis of ranking data

ABSTRACT

In the parametric setting, the notion of a likelihood function forms the basis for the development of tests of hypotheses and estimation of parameters. Tests in connection with the analysis of variance stem entirely from considerations of the likelihood function. On the other hand, non parametric procedures have generally been derived without any formal mechanism and are often the result of clever intuition. In the present article, we propose a more formal approach for deriving tests involving the use of ranks. Specifically, we define a likelihood function motivated by characteristics of the ranks of the data and demonstrate that this leads to well-known tests of hypotheses. We also point to various areas of further exploration such as how co-variates may be incorporated.     

Bayesian t-tests for correlations and partial correlations

In this paper, we develop Bayes factor based testing procedures for the presence of a correlation or a partial correlation. The proposed Bayesian tests are obtained by restricting the class of the alternative hypotheses to maximize the probability of rejecting the null hypothesis when the Bayes factor is larger than a specified threshold. It turns out that they depend simply on the frequentist t-statistics with the associated critical values and can thus be easily calculated by using a spreadsheet in Excel and in fact by just adding one more step after one has performed the frequentist correlation tests. In addition, they are able to yield an identical decision with the frequentist paradigm, provided that the evidence threshold of the Bayesian tests is determined by the significance level of the frequentist paradigm. We illustrate the performance of the proposed procedures through simulated and real-data examples.     

Average Most Powerful Tests for a Segmented Regression

To improve the goodness of fit between a regression model and observations, the model can be complicated; however, that can reduce the statistical power when the complication does not lead significantly to an improved model. In the context of two-phase (segmented) logistic regressions, the model evaluation needs to include testing for simple (one-phase) versus two-phase logistic regression models. In this article, we propose and examine a class of likelihood ratio type tests for detecting a change in logistic regression parameters that splits the model into two-phases. We show that the proposed tests, based on Shiryayev–Roberts type statistics, are on average the most powerful. The article argues in favor of a new approach for fixing Type I errors of tests when the parameters of null hypotheses are unknown. Although the suggested approach is partly based on Bayes–Factor-type testing procedures, the classical significance levels of the proposed tests are under control. We demonstrate applications of the average most powerful tests to an epidemiologic study entitled “Time to pregnancy and multiple births.”     

Estimating the Proportion of True Null Hypotheses in Nonparametric Exponential Mixture Model with Appication to the Leukemia Gene Expression Data

We revisit the problem of estimating the proportion π of true null hypotheses where a large scale of parallel hypothesis tests are performed independently. While the proportion is a quantity of interest in its own right in applications, the problem has arisen in assessing or controlling an overall false discovery rate. On the basis of a Bayes interpretation of the problem, the marginal distribution of the p-value is modeled in a mixture of the uniform distribution (null) and a non-uniform distribution (alternative), so that the parameter π of interest is characterized as the mixing proportion of the uniform component on the mixture. In this article, a nonparametric exponential mixture model is proposed to fit the p-values. As an alternative approach to the convex decreasing mixture model, the exponential mixture model has the advantages of identifiability, flexibility, and regularity. A computation algorithm is developed. The new approach is applied to a leukemia gene expression data set where multiple significance tests over 3,051 genes are performed. The new estimate for π with the leukemia gene expression data appears to be about 10% lower than the other three estimates that are known to be conservative. Simulation results also show that the new estimate is usually lower and has smaller bias than the other three estimates.     

Efficient and easy-to-use sample size formulas in ratio-based non-inferiority tests

In many biomedical applications, tests for the classical hypotheses based on the difference of treatment means in a one-way layout can be replaced by tests for ratios (or tests for relative changes). This approach is well noted for its simplicity in defining the margins, as for example in tests for non-inferiority. Here, we derive approximate and efficient sample size formulas in a multiple testing situation and then thoroughly investigate the relative performance of hypothesis testing based on the ratios of treatment means when compared with differences of means. The results will be illustrated with an example on simultaneous tests for non-inferiority.     

Serial P-values

When a collection of hypotheses is to be tested it is necessary to maintain a bound on the simultaneous Type I error rate. Serial P-values are used to define a serial test that does provide such a bound. Moreover, serial P-values are meaningful in the context of multiple tests, with or without the ‘rejection-confirmation’ decisions. The method is particularly suited to the analysis of unbalanced data, especially contingency tables.     

The split sample permutation t-tests

Without the exchangeability assumption, permutation tests for comparing two population means do not provide exact control of the probability of making a Type I error. Another drawback of permutation tests is that it cannot be used to test hypothesis about one population. In this paper, we propose a new type of permutation tests for testing the difference between two population means: the split sample permutation t-tests. We show that the split sample permutation t-tests do not require the exchangeability assumption, are asymptotically exact and can be easily extended to testing hypothesis about one population. Extensive simulations were carried out to evaluate the performance of two specific split sample permutation t-tests: the split in the middle permutation t-test and the split in the end permutation t-test. The simulation results show that the split in the middle permutation t-test has comparable performance to the permutation test if the population distributions are symmetric and satisfy the exchangeability assumption. Otherwise, the split in the end permutation t-test has significantly more accurate control of level of significance than the split in the middle permutation t-test and other existing permutation tests.     

Independence and sphericity tests for the residuals of space-time arma models

The mechanics of the procedure for building space-time autoregressive moving average (STARMA) models is dependent upon the form of G, the variance-covariance matrix of the underlying errors.This paper presents large sample tests of the hypotheses that G is diagonal and that G equals o² I. Tables of the critical values for these tests are constructed     

Powerful Parametric Tests Based on Sum‐Functions of Spacings

Assume that we have a sequence of n independent and identically distributed random variables with a continuous distribution function F, which is specified up to a few unknown parameters. In this paper, tests based on sum‐functions of sample spacings are proposed, and large sample theory of the tests are presented under simple null hypotheses as well as under close alternatives. Tests, which are optimal within this class, are constructed, and it is noted that these tests have properties that closely parallel those of the likelihood ratio test in regular parametric models. Some examples are given, which show that the proposed tests work also in situations where the likelihood ratio test breaks down. Extensions to more general hypotheses are discussed.     

Development of Specific hypothesis tests for estimated markov chains

We develop and evaluate the validity and power of two specific tests for the transition probabilities in a Markov chain estimated from aggregate frequency data. The two null hypotheses considered are (1) constancy of the diagonal elements of the one-step transition probability matrix and (2) an arbitrarily chosen transition probability’s being equal to a specific value. The formation of tests uses a general framework for statistical inference on estimated Markov processes; we also indicate how this framework can be used to form tests for a variety of other hypotheses. The validity and power performance of the two tests formed in this paper are examined in factorially designed Monte Carlo experiments. The results indicate that the proposed tests lead to type I error probabilities which are close to the desired levels and to high power against even small deviations from the null hypotheses considered.     

On the role of minque in testing of hypotheses under hiked linear models

Testing of hypotheses under balanced ANOVA models is fairly simple and generally based on the usual ANOVA sums of squares. Difficulties may arise in special cases when these sums of squares do not form a complete sufficient statistic. There is a huge literature on this subject which was recently surveyed in Seifert's contribution to the book of Mumak (1904). But there are only a few results about unbalanced models. In such models the consideration of likelihood ratios leads to more complex sums of squares known from MINQUE theory.

Uniform optimality of testsusually reduces to local optimality. Here we prespnt a small review of methods proposed for testing of hypotheses in unbalanced models. where MINQUEI playb a major role. We discuss the use of iterated MINQUE for the construction of asymptotically optimal tests described in Humak (1984) and approximate tests based on locally uncorrelated linear combinations of MINQUE estimators by Seifert (1985), We show that the latter tests coincide with robust locally optimal invariant tests proposeci by Kariya and Sinha and Das and Sinha, if the number of variance components is two. Explicit expressions for corresponding tests are given for the unbalanced two-way cross classification random model, which covers some other models as special cases. A simulation study under lines the relevance of MINQUE for testing of hypotheses problems.     

Permuiation tesis for k-sample binomial data with comparisons of exact and approximate P-levels

Exact ksample permutation tests for binary data for three commonly encountered hypotheses tests are presented,, The tests are derived both under the population and randomization models . The generating function for the number of cases in the null distribution is obtained, The asymptotic distributions of the test statistics are derived . Actual significance levels are computed for the asymptotic test versions , Random sampling of the null distribution is suggested as a superior alternative to the asymptotics and an efficient computer technique for implementing the random sampling is described., finally, some numerical examples are presented and sample size guidelines given for computer implementation of the exact tests.     

Bayesian Analysis of Contingency Tables

ABSTRACT

The display of the data by means of contingency tables is used in different approaches to statistical inference, for example, to broach the test of homogeneity of independent multinomial distributions. We develop a Bayesian procedure to test simple null hypotheses versus bilateral alternatives in contingency tables. Given independent samples of two binomial distributions and taking a mixed prior distribution, we calculate the posterior probability that the proportion of successes in the first population is the same as in the second. This posterior probability is compared with the p-value of the classical method, obtaining a reconciliation between both results, classical and Bayesian. The obtained results are generalized for r × s tables.     

Parametric bootstrap and approximate tests for two Poisson variates

The parametric bootstrap tests and the asymptotic or approximate tests for detecting difference of two Poisson means are compared. The test statistics used are the Wald statistics with and without log-transformation, the Cox F statistic and the likelihood ratio statistic. It is found that the type I error rate of an asymptotic/approximate test may deviate too much from the nominal significance level α under some situations. It is recommended that we should use the parametric bootstrap tests, under which the four test statistics are similarly powerful and their type I error rates are all close to α. We apply the tests to breast cancer data and injurious motor vehicle crash data.     

Interval estimation for conformance proportions of multiple quality characteristics

A conformance proportion is an important and useful index to assess industrial quality improvement. Statistical confidence limits for a conformance proportion are usually required not only to perform statistical significance tests, but also to provide useful information for determining practical significance. In this article, we propose approaches for constructing statistical confidence limits for a conformance proportion of multiple quality characteristics. Under the assumption that the variables of interest are distributed with a multivariate normal distribution, we develop an approach based on the concept of a fiducial generalized pivotal quantity (FGPQ). Without any distribution assumption on the variables, we apply some confidence interval construction methods for the conformance proportion by treating it as the probability of a success in a binomial distribution. The performance of the proposed methods is evaluated through detailed simulation studies. The results reveal that the simulated coverage probability (cp) for the FGPQ-based method is generally larger than the claimed value. On the other hand, one of the binomial distribution-based methods, that is, the standard method suggested in classical textbooks, appears to have smaller simulated cps than the nominal level. Two alternatives to the standard method are found to maintain their simulated cps sufficiently close to the claimed level, and hence their performances are judged to be satisfactory. In addition, three examples are given to illustrate the application of the proposed methods.     

Bayesian testing of an exponential point null hypothesis

The problem of testing a point null hypothesis involving an exponential mean is The problem of testing a point null hypothesis involving an exponential mean is usual interpretation of P-values as evidence against precise hypotheses is faulty. As in Berger and Delampady (1986) and Berger and Sellke (1987), lower bounds on Bayesian measures of evidence over wide classes of priors are found emphasizing the conflict between posterior probabilities and P-values. A hierarchical Bayes approach is also considered as an alternative to computing lower bounds and “automatic” Bayesian significance tests which further illustrates the point that P-values are highly misleading measures of evidence for tests of point null hypotheses.     

Comment

Bonferroni inequalities often provide tight upper and lower bounds for the probability of a union of events. The bounds are especially useful when this probability is difficult to compute exactly. There are situations, however, in which the Bonferroni approach gives very poor results. An example is given in which the upper and lower Bonferroni bounds are far from the probability they seek to approximate and successive bounds do not converge. Even an improved first upper Bonferroni bound may not be close to the probability of the union of events.     

Asyhptotic efficiency of sequential and non-sequential procedures based on fisher's method of combining tests

In this paper we propose test statistics based on Fisher's method of combining tests for hypotheses involving two or more parameters simultaneously, It Is shown that these tests are asymptotically efficient In the sense of Bahadur, It is then shown how these tests can be modified to give sequential test procedures which are efficient in the sense of Berk and Brown (1978).

The results in section 3 generalize the work of Perng (1977) and Durairajan (1980).