A combination multiple comparisons and subset selection procedure to identify treatments that are strictly inferior to the best

This paper considers the problem of identifying which treatments are strictly worse than the best treatment or treatments in a one-way layout, which has many important applications in screening trials for new product development. A procedure is proposed that selects a subset of the treatments containing only treatments that are known to be strictly worse than the best treatment or treatments. In addition, simultaneous confidence intervals are obtained which provide upper bounds on how inferior the treatments are compared with these best treatments. In this way, the new procedure shares the characteristics of both subset selection procedures and multiple comparison procedures. Some tables of critical points are provided for implementing the new procedure, and some examples of its use are given.     

A generalized step-up-down multiple test procedure

A generalization of step-up and step-down multiple test procedures is proposed. This step-up-down procedure is useful when the objective is to reject a specified minimum number, q, out of a family of k hypotheses. If this basic objective is met at the first step, then it proceeds in a step-down manner to see if more than q hypotheses can be rejected. Otherwise it proceeds in a step-up manner to see if some number less than q hypotheses can be rejected. The usual step-down procedure is the special case where q = 1, and the usual step-up procedure is the special case where q = k. Analytical and numerical comparisons between the powers of the step-up-down procedures with different choices of q are made to see how these powers depend on the actual number of false hypotheses. Examples of application include comparing the efficacy of a treatment to a control for multiple endpoints and testing the sensitivity of a clinical trial for comparing the efficacy of a new treatment with a set of standard treatments.     

Tests for a simple tree order restriction with application to dose–response studies

Summary.  We propose 'Dunnett-type' test procedures to test for simple tree order restrictions on the means of p independent normal populations. The new tests are based on the estimation procedures that were introduced by Hwang and Peddada and later by Dunbar, Conaway and Peddada. The procedures proposed are also extended to test for 'two-sided' simple tree order restrictions. For non-normal data, nonparametric versions based on ranked data are also suggested. Using computer simulations, we compare the proposed test procedures with some existing test procedures in terms of size and power. Our simulation study suggests that the procedures compete well with the existing procedures for both one-sided and two-sided simple tree alternatives. In some instances, especially in the case of two-sided alternatives or for non-normally distributed data, the gains in power due to the procedures proposed can be substantial.     

Adaptive graph‐based multiple testing procedures

Multiple testing procedures defined by directed, weighted graphs have recently been proposed as an intuitive visual tool for constructing multiple testing strategies that reflect the often complex contextual relations between hypotheses in clinical trials. Many well‐known sequentially rejective tests, such as (parallel) gatekeeping tests or hierarchical testing procedures are special cases of the graph based tests. We generalize these graph‐based multiple testing procedures to adaptive trial designs with an interim analysis. These designs permit mid‐trial design modifications based on unblinded interim data as well as external information, while providing strong family wise error rate control. To maintain the familywise error rate, it is not required to prespecify the adaption rule in detail. Because the adaptive test does not require knowledge of the multivariate distribution of test statistics, it is applicable in a wide range of scenarios including trials with multiple treatment comparisons, endpoints or subgroups, or combinations thereof. Examples of adaptations are dropping of treatment arms, selection of subpopulations, and sample size reassessment. If, in the interim analysis, it is decided to continue the trial as planned, the adaptive test reduces to the originally planned multiple testing procedure. Only if adaptations are actually implemented, an adjusted test needs to be applied. The procedure is illustrated with a case study and its operating characteristics are investigated by simulations. Copyright © 2014 John Wiley & Sons, Ltd.     

Point estimates and confidence regions for sequential trials involving selection

A number of authors have proposed clinical trial designs involving the comparison of several experimental treatments with a control treatment in two or more stages. At the end of the first stage, the most promising experimental treatment is selected, and all other experimental treatments are dropped from the trial. Provided it is good enough, the selected experimental treatment is then compared with the control treatment in one or more subsequent stages. The analysis of data from such a trial is problematic because of the treatment selection and the possibility of stopping at interim analyses. These aspects lead to bias in the maximum-likelihood estimate of the advantage of the selected experimental treatment over the control and to inaccurate coverage for the associated confidence interval. In this paper, we evaluate the bias of the maximum-likelihood estimate and propose a bias-adjusted estimate. We also propose an approach to the construction of a confidence region for the vector of advantages of the experimental treatments over the control based on an ordering of the sample space. These regions are shown to have accurate coverage, although they are also shown to be necessarily unbounded. Confidence intervals for the advantage of the selected treatment are obtained from the confidence regions and are shown to have more accurate coverage than the standard confidence interval based upon the maximum-likelihood estimate and its asymptotic standard error.     

Multivariate analysis of variance with additional data from an unknown subset of the populations

In this paper we consider the problem of testing the means of k multivariate normal populations with additional data from an unknown subset of the k populations. The purpose of this research is to offer test procedures utilizing all the available data for the multivariate analysis of variance problem because the additional data may contain valuable information about the parameters of the k populations. The standard procedure uses only the data from identified populations. We provide a test using all available data based upon Hotelling' s generalized T²statistic. The power of this test is computed using Betz's approximation of Hotelling' s generalized T²statistic by an F-distribution. A comparison of the power of the test and the standard test procedure is also given.     

On the performance of some subset selection procedures

In this paper, we restrict attention to the problem of subset selection of normal populations. The approaches and results of some previous comparison studies of subset selection procedures are discussed briefly. And then the result of a new Monte Carlo study comparing the performance of two classical procedures and the Bayes procedure is presented.     

A unified sequential test procedure for simultaneous testing the equality of several binomial proportions to a specified standard

In this article, we propose a unified sequentially rejective test procedure for testing simultaneously the equality of several independent binomial proportions to a specified standard. The proposed test procedure is general enough to include some well-known multiple testing procedures such as the Ordinary Bonferroni procedure, Hochberg procedure and Rom procedure. It involves multiple tests of significance based on the simple binomial tests (exact or approximate) which can be easily found in many elementary standard statistics textbooks. Unlike the traditional Chi-square test of the overall hypothesis, the procedure can identify the subset of the binomial proportions, which are different from the prespecified standard with the control of the familywise type I error rate. Moreover, the power computation of the procedure is provided and the procedure is illustrated by two real examples from an ecological study and a carcinogenicity study.     

On selecting the treatment with the largest probability of survival

Suppose there are k(>= 2) treatments and each treatment is a Bernoulli process with binomial sampling. The problem of selecting a random-sized subset which contains the treatment with the largest survival probability (reliability or probability of success) is considered. Based on the ideas from both classical approaches and general Bayesian statistical decision approach, a new subset selection procedure is proposed to solve this kind of problem in both balanced and unbalanced designs. Comparing with the classical procedures, the proposed procedure has a significantly smaller selected subset. The optimal properties and performance of it were examined. The methods of selecting and fitting the priors and the results of Monte Carlo simulations on selected important cases are also studied.     

Power comparisons between the F-test,the studentised range test,and an optimal test of the equality of several normal means

The power assessment of tests of the equality of k normal means such as the k treatment means in a one-way fixed effects analysis of variance model is addressed. Power assessment is considered in terms of a constraint on the range of the treatment means. The power properties of the standard F-test and Studentised range test are compared with those of an optimal (minimax) test procedure, which is known to maximise power levels under this constraint. It is shown that the standard test procedures compare well with the optimal test procedure, and in particular, the Studentised range test is shown to be practically as good as optimal in this setting.     

Some multiple decision problems in analysis of variance

In most practical situations to which the analysis of variance tests are applied, they do not supply the information that the experimenter aims at. If, for example, in one-way ANOVA the hypothesis is rejected in actual application of the F-test, the resulting conclusion that the true means θ₁,…,θ_k are not all equal, would by itself usually be insufficient to satisfy the experimenter. In fact his problems would begin at this stage. The experimenter may desire to select the “best” population or a subset of the “good” populations; he may like to rank the populations in order of “goodness” or he may like to draw some other inferences about the parameters of interest.

The extensive literature on selection and ranking procedures depends heavily on the use of independence between populations (block, treatments, etc.) in the analysis of variance. In practical applications, it is desirable to drop this assumption or independence and consider cases more general than the normal.

In the present paper, we derive a method to construct optimal (in some sense) selection procedures to select a nonempty subset of the k populations containing the best population as ranked in terms of θ_i’s which control the size of the selected subset and which maximizes the minimum average probability of selecting the best. We also consider the usual selection procedures in one-way ANOVA based on the generalized least squares estimates and apply the method to two-way layout case. Some examples are discussed and some results on comparisons with other procedures are also obtained.     

Multiple comparisons of several populations with more than one control with respectto scale parameter

In this paper, one-sided and two-sided test procedures for comparing several treatments with more than one control with respect to scale parameter are proposed. The proposed test procedures are inverted to obtain the associated simultaneous confidence intervals. The multiple comparisons of test treatments with the best control are also developed. The computation of the critical points, required to implement the proposed procedures, is discussed by taking the normal probability model. Applications of the proposed test procedures to two-parameter exponential probability model are also demonstrated.     

Identifying the determinants of foreign direct investment: a data-specific model selection approach

In this paper, we examine the potential determinants of foreign direct investment. For this purpose, we apply new exact subset selection procedures, which are based on idealized assumptions, as well as their possibly more plausible empirical counterparts to an international data set to select the optimal set of predictors. Unlike the standard model selection procedures AIC and BIC, which penalize only the number of variables included in a model, and the subset selection procedures RIC and MRIC, which consider also the total number of available candidate variables, our data-specific procedures even take the correlation structure of all candidate variables into account. Our main focus is on a new procedure, which we have designed for situations where some of the potential predictors are certain to be included in the model. For a sample of 73 developing countries, this procedure selects only four variables, namely imports, net income from abroad, gross capital formation, and GDP per capita. An important secondary finding of our study is that the data-specific procedures, which are based on extensive simulations and are therefore very time-consuming, can be approximated reasonably well by the much simpler exact methods.     

Model Selection,Transformations and Variance Estimation in Nonlinear Regression

The results of analyzing experimental data using a parametric model may heavily depend on the chosen model for regression and variance functions, moreover also on a possibly underlying preliminary transformation of the variables. In this paper we propose and discuss a complex procedure which consists in a simultaneous selection of parametric regression and variance models from a relatively rich model class and of Box-Cox variable transformations by minimization of a cross-validation criterion. For this it is essential to introduce modifications of the standard cross-validation criterion adapted to each of the following objectives: 1. estimation of the unknown regression function, 2. prediction of future values of the response variable, 3. calibration or 4. estimation of some parameter with a certain meaning in the corresponding field of application. Our idea of a criterion oriented combination of procedures (which usually if applied, then in an independent or sequential way) is expected to lead to more accurate results. We show how the accuracy of the parameter estimators can be assessed by a “moment oriented bootstrap procedure", which is an essential modification of the “wild bootstrap” of Härdle and Mammen by use of more accurate variance estimates. This new procedure and its refinement by a bootstrap based pivot (“double bootstrap”) is also used for the construction of confidence, prediction and calibration intervals. Programs written in Splus which realize our strategy for nonlinear regression modelling and parameter estimation are described as well. The performance of the selected model is discussed, and the behaviour of the procedures is illustrated, e.g., by an application in radioimmunological assay.     

Type I error rate control in adaptive designs for confirmatory clinical trials with treatment selection at interim

Interest in confirmatory adaptive combined phase II/III studies with treatment selection has increased in the past few years. These studies start comparing several treatments with a control. One (or more) treatment(s) is then selected after the first stage based on the available information at an interim analysis, including interim data from the ongoing trial, external information and expert knowledge. Recruitment continues, but now only for the selected treatment(s) and the control, possibly in combination with a sample size reassessment. The final analysis of the selected treatment(s) includes the patients from both stages and is performed such that the overall Type I error rate is strictly controlled, thus providing confirmatory evidence of efficacy at the final analysis. In this paper we describe two approaches to control the Type I error rate in adaptive designs with sample size reassessment and/or treatment selection. The first method adjusts the critical value using a simulation-based approach, which incorporates the number of patients at an interim analysis, the true response rates, the treatment selection rule, etc. We discuss the underlying assumptions of simulation-based procedures and give several examples where the Type I error rate is not controlled if some of the assumptions are violated. The second method is an adaptive Bonferroni-Holm test procedure based on conditional error rates of the individual treatment-control comparisons. We show that this procedure controls the Type I error rate, even if a deviation from a pre-planned adaptation rule or the time point of such a decision is necessary.     

Subset selection in two-factor experiments using randomization restricted designs

This paper studies subset selection procedures for screening in two-factor treatment designs that employ either a split-plot or strip-plot randomization restricted experimental design laid out in blocks. The goal is to select a subset of treatment combinations associated with the largest mean. In the split-plot design, it is assumed that the block effects, the confounding effects (whole-plot error) and the measurement errors are normally distributed. None of the selection procedures developed depend on the block variances. Subset selection procedures are given for both the case of additive and non-additive factors and for a variety of circumstances concerning the confounding effect and measurement error variances. In particular, procedures are given for (1) known confounding effect and measurement error variances (2) unknown measurement error variance but known confounding effect (3) unknown confounding effect and measurement error variances. The constants required to implement the procedures are shown to be obtainable from available FORTRAN programs and tables. Generalization to the case of strip-plot randomization restriction is considered.     

A Five-Decision Testing Procedure to Infer the Value of a Unidimensional Parameter

ABSTRACT

A statistical test can be seen as a procedure to produce a decision based on observed data, where some decisions consist of rejecting a hypothesis (yielding a significant result) and some do not, and where one controls the probability to make a wrong rejection at some prespecified significance level. Whereas traditional hypothesis testing involves only two possible decisions (to reject or not a null hypothesis), Kaiser’s directional two-sided test as well as the more recently introduced testing procedure of Jones and Tukey, each equivalent to running two one-sided tests, involve three possible decisions to infer the value of a unidimensional parameter. The latter procedure assumes that a point null hypothesis is impossible (e.g., that two treatments cannot have exactly the same effect), allowing a gain of statistical power. There are, however, situations where a point hypothesis is indeed plausible, for example, when considering hypotheses derived from Einstein’s theories. In this article, we introduce a five-decision rule testing procedure, equivalent to running a traditional two-sided test in addition to two one-sided tests, which combines the advantages of the testing procedures of Kaiser (no assumption on a point hypothesis being impossible) and Jones and Tukey (higher power), allowing for a nonnegligible (typically 20%) reduction of the sample size needed to reach a given statistical power to get a significant result, compared to the traditional approach.     

Order Statistics from the Generalized Exponential Distribution and Applications

With linear dispersion effects, the standard factorial designs are not optimal estimation of a mean model. A sequential two-stage experimental design procedure has been proposed that first estimates the variance structure, and then uses the variance estimates and the variance optimality criterion to develop a second stage design that efficiency estimates the mean model. This procedure has been compared to an equal replicate design analyzed by ordinary least squares, and found to be a superior procedure in many situations.

However with small first stage sample sizes the variance estiamtes are not reliable, and hence an alternative procedure could be more beneficial. For this reason a Bayesian modification to the two-stage procedure is proposed which will combine the first stage variance estiamtes with some prior variance information that will produce a more efficient procedure. This Bayesian procedure will be compared to the non-Bayesian twostage procedure and to the two one-stage alternative procedures listed above. Finally, a recommendation will be made as to which procedure is preferred in certain situations.     

An Improved Method on Wilcoxon Rank Sum Test for Gene Selection from Microarray Experiments

Selecting a small subset out of the thousands of genes in microarray data is important for accurate classification of phenotypes. In this paper, we propose a flexible rank-based nonparametric procedure for gene selection from microarray data. In the method we propose a statistic for testing whether area under receiver operating characteristic curve (AUC) for each gene is equal to 0.5 allowing different variance for each gene. The contribution to this “single gene” statistic is the studentization of the empirical AUC, which takes into account the variances associated with each gene in the experiment. Delong et al. proposed a nonparametric procedure for calculating a consistent variance estimator of the AUC. We use their variance estimation technique to get a test statistic, and we focus on the primary step in the gene selection process, namely, the ranking of genes with respect to a statistical measure of differential expression. Two real datasets are analyzed to illustrate the methods and a simulation study is carried out to assess the relative performance of different statistical gene ranking measures. The work includes how to use the variance information to produce a list of significant targets and assess differential gene expressions under two conditions. The proposed method does not involve complicated formulas and does not require advanced programming skills. We conclude that the proposed methods offer useful analytical tools for identifying differentially expressed genes for further biological and clinical analysis.     

A Rejection Principle for Sequential Tests of Multiple Hypotheses Controlling Familywise Error Rates

We present a unifying approach to multiple testing procedures for sequential (or streaming) data by giving sufficient conditions for a sequential multiple testing procedure to control the familywise error rate (FWER). Together, we call these conditions a ‘rejection principle for sequential tests’, which we then apply to some existing sequential multiple testing procedures to give simplified understanding of their FWER control. Next, the principle is applied to derive two new sequential multiple testing procedures with provable FWER control, one for testing hypotheses in order and another for closed testing. Examples of these new procedures are given by applying them to a chromosome aberration data set and finding the maximum safe dose of a treatment.