Gatekeeping procedures with clinical trial applications

The objective of this paper is to give an overview of a relatively new area of multiplicity research that deals with the analysis of hierarchically ordered multiple objectives. Testing procedures for this problem are known as gatekeeping procedures and have found a variety of applications in clinical trials. This paper reviews main classes of these procedures, including serial and parallel gatekeeping procedures, and tree gatekeeping procedures that account for logical restrictions among multiple objectives. We focus on procedures based on marginal p-values; extensions to procedures that exploit the joint distribution of the p-values are also noted. Clinical trial examples are used to illustrate the procedures and their important properties.     

Tests for a simple tree order restriction with application to dose–response studies

Summary.  We propose 'Dunnett-type' test procedures to test for simple tree order restrictions on the means of p independent normal populations. The new tests are based on the estimation procedures that were introduced by Hwang and Peddada and later by Dunbar, Conaway and Peddada. The procedures proposed are also extended to test for 'two-sided' simple tree order restrictions. For non-normal data, nonparametric versions based on ranked data are also suggested. Using computer simulations, we compare the proposed test procedures with some existing test procedures in terms of size and power. Our simulation study suggests that the procedures compete well with the existing procedures for both one-sided and two-sided simple tree alternatives. In some instances, especially in the case of two-sided alternatives or for non-normally distributed data, the gains in power due to the procedures proposed can be substantial.     

Closed testing procedures for all pairwise comparisons in a randomized block design

We consider multiple comparison test procedures among treatment effects in a randomized block design. We propose closed testing procedures based on maximum values of some two-sample t test statistics and based on F test statistics. It is shown that the proposed procedures are more powerful than single-step procedures and the REGW (Ryan/Einot–Gabriel/Welsch)-type tests. Next, we consider the randomized block design under simple ordered restrictions of treatment effects. We propose closed testing procedures based on maximum values of two-sample one-sided t test statistics and based on Batholomew’s statistics for all pairwise comparisons of treatment effects. Although single-step multiple comparison procedures are utilized in general, the power of these procedures is low for a large number of groups. The closed testing procedures stated in the present article are more powerful than the single-step procedures. Simulation studies are performed under the null hypothesis and some alternative hypotheses. In this studies, the proposed procedures show a good performance.     

Maximum of the weighted Kaplan–Meier tests for the two-sample censored data

In this paper, some versatile test procedures are considered, which are useful for the case where the association of survival functions is unclear. These procedures are based on a maximum of a class of the weighted Kaplan–Meier statistics. The weight functions used in the procedures account for both the censored and non-censored data points. Numerical simulations with these weight functions show that, under various levels of censoring, the proposed procedures perform well across a wide range of alternative configurations. Implementation of the proposed procedures is illustrated in a real data example.     

The effect of conditional heteroskedasticity on common statistical procedures for means and variances

Summary: Commonly used standard statistical procedures for means and variances (such as the t–test for means or the F–test for variances and related confidence procedures) require observations from independent and identically normally distributed variables. These procedures are often routinely applied to financial data, such as asset or currency returns, which do not share these properties. Instead, they are nonnormal and show conditional heteroskedasticity, hence they are dependent. We investigate the effect of conditional heteroskedasticity (as modelled by GARCH(1,1)) on the level of these tests and the coverage probability of the related confidence procedures. It can be seen that conditional heteroskedasticity has no effect on procedures for means (at least in large samples). There is, however, a strong effect of conditional heteroskedasticity on procedures for variances. These procedures should therefore not be used if conditional heteroskedasticity is prevalent in the data.*We are grateful to the referees for their useful and constructive comments.     

Sequential selectiok procedures based on confidence sequences for normal populations

In this paper confidence sequences are used to construct sequential procedures for selecting the population with the a common variance. These procedures are shown to provide substantial saving, particularly in the expected samplw sizes of the inferior populations,over various procedures in the literature. A new “indifference zone” formulation is given for the correct selection probability requirement, and confidence sequences are also applied to construct sequential procedures for this new selection goal.     

Generation of d–optimal designs in a finite design space

This paper presents some considerations about the numerical procedures for generating D–optimal design in a finite design space. The influence of starting procedures and the finite set of points on the design efficiency is considered. Some modifications of the existing procedures for D–optimal designs generation are described. It is shown that for large number of factors the sequential procedures are more appropriate than the nonsequential ones     

Robust tests in nonlinear regression models

Robust tests for testing subhypotheses in nonlinear models are developed. These are drop-in-dispersion testing procedures, score-type and Wald-type testing procedures. The asymptotic properties and influence functions are obtained. Robust tests that perform well in the presence of heteroscedasticity are also developed. Simulation results are provided to illustrate these procedures.     

Selection of the best with a preliminary test for two-parameter exponential distributions

This paper deals with the problem of selecting the best population from among k(≥ 2) two-parameter exponential populations. New selection procedures are proposed for selecting the unique best. The procedures include preliminary tests which allow the xperimenter to have an option to not select if the statistical evidence is not significant. Two probabilities, the probability to make a selection and the probability of a correct selection, are controlled by these selection procedures. Comparisons between the proposed selection procedures and certain earlier existing procedures are also made. The results show the superiority of the proposed selection procedures in terms of the required sample size.     

Closure properties of classes of multiple testing procedures

Statistical discoveries are often obtained through multiple hypothesis testing. A variety of procedures exists to evaluate multiple hypotheses, for instance the ones of Benjamini–Hochberg, Bonferroni, Holm or Sidak. We are particularly interested in multiple testing procedures with two desired properties: (solely) monotonic and well-behaved procedures. This article investigates to which extent the classes of (monotonic or well-behaved) multiple testing procedures, in particular the subclasses of so-called step-up and step-down procedures, are closed under basic set operations, specifically the union, intersection, difference and the complement of sets of rejected or non-rejected hypotheses. The present article proves two main results: First, taking the union or intersection of arbitrary (monotonic or well-behaved) multiple testing procedures results in new procedures which are monotonic but not well-behaved, whereas the complement or difference generally preserves neither property. Second, the two classes of (solely monotonic or well-behaved) step-up and step-down procedures are closed under taking the union or intersection, but not the complement or difference.     

Adaptive sequential procedures for comparing new treatments with a standard

In this paper we propose and study two sequential elimination procedures for selecting all new treatments better than a standard or control treatment. These procedures differ from those previously proposed in that we assume variances are unequal and unknown. Expressions for asymptotic expected sample sizes are given. Confidence intervals associated with the procedures are also discussed.     

Multiple comparisons of several populations with more than one control with respectto scale parameter

In this paper, one-sided and two-sided test procedures for comparing several treatments with more than one control with respect to scale parameter are proposed. The proposed test procedures are inverted to obtain the associated simultaneous confidence intervals. The multiple comparisons of test treatments with the best control are also developed. The computation of the critical points, required to implement the proposed procedures, is discussed by taking the normal probability model. Applications of the proposed test procedures to two-parameter exponential probability model are also demonstrated.     

On the Skew-Normal-Cauchy Distribution

A fixed effects one-way layout model of analysis of variance is considered where the variances are taken to be possibly unequal. Conservative single-stage procedures based on Banerjee’s method for the solution of the Behrens-Fisher problem are proposed for the following multiple comparisons problems: 1) all pairwise comparisons with a control population mean, and 2) all pairwise comparisons and all linear contrasts among the means. Since these procedures are likely to be very conservative in practice, approximate procedures based on Welch’s method for the solution of the Behrens-Fisher problem are suggested as alternatives. Monte Carlo studies indicate that the latter are much less conservative and hence may be better in practice. Both these sets of procedures need only the tables of the Student’s t-distribution for their application and are very simple to use. Exact two-stage procedures are proposed for the following multiple comparisons problems: 1) all pairwise comparisons and all linear contrasts among the means, and 2) all linear combinations of the means.     

Selecting all treatments better than a control using existing tables

In this paper subset selection procedures for selecting all treatment populations with means larger than a control population are proposed. The treatments and control are assumed to have a multivariate normal distribution. Various covariance structures are considered. All of the proposed procedures are easily implemented using existing tables of the multivariate normal and multivariate t distributions. Some other procedures which have been proposed require extensive and unavailable tables for their implementation     

Multiple comparisons with a control in repeated measures incomplete block designs using R-estimators

New multiple comparison with a control (MCC) procedures are developed in repeated measures incomplete block design settings based on R-estimates. It is assumed that the errors within each subject are exchangeable random variables. The R-estimators of the treatment effects are obtained by minimizing a sum of Jaeckel (1972)-type dispersion functions. Based on the R-estimators, Dunnett-type multiple comparison procedures are developed for comparing test-treatments with a control-treatment. Under exchangeable errors, it is demonstrated that for Cox-type designs, the new procedures are more efficient than the existing nonparametric procedures. The new MCC procedures are applied to a data set in a clinical trial which consists of patients with reversible obstructive pulmonary disease.     

One-sided sign-type non-parametric procedures for comparing treatments with a control in a randomized complete block design

SUMMARY Non-parametric procedures are presented for comparing several treatments with a control when the data are collected in a randomized complete block design with no interaction. The procedures are generalizations of some well-known sign-type tests, and include both overall tests and multiple comparisons procedures. A numerical example is used to motivate the problem and illustrate the proposed methods. Some concluding remarks are offered.     

Subset selection in two-factor experiments using randomization restricted designs

This paper studies subset selection procedures for screening in two-factor treatment designs that employ either a split-plot or strip-plot randomization restricted experimental design laid out in blocks. The goal is to select a subset of treatment combinations associated with the largest mean. In the split-plot design, it is assumed that the block effects, the confounding effects (whole-plot error) and the measurement errors are normally distributed. None of the selection procedures developed depend on the block variances. Subset selection procedures are given for both the case of additive and non-additive factors and for a variety of circumstances concerning the confounding effect and measurement error variances. In particular, procedures are given for (1) known confounding effect and measurement error variances (2) unknown measurement error variance but known confounding effect (3) unknown confounding effect and measurement error variances. The constants required to implement the procedures are shown to be obtainable from available FORTRAN programs and tables. Generalization to the case of strip-plot randomization restriction is considered.     

Bootstrap for finite populations

The bootstrap method is compared with the classical (linearization) and jackknife procedures for estimating the mean square errors (MSEs) of the ratio estimator and the combined ratio estimator. The initial samples are considered to be selected without replacement, and different procedures for selecting the bootstrap samples with or without replacement from them are examined. The biases, stabilities, coverage probabilities and confidence widths of all the procedures are compared.     

Optimal transitive procedures for comparing large numbers of parameters

Independent random samples are selected from each of a set of N independent populations, P₁,…,P_n. Interest centers around comparing N (unknown) scalar parameters θ₁,…,θ_N associated respectively with the N populations P₁,…,P_n. Procedures are constructed for estimating the magnitude of each of the differences δ_t,j = θ_i ? θ_j (1 ≤ i,j ≤ N) between pairs of populations. A loss function which adopts appropriate penalties for magnitude errors in estimation of differences is constructed. Magnitude estimators of differences are called transitive if they give rise to a transitive (i.e., consistent) relationship between pairwise differences of parameters. We show how to construct optimal effcient transitive magnitude–estimation procedures and demonstrate their usefulness through an example involving estimating the magnitude of the differences between disease incidence in paired towns for different pairs. Optimal transitive pairwise–comparison procedures are optimum (i.e., have the smallest posterior Bayes risks) in the class of all transitive pairwise–comparison procedures; as such they replace classical Bayes procedures which are usually not transitive when the number N of parameters compared is large. The posterior Bayes risk of optimal transitive pairwise comparison procedures are compared with that for alternative ‘adapted’ procedures, constructed from optimal simultaneous estimators and adapted for the purpose of pairwise comparisons. It is shown that the optimal transitive pairwise comparison procedures dominated the adapted procedures (in posterior Bayes risk) and typically represent only a small increase in posterior risk over the classical Bayes procedures which generally fail to be consistent. Optimal Bayes procedures are shown, for large numbers of parameters to be reasonably easy to construct using the algorithms outlined in this paper     

Testing equality of ratios from two finite populations

Approximations are given for the bias and variance of both the regression and ratio estimator when sampling from a finite population, and simulation results are given indicating the accuracy of the approximations and the bias of the estimated approximations. A different estimator for the variance of the regression estimator is recommended. Test procedures are proposed for testing the hypothesis of equality of ratios from two finite populations, the procedures depending upon the types of populations being sampled. Simulation results indicating the effectiveness of the test procedures in controlling their size are given.