A fiducial approach to multiple comparisons

Comparing treatment means from populations that follow independent normal distributions is a common statistical problem. Many frequentist solutions exist to test for significant differences amongst the treatment means. A different approach would be to determine how likely it is that particular means are grouped as equal. We developed a fiducial framework for this situation. Our method provides fiducial probabilities that any number of means are equal based on the data and the assumed normal distributions. This methodology was developed when there is constant and non-constant variance across populations. Simulations suggest that our method selects the correct grouping of means at a relatively high rate for small sample sizes and asymptotic calculations demonstrate good properties. Additionally, we have demonstrated the flexibility in the methods ability to calculate the fiducial probability for any number of equal means. This was done by analyzing a simulated data set and a data set measuring the nitrogen levels of red clover plants that were inoculated with different treatments.     

New monte carlo results on the robustness of the anova f,w and f statistics

Because the usual F test for equal means is not robust to unequal variances, Brown and Forsythe (1974a) suggest replacing F with the statistics F or W which are based on the Satterthwaite and Welch adjusted degrees of freedom procedures. This paper reports practical situations where both F and W give * unsatisfactory results. In particular, both F and W may not provide adequate control over Type I errors. Moreover, for equal variances, but unequal sample sizes, W should be avoided in favor of F (or F ), but for equal sample sizes, and possibly unequal variances, W was the only satisfactory statistic. New results on power are included as well. The paper also considers the effect of using F or W only after a significant test for equal variances has been obtained, and new results on the robustness of the F test are described. It is found that even for equal sample sizes as large as 50 per treatment group, there are practical situations where the F test does not provide adequately control over the probability of a Type I error.     

Operating Characteristics for Group Sequential Trials Monitored Under Fractional Brownian Motion

Brownian motion has been used to derive stopping boundaries for group sequential trials, however, when we observe dependent increment in the data, fractional Brownian motion is an alternative to be considered to model such data. In this article we compared expected sample sizes and stopping times for different stopping boundaries based on the power family alpha spending function under various values of Hurst coefficient. Results showed that the expected sample sizes and stopping times will decrease and power increases when the Hurst coefficient increases. With same Hurst coefficient, the closer the boundaries are to that of O'Brien-Fleming, the higher the expected sample sizes and stopping times are; however, power has a decreasing trend for values start from H = 0.6 (early analysis), 0.7 (equal space), 0.8 (late analysis). We also illustrate study design changes using results from the BHAT study.     

P-values for two-sided comparisons with a control

Critical values for Dunnett's multiple comparison procedure for simultaneously comparing the means of several experimental treatments with the mean of a control treatment are available in many references. However, these values are primarily applicable to one-way analysis of variance with equal sample sizes for the experimental treatments. Even for this design, these critical values do not suffice for situations where the corresponding p-values are desired. Here the technique is presented for calculation of the p-value for multiple comparisons with a control when a general linear model is used for the analysis. A FORTRAN program using IMSL subroutines is described for the calculations.     

TEST FOR TREATMENT EFFECT BASED ON BINARY DATA WITH RANDOM SAMPLE SIZES

The problem of testing for treatment effect based on binary response data is considered, assuming that the sample size for each experimental unit and treatment combination is random. It is assumed that the sample size follows a distribution that belongs to a parametric family. The uniformly most powerful unbiased tests, which are equivalent to the likelihood ratio tests, are obtained when the probability of the sample size being zero is positive. For the situation where the sample sizes are always positive, the likelihood ratio tests are derived. These test procedures, which are unconditional on the random sample sizes, are useful even when the random sample sizes are not observed. Some examples are presented as illustration.     

Distances between normal populations when covariance matrices are unequal

The definition of distance between two populations of equal covariance matrices is extended to two and more than two populations with unequal covariance matrices and Rao’s U test for testing the conditional contribution of a subset of variables to the distance is extended to this situation, even when sample sizes are not necessarily the same.     

A test for independence in a multivariate exponential distribution with equal correlation coefficients

A rank statistic is considered which may be used for testing for total independence in a p-variate exponential distribution with equal correlation coefficients. Critical values for the statistic are provided for p = 3.4 and sample sizes less than or equal to 20. Finally, the small sample power performance of the rank test relative to that of the locally most powerful similar lest under the exponential alternative is evaluated.     

Testing for equivalence of variances using Hartley's ratio

Hartley's test for homogeneity of k normal‐distribution variances is based on the ratio between the maximum sample variance and the minimum sample variance. In this paper, the author uses the same statistic to test for equivalence of k variances. Equivalence is defined in terms of the ratio between the maximum and minimum population variances, and one concludes equivalence when Hartley's ratio is small. Exact critical values for this test are obtained by using an integral expression for the power function and some theoretical results about the power function. These exact critical values are available both when sample sizes are equal and when sample sizes are unequal. One related result in the paper is that Hartley's test for homogeneity of variances is no longer unbiased when the sample sizes are unequal. The Canadian Journal of Statistics 38: 647–664; 2010 © 2010 Statistical Society of Canada     

Dominance refinements of the Smirnov two-sample test

We prove the following conjecture of Narayana: there are no nontrivial dominance refinements of the Smirnov two-sample test if and only if the two sample sizes are relatively prime. We also count the number of natural significance levels of the Smirnov two-sample test in terms of the sample sizes and relate this to the Narayana conjecture. In particular, Smirnov tests with relatively prime sample sizes turn out to have many more natural significance levels than do Smirnov tests whose sample sizes are not relatively prime (for example, equal sample sizes).     

The Exact Null Distribution of Bartlett's Criterion

In this paper the exact null distribution of Bartlett's criterion for testing the homogeneity of variances in normal samples with unequal sizes is derived. The most general form of the density function is obtained by using contour integration. The expression for the cumulative distribution, being a series in simple algebraic functions, seems quite tractable for computation of the exact critical values. In the special case of equal sample sizes, some indication of the relation of the work of others to our series expansions has also been given.     

Large-sample quantile estimation in pareto laws

A large-sample method of estimation for the parameters of Pareto laws is investigatedo The estimates are derived by using a small subset of k sample quantiles out of the original observations. The optimum spacing of the k quantiles is also examined. A Monte Carlo study compares this method with the method of moments and that of maximum likelihood for a selected set of parameter values and sample sizes.     

Sample size recalculation for binary data in internal pilot study designs

For binary endpoints, the required sample size depends not only on the known values of significance level, power and clinically relevant difference but also on the overall event rate. However, the overall event rate may vary considerably between studies and, as a consequence, the assumptions made in the planning phase on this nuisance parameter are to a great extent uncertain. The internal pilot study design is an appealing strategy to deal with this problem. Here, the overall event probability is estimated during the ongoing trial based on the pooled data of both treatment groups and, if necessary, the sample size is adjusted accordingly. From a regulatory viewpoint, besides preserving blindness it is required that eventual consequences for the Type I error rate should be explained. We present analytical computations of the actual Type I error rate for the internal pilot study design with binary endpoints and compare them with the actual level of the chi‐square test for the fixed sample size design. A method is given that permits control of the specified significance level for the chi‐square test under blinded sample size recalculation. Furthermore, the properties of the procedure with respect to power and expected sample size are assessed. Throughout the paper, both the situation of equal sample size per group and unequal allocation ratio are considered. The method is illustrated with application to a clinical trial in depression. Copyright © 2004 John Wiley & Sons Ltd.     

A Note on Sample Size Determination for Akaike Information Criterion (AIC) Approach to Clinical Data Analysis

ABSTRACT

Because of its flexibility and usefulness, Akaike Information Criterion (AIC) has been widely used for clinical data analysis. In general, however, AIC is used without paying much attention to sample size. If sample sizes are not large enough, it is possible that the AIC approach does not lead us to the conclusions which we seek. This article focuses on the sample size determination for AIC approach to clinical data analysis. We consider a situation in which outcome variables are dichotomous and propose a method for sample size determination under this situation. The basic idea is also applicable to the situations in which outcome variables have more than two categories or outcome variables are continuous. We present simulation studies and an application to an actual clinical trial.     

Minimal neighbor designs in circular blocks of unequal sizes

Neighbor designs have their own importance in the experiments to remove the neighbor effects where the performance of a treatment is affected by the treatments applied to its adjacent plots. If each pair of distinct treatments appears exactly once as neighbors, neighbor designs are called minimal. Most of the neighbor designs require a large number of blocks of equal sizes. In this situation minimal neighbor designs in unequal block sizes are preferred to reduce the experimental material. In this article some series are presented to construct minimal neighbor designs in circular blocks of unequal sizes.     

A Bayesian sequential design with adaptive randomization for 2‐sided hypothesis test

Bayesian sequential and adaptive randomization designs are gaining popularity in clinical trials thanks to their potentials to reduce the number of required participants and save resources. We propose a Bayesian sequential design with adaptive randomization rates so as to more efficiently attribute newly recruited patients to different treatment arms. In this paper, we consider 2‐arm clinical trials. Patients are allocated to the 2 arms with a randomization rate to achieve minimum variance for the test statistic. Algorithms are presented to calculate the optimal randomization rate, critical values, and power for the proposed design. Sensitivity analysis is implemented to check the influence on design by changing the prior distributions. Simulation studies are applied to compare the proposed method and traditional methods in terms of power and actual sample sizes. Simulations show that, when total sample size is fixed, the proposed design can obtain greater power and/or cost smaller actual sample size than the traditional Bayesian sequential design. Finally, we apply the proposed method to a real data set and compare the results with the Bayesian sequential design without adaptive randomization in terms of sample sizes. The proposed method can further reduce required sample size.     

Comparison of Three Sampling Designs

Three sampling designs are considered for estimating the sum of k population means by the sum of the corresponding sample means. These are (a) the optimal design; (b) equal sample sizes from all populations; and (c) sample sizes that render equal variances to all sample means. Designs (b) and (c) are equally inefficient, and may yield a variance up to k times as large as that of (a). Similar results are true when the cost of sampling is introduced, and they depend on the population sampled.     

Optimality of Equal vs. Unequal Cluster Sizes in Multilevel Intervention Studies: A Monte Carlo Study for Small Sample Sizes

Optimality of equal versus unequal cluster sizes in the context of multilevel intervention studies is examined. A Monte Carlo study is done to examine to what degree asymptotic results on the optimality hold for realistic sample sizes and for different estimation methods. The relative D-criterion, comparing equal versus unequal cluster sizes, almost always exceeded 85%, implying that loss of information due to unequal cluster sizes can be compensated for by increasing the number of clusters by 18%. The simulation results are in line with asymptotic results, showing that, for realistic sample sizes and various estimation methods, the asymptotic results can be used in planning multilevel intervention studies.     

Admissible two-stage designs for phase II cancer clinical trials that incorporate the expected sample size under the alternative hypothesis

two‐stage studies may be chosen optimally by minimising a single characteristic like the maximum sample size. However, given that an investigator will initially select a null treatment e?ect and the clinically relevant di?erence, it is better to choose a design that also considers the expected sample size for each of these values. The maximum sample size and the two expected sample sizes are here combined to produce an expected loss function to ?nd designs that are admissible. Given the prior odds of success and the importance of the total sample size, minimising the expected loss gives the optimal design for this situation. A novel triangular graph to represent the admissible designs helps guide the decision‐making process. The H ₀‐optimal, H ₁‐optimal, H ₀‐minimax and H ₁‐minimax designs are all particular cases of admissible designs. The commonly used H ₀‐optimal design is rarely good when allowing stopping for e?cacy. Additionally, the δ‐minimax design, which minimises the maximum expected sample size, is sometimes admissible under the loss function. However, the results can be varied and each situation will require the evaluation of all the admissible designs. Software to do this is provided. Copyright © 2012 John Wiley & Sons, Ltd.     

Two-moment approximations to some null distributions in order-restricted inference: Unequal sample sizes

Bartholomew's statistics for testing homogeneity of normal means with ordered alternatives have null distributions which are mixtures of chi-squared or beta distributions according as the variances are known or not. If the sample sizes are not equal, the mixing coefficients can be difficult to compute. For a simple order and a simple tree ordering, approximations to the significance levels of these tests have been developed which are based on patterns in the weight sets. However, for a moderate or large number of means, these approximations can be tedious to implement. Employing the same approach that was used in the development of these approximations, two-moment chisquared and beta approximations are derived for these significance levels. Approximations are also developed for the testing situation in which the order restriction is the null hypothesis. Numerical studies show that in each of the cases the two-moment approximation is quite satisfactory for most practical purposes.     

Additional Tables for Steel–Dwass–Critchlow–Fligner Distribution-Free Multiple Comparisons of Three Treatments

ABSTRACT

Additional critical points are presented for the Steel–Dwass–Critchlow–Fligner distribution-free multiple comparison procedure for comparing all pairs of three population medians in the one-way layout. A computational technique developed by van de Wiel is used to find critical points yielding an experimentwise error rate of approximately 0.01, 0.05, and 0.10 for a total sample size of at most 30, with individual sample sizes from 4 to 10 and a maximum sample size of at least 8, and for equal sample sizes from 8 to 14. Additional discussion is given regarding step-down testing methods and the dangers of using the Steel–Dwass–Critchlow–Fligner procedure with unequal sample sizes if two of the sample sizes are very small.