Confidence intervals for the difference between independent binomial proportions: comparison using a graphical approach and moving averages

This paper uses graphical methods to illustrate and compare the coverage properties of a number of methods for calculating confidence intervals for the difference between two independent binomial proportions. We investigate both small‐sample and large‐sample properties of both two‐sided and one‐sided coverage, with an emphasis on asymptotic methods. In terms of aligning the smoothed coverage probability surface with the nominal confidence level, we find that the score‐based methods on the whole have the best two‐sided coverage, although they have slight deficiencies for confidence levels of 90% or lower. For an easily taught, hand‐calculated method, the Brown‐Li ‘Jeffreys’ method appears to perform reasonably well, and in most situations, it has better one‐sided coverage than the widely recommended alternatives. In general, we find that the one‐sided properties of many of the available methods are surprisingly poor. In fact, almost none of the existing asymptotic methods achieve equal coverage on both sides of the interval, even with large sample sizes, and consequently if used as a non‐inferiority test, the type I error rate (which is equal to the one‐sided non‐coverage probability) can be inflated. The only exception is the Gart‐Nam ‘skewness‐corrected’ method, which we express using modified notation in order to include a bias correction for improved small‐sample performance, and an optional continuity correction for those seeking more conservative coverage. Using a weighted average of two complementary methods, we also define a new hybrid method that almost matches the performance of the Gart‐Nam interval. Copyright © 2014 John Wiley & Sons, Ltd.     

Exact likelihood ratio and score confidence intervals for the binomial proportion

Many methods are available for computing a confidence interval for the binomial parameter, and these methods differ in their operating characteristics. It has been suggested in the literature that the use of the exact likelihood ratio (LR) confidence interval for the binomial proportion should be considered. This paper provides an evaluation of the operating characteristics of the two‐sided exact LR and exact score confidence intervals for the binomial proportion and compares these results to those for three other methods that also strictly maintain nominal coverage: Clopper‐Pearson, Blaker, and Casella. In addition, the operating characteristics of the two‐sided exact LR method and exact score method are compared with those of the corresponding asymptotic methods to investigate the adequacy of the asymptotic approximation. Copyright © 2013 John Wiley & Sons, Ltd.     

Confidence interval estimation for number of patient‐years needed to treat

The number of patient‐years needed to treat (N_PYNT), also called the event‐based number needed to treat, to avoid one additional exacerbation has been reported in recently published respiratory trials, but the confidence intervals are not routinely reported. The challenge of constructing confidence intervals for N_PYNT is due to the fact that exacerbation data or count data in general are usually analyzed using Poisson‐based models such as Poisson or negative binomial regression and the rate ratio is the natural metric for between‐treatment comparison, while N_PYNT is based on rate difference, which is not usually calculated for those models. Therefore, the variance estimates from these analysis models are directly related to the rate ratio rather than the rate difference. In this paper, we propose several methods to construct confidence intervals for the N_PYNT, assuming that the event rates are estimated using Poisson or negative binomial regression models. The coverage property of the confidence intervals constructed with these methods is assessed by simulations. Copyright © 2014 John Wiley & Sons, Ltd.     

Matching pseudocounts for interval estimation of binomial and Poisson parameters

ABSTRACT

For interval estimation of a binomial proportion and a Poisson mean, matching pseudocounts are derived, which give the one-sided Wald confidence intervals with second-order accuracy. The confidence intervals remove the bias of coverage probabilities given by the score confidence intervals. Partial poor behavior of the confidence intervals by the matching pseudocounts is corrected by hybrid methods using the score confidence interval depending on sample values.     

Inference for functions of parameters in discrete distributions based on fiducial approach: Binomial and Poisson cases

In this article, we propose a simple method of constructing confidence intervals for a function of binomial success probabilities and for a function of Poisson means. The method involves finding an approximate fiducial quantity (FQ) for the parameters of interest. A FQ for a function of several parameters can be obtained by substitution. For the binomial case, the fiducial approach is illustrated for constructing confidence intervals for the relative risk and the ratio of odds. Fiducial inferential procedures are also provided for estimating functions of several Poisson parameters. In particular, fiducial inferential approach is illustrated for interval estimating the ratio of two Poisson means and for a weighted sum of several Poisson means. Simple approximations to the distributions of the FQs are also given for some problems. The merits of the procedures are evaluated by comparing them with those of existing asymptotic methods with respect to coverage probabilities, and in some cases, expected widths. Comparison studies indicate that the fiducial confidence intervals are very satisfactory, and they are comparable or better than some available asymptotic methods. The fiducial method is easy to use and is applicable to find confidence intervals for many commonly used summary indices. Some examples are used to illustrate and compare the results of fiducial approach with those of other available asymptotic methods.     

Interval estimation for the scale parameter of the two-parameter exponential distribution based on time-censored data

This paper is concerned with developing procedures for construcing confidence intervals, which would hold approximately equal tail probabilities and coverage probabilities close to the normal, for the scale parameter θ of the two-parameter exponential lifetime model when the data are time censored. We use a conditional approach to eliminate the nuisance parameter and develop several procedures based on the conditional likelihood. The methods are (a) a method based on the likelihood ratio, (b) a method based on the skewness corrected score (Bartlett, Biometrika 40 (1953), 12–19), (c) a method based on an adjustment to the signed root likelihood ratio (Diciccio, Field et al., Biometrika 77 (1990), 77–95), and (d) a method based on parameter transformation to the normal approximation. The performances of these procedures are then compared, through simulations, with the usual likelihood based procedure. The skewness corrected score procedure performs best in terms of holding both equal tail probabilities and nominal coverage probabilities even for small samples.     

Confidence Intervals for the Weighted Sum of Two Independent Binomial Proportions

Confidence intervals for the difference of two binomial proportions are well known, however, confidence intervals for the weighted sum of two binomial proportions are less studied. We develop and compare seven methods for constructing confidence intervals for the weighted sum of two independent binomial proportions. The interval estimates are constructed by inverting the Wald test, the score test and the Likelihood ratio test. The weights can be negative, so our results generalize those for the difference between two independent proportions. We provide a numerical study that shows that these confidence intervals based on large‐sample approximations perform very well, even when a relatively small amount of data is available. The intervals based on the inversion of the score test showed the best performance. Finally, we show that as for the difference of two binomial proportions, adding four pseudo‐outcomes to the Wald interval for the weighted sum of two binomial proportions improves its coverage significantly, and we provide a justification for this correction.     

One-sided confidence intervals in discrete distributions

One-sided confidence intervals in the binomial, negative binomial, and Poisson distributions are considered. It is shown that the standard Wald interval suffers from a serious systematic bias in the coverage and so does the one-sided score interval. Alternative confidence intervals with better performance are considered. The coverage and length properties of the confidence intervals are compared through numerical and analytical calculations. Implications to hypothesis testing are also discussed.     

Interval Estimations for the Mean of the Generalized Poisson and Generalized Negative Binomial Distributions

In this article, we extend the Wald, score, skewness-corrected score, likelihood ratio, and mid-P intervals for the means of the generalized Poisson and generalized negative binomial distributions. These distributions are the members of the discrete version of the natural exponential family (NEF) with cubic variance function (CVF). Also, the coverage probabilities, the distal and mesial noncoverage probabilities, and the lengths of the proposed confidence intervals are estimated by means of a Monte Carlo simulation study. Finally, some practical examples are provided to show the applicability of the proposed intervals in applied studies.     

Improved closed-form prediction intervals for binomial and Poisson distributions

The problems of constructing prediction intervals for the binomial and Poisson distributions are considered. Available approximate, exact and conditional methods for both distributions are reviewed and compared. Simple approximate prediction intervals based on the joint distribution of the past samples and the future sample are proposed. Exact coverage studies and expected widths of prediction intervals show that the new prediction intervals are comparable to or better than the available ones in most cases. The methods are illustrated using two practical examples.     

Confidence Intervals for Difference Between Two Poisson Rates

In this article, we develop four explicit asymptotic two-sided confidence intervals for the difference between two Poisson rates via a hybrid method. The basic idea of the proposed method is to estimate or recover the variances of the two Poisson rate estimates, which are required for constructing the confidence interval for the rate difference, from the confidence limits for the two individual Poisson rates. The basic building blocks of the approach are reliable confidence limits for the two individual Poisson rates. Four confidence interval estimators that have explicit solutions and good coverage levels are employed: the first normal with continuity correction, Rao score, Freeman and Tukey, and Jeffreys confidence intervals. Using simulation studies, we examine the performance of the four hybrid confidence intervals and compare them with three existing confidence intervals: the non-informative prior Bayes confidence interval, the t confidence interval based on Satterthwait's degrees of freedom, and the Bayes confidence interval based on Student's t confidence coefficient. Simulation results show that the proposed hybrid Freeman and Tukey, and the hybrid Jeffreys confidence intervals can be highly recommended because they outperform the others in terms of coverage probabilities and widths. The other methods tend to be too conservative and produce wider confidence intervals. The application of these confidence intervals are illustrated with three real data sets.     

Applications of asymptotic confidence intervals with continuity corrections for asymmetric comparisons in noninferiority trials

A large‐sample problem of illustrating noninferiority of an experimental treatment over a referent treatment for binary outcomes is considered. The methods of illustrating noninferiority involve constructing the lower two‐sided confidence bound for the difference between binomial proportions corresponding to the experimental and referent treatments and comparing it with the negative value of the noninferiority margin. The three considered methods, Anbar, Falk–Koch, and Reduced Falk–Koch, handle the comparison in an asymmetric way, that is, only the referent proportion out of the two, experimental and referent, is directly involved in the expression for the variance of the difference between two sample proportions. Five continuity corrections (including zero) are considered with respect to each approach. The key properties of the corresponding methods are evaluated via simulations. First, the uncorrected two‐sided confidence intervals can, potentially, have smaller coverage probability than the nominal level even for moderately large sample sizes, for example, 150 per group. Next, the 15 testing methods are discussed in terms of their Type I error rate and power. In the settings with a relatively small referent proportion (about 0.4 or smaller), the Anbar approach with Yates’ continuity correction is recommended for balanced designs and the Falk–Koch method with Yates’ correction is recommended for unbalanced designs. For relatively moderate (about 0.6) and large (about 0.8 or greater) referent proportion, the uncorrected Reduced Falk–Koch method is recommended, although in this case, all methods tend to be over‐conservative. These results are expected to be used in the design stage of a noninferiority study when asymmetric comparisons are envisioned. Copyright © 2013 John Wiley & Sons, Ltd.     

Score tests for heterogeneity and overdispersion in zero‐inflated Poisson and binomial regression models

Hall (2000) has described zero‐inflated Poisson and binomial regression models that include random effects to account for excess zeros and additional sources of heterogeneity in the data. The authors of the present paper propose a general score test for the null hypothesis that variance components associated with these random effects are zero. For a zero‐inflated Poisson model with random intercept, the new test reduces to an alternative to the overdispersion test of Ridout, Demério & Hinde (2001). The authors also examine their general test in the special case of the zero‐inflated binomial model with random intercept and propose an overdispersion test in that context which is based on a beta‐binomial alternative.     

Coverage‐adjusted Confidence Intervals for a Binomial Proportion

We consider the classic problem of interval estimation of a proportion p based on binomial sampling. The ‘exact’ Clopper–Pearson confidence interval for p is known to be unnecessarily conservative. We propose coverage adjustments of the Clopper–Pearson interval that incorporate prior or posterior beliefs into the interval. Using heatmap‐type plots for comparing confidence intervals, we show that the coverage‐adjusted intervals have satisfying coverage and shorter expected lengths than competing intervals found in the literature.     

Estimating the number of classes in multiple populations: A geometric analysis

The authors study estimation of the total number of classes present in multiple overlapping populations. They show that the number of classes is identifiable in a nonparametric mixture model of multivariate Poisson densities. Unusual phenomena occur in both point estimation and confidence inference for the parameter defined as the odds of a class being unidentified in the data. Consequently only one‐sided confidence intervals are available for the number of classes.     

Better approximate confidence intervals for a binomial parameter

This paper discusses five methods for constructing approximate confidence intervals for the binomial parameter Θ, based on Y successes in n Bernoulli trials. In a recent paper, Chen (1990) discusses various approximate methods and suggests a new method based on a Bayes argument, which we call method I here. Methods II and III are based on the normal approximation without and with continuity correction. Method IV uses the Poisson approximation of the binomial distribution and then exploits the fact that the exact confidence limits for the parameter of the Poisson distribution can be found through the x² distribution. The confidence limits of method IV are then provided by the Wilson-Hilferty approximation of the x². Similarly, the exact confidence limits for the binomial parameter can be expressed through the F distribution. Method V approximates these limits through a suitable version of the Wilson-Hilferty approximation. We undertake a comparison of the five methods in respect to coverage probability and expected length. The results indicate that method V has an advantage over Chen's Bayes method as well as over the other three methods.     

Small Sample Confidence Intervals for the Odds Ratio

Abstract

Numerous methods—based on exact and asymptotic distributions—can be used to obtain confidence intervals for the odds ratio in 2 × 2 tables. We examine ten methods for generating these intervals based on coverage probability, closeness of coverage probability to target, and length of confidence intervals. Based on these criteria, Cornfield’s method, without the continuity correction, performed the best of the methods examined here. A drawback to use of this method is the significant possibility that the attained coverage probability will not meet the nominal confidence level. Use of a mid-P value greatly improves methods based on the “exact” distribution. When combined with the Wilson rule for selection of a rejection set, the resulting method is a procedure that performed very well. Crow’s method, with use of a mid-P, performed well, although it was only a slight improvement over the Wilson mid-P method. Its cumbersome calculations preclude its general acceptance. Woolf's (logit) method—with the Haldane–Anscombe correction— performed well, especially with regard to length of confidence intervals, and is recommended based on ease of computation.     

Interval estimation for misclassification rate parameters in a complementary Poisson model

We investigate three interval estimators for binomial misclassification rates in a complementary Poisson model where the data are possibly misclassified: a Wald-based interval, a score-based interval, and an interval based on the profile log-likelihood statistic. We investigate the coverage and average width properties of these intervals via a simulation study. For small Poisson counts and small misclassification rates, the intervals can perform poorly in terms of coverage. The profile log-likelihood confidence interval (CI) is often proved to outperform the other intervals with good coverage and width properties. Lastly, we apply the CIs to a real data set involving traffic accident data that contain misclassified counts.     

Comparison of Bayesian sample size criteria: ACC, ALC, and WOC

A challenge for implementing performance-based Bayesian sample size determination is selecting which of several methods to use. We compare three Bayesian sample size criteria: the average coverage criterion (ACC) which controls the coverage rate of fixed length credible intervals over the predictive distribution of the data, the average length criterion (ALC) which controls the length of credible intervals with a fixed coverage rate, and the worst outcome criterion (WOC) which ensures the desired coverage rate and interval length over all (or a subset of) possible datasets. For most models, the WOC produces the largest sample size among the three criteria, and sample sizes obtained by the ACC and the ALC are not the same. For Bayesian sample size determination for normal means and differences between normal means, we investigate, for the first time, the direction and magnitude of differences between the ACC and ALC sample sizes. For fixed hyperparameter values, we show that the difference of the ACC and ALC sample size depends on the nominal coverage, and not on the nominal interval length. There exists a threshold value of the nominal coverage level such that below the threshold the ALC sample size is larger than the ACC sample size, and above the threshold the ACC sample size is larger. Furthermore, the ACC sample size is more sensitive to changes in the nominal coverage. We also show that for fixed hyperparameter values, there exists an asymptotic constant ratio between the WOC sample size and the ALC (ACC) sample size. Simulation studies are conducted to show that similar relationships among the ACC, ALC, and WOC may hold for estimating binomial proportions. We provide a heuristic argument that the results can be generalized to a larger class of models.     

Attribute Charts for Zero-Inflated Processes

The classical Shewhart c-chart and p-chart which are constructed based on the Poisson and binomial distributions are inappropriate in monitoring zero-inflated counts. They tend to underestimate the dispersion of zero-inflated counts and subsequently lead to higher false alarm rate in detecting out-of-control signals. Another drawback of these charts is that their 3-sigma control limits, evaluated based on the asymptotic normality assumption of the attribute counts, have a systematic negative bias in their coverage probability. We recommend that the zero-inflated models which account for the excess number of zeros should first be fitted to the zero-inflated Poisson and binomial counts. The Poisson parameter λ estimated from a zero-inflated Poisson model is then used to construct a one-sided c-chart with its upper control limit constructed based on the Jeffreys prior interval that provides good coverage probability for λ. Similarly, the binomial parameter p estimated from a zero-inflated binomial model is used to construct a one-sided np-chart with its upper control limit constructed based on the Jeffreys prior interval or Blyth–Still interval of the binomial proportion p. A simple two-of-two control rule is also recommended to improve further on the performance of these two proposed charts.