The minimum coverage probability of confidence intervals in regression after a preliminary F test

Consider a linear regression model with regression parameter β=(β₁,…,β_p) and independent normal errors. Suppose the parameter of interest is θ=a^Tβ, where a is specified. Define the s-dimensional parameter vector τ=C^Tβ−t, where C and t are specified. Suppose that we carry out a preliminary F test of the null hypothesis H₀:τ=0 against the alternative hypothesis H₁:τ≠0. It is common statistical practice to then construct a confidence interval for θ with nominal coverage 1−α, using the same data, based on the assumption that the selected model had been given to us a priori (as the true model). We call this the naive 1−α confidence interval for θ. This assumption is false and it may lead to this confidence interval having minimum coverage probability far below 1−α, making it completely inadequate. We provide a new elegant method for computing the minimum coverage probability of this naive confidence interval, that works well irrespective of how large s is. A very important practical application of this method is to the analysis of covariance. In this context, τ can be defined so that H₀ expresses the hypothesis of “parallelism”. Applied statisticians commonly recommend carrying out a preliminary F test of this hypothesis. We illustrate the application of our method with a real-life analysis of covariance data set and a preliminary F test for “parallelism”. We show that the naive 0.95 confidence interval has minimum coverage probability 0.0846, showing that it is completely inadequate.     

The performance of model averaged tail area confidence intervals

We investigate the exact coverage and expected length properties of the model averaged tail area (MATA) confidence interval proposed by Turek and Fletcher, CSDA, 2012, in the context of two nested, normal linear regression models. The simpler model is obtained by applying a single linear constraint on the regression parameter vector of the full model. For given length of response vector and nominal coverage of the MATA confidence interval, we consider all possible models of this type and all possible true parameter values, together with a wide class of design matrices and parameters of interest. Our results show that, while not ideal, MATA confidence intervals perform surprisingly well in our regression scenario, provided that we use the minimum weight within the class of weights that we consider on the simpler model.     

Construction of fixed width confidence intervals for a Bernoulli success probability using sequential sampling: a simulation study

This article considers the construction of level 1?α fixed width 2d confidence intervals for a Bernoulli success probability p, assuming no prior knowledge about p and so p can be anywhere in the interval [0, 1]. It is shown that some fixed width 2d confidence intervals that combine sequential sampling of Hall [Asymptotic theory of triple sampling for sequential estimation of a mean, Ann. Stat. 9 (1981), pp. 1229–1238] and fixed-sample-size confidence intervals of Agresti and Coull [Approximate is better than ‘exact’ for interval estimation of binomial proportions, Am. Stat. 52 (1998), pp. 119–126], Wilson [Probable inference, the law of succession, and statistical inference, J. Am. Stat. Assoc. 22 (1927), pp. 209–212] and Brown et al. [Interval estimation for binomial proportion (with discussion), Stat. Sci. 16 (2001), pp. 101–133] have close to 1?α confidence level. These sequential confidence intervals require a much smaller sample size than a fixed-sample-size confidence interval. For the coin jamming example considered, a fixed-sample-size confidence interval requires a sample size of 9457, while a sequential confidence interval requires a sample size that rarely exceeds 2042.     

The monotone boundary property and the full coverage property of confidence intervals for a binomial proportion

The methodology for deriving the exact confidence coefficient of some confidence intervals for a binomial proportion is proposed in Wang [2007. Exact confidence coefficients of confidence intervals for a binomial proportion. Statist. Sinica 17, 361–368]. The methodology requires two conditions of confidence intervals: the monotone boundary property and the full coverage property. In this paper, we show that for some confidence intervals of a binomial proportion, the two properties hold for any sample size. Based on results presented in this paper, the procedure in Wang [2007. Exact confidence coefficients of confidence intervals for a binomial proportion. Statist. Sinica 17, 361–368] can be directly used to calculate the exact confidence coefficients of these confidence intervals for any fixed sample size.     

A comparison of alternative methods to construct confidence intervals for the estimate of a break date in linear regression models

This article considers constructing confidence intervals for the date of a structural break in linear regression models. Using extensive simulations, we compare the performance of various procedures in terms of exact coverage rates and lengths of the confidence intervals. These include the procedures of Bai (1997 Bai, J. (1997). Estimation of a change point in multiple regressions. Review of Economics and Statistics 79:551–563.[Crossref], [Web of Science ®] , [Google Scholar]) based on the asymptotic distribution under a shrinking shift framework, Elliott and Müller (2007 Elliott, G., Müller, U. (2007). Confidence sets for the date of a single break in linear time series regressions. Journal of Econometrics 141:1196–1218.[Crossref], [Web of Science ®] , [Google Scholar]) based on inverting a test locally invariant to the magnitude of break, Eo and Morley (2015 Eo, Y., Morley, J. (2015). Likelihood-ratio-based confidence sets for the timing of structural breaks. Quantitative Economics 6:463–497.[Crossref], [Web of Science ®] , [Google Scholar]) based on inverting a likelihood ratio test, and various bootstrap procedures. On the basis of achieving an exact coverage rate that is closest to the nominal level, Elliott and Müller's (2007 Elliott, G., Müller, U. (2007). Confidence sets for the date of a single break in linear time series regressions. Journal of Econometrics 141:1196–1218.[Crossref], [Web of Science ®] , [Google Scholar]) approach is by far the best one. However, this comes with a very high cost in terms of the length of the confidence intervals. When the errors are serially correlated and dealing with a change in intercept or a change in the coefficient of a stationary regressor with a high signal-to-noise ratio, the length of the confidence interval increases and approaches the whole sample as the magnitude of the change increases. The same problem occurs in models with a lagged dependent variable, a common case in practice. This drawback is not present for the other methods, which have similar properties. Theoretical results are provided to explain the drawbacks of Elliott and Müller's (2007 Elliott, G., Müller, U. (2007). Confidence sets for the date of a single break in linear time series regressions. Journal of Econometrics 141:1196–1218.[Crossref], [Web of Science ®] , [Google Scholar]) method.     

The challenges of evaluating dose response in flexible‐dose trials using marginal structural models

Assessing dose response from flexible‐dose clinical trials is problematic. The true dose effect may be obscured and even reversed in observed data because dose is related to both previous and subsequent outcomes. To remove selection bias, we propose marginal structural models, inverse probability of treatment‐weighting (IPTW) methodology. Potential clinical outcomes are compared across dose groups using a marginal structural model (MSM) based on a weighted pooled repeated measures analysis (generalized estimating equations with robust estimates of standard errors), with dose effect represented by current dose and recent dose history, and weights estimated from the data (via logistic regression) and determined as products of (i) inverse probability of receiving dose assignments that were actually received and (ii) inverse probability of remaining on treatment by this time. In simulations, this method led to almost unbiased estimates of true dose effect under various scenarios. Results were compared with those obtained by unweighted analyses and by weighted analyses under various model specifications. The simulation showed that the IPTW MSM methodology is highly sensitive to model misspecification even when weights are known. Practitioners applying MSM should be cautious about the challenges of implementing MSM with real clinical data. Clinical trial data are used to illustrate the methodology. Copyright © 2012 John Wiley & Sons, Ltd.