Model‐Averaged Confidence Intervals

We develop an approach to evaluating frequentist model averaging procedures by considering them in a simple situation in which there are two‐nested linear regression models over which we average. We introduce a general class of model averaged confidence intervals, obtain exact expressions for the coverage and the scaled expected length of the intervals, and use these to compute these quantities for the model averaged profile likelihood (MPI) and model‐averaged tail area confidence intervals proposed by D. Fletcher and D. Turek. We show that the MPI confidence intervals can perform more poorly than the standard confidence interval used after model selection but ignoring the model selection process. The model‐averaged tail area confidence intervals perform better than the MPI and postmodel‐selection confidence intervals but, for the examples that we consider, offer little over simply using the standard confidence interval for θ under the full model, with the same nominal coverage.     

Confidence intervals centred on bootstrap smoothed estimators

Bootstrap smoothed (bagged) parameter estimators have been proposed as an improvement on estimators found after preliminary data‐based model selection. A result of Efron in 2014 is a very convenient and widely applicable formula for a delta method approximation to the standard deviation of the bootstrap smoothed estimator. This approximation provides an easily computed guide to the accuracy of this estimator. In addition, Efron considered a confidence interval centred on the bootstrap smoothed estimator, with width proportional to the estimate of this approximation to the standard deviation. We evaluate this confidence interval in the scenario of two nested linear regression models, the full model and a simpler model, and a preliminary test of the null hypothesis that the simpler model is correct. We derive computationally convenient expressions for the ideal bootstrap smoothed estimator and the coverage probability and expected length of this confidence interval. In terms of coverage probability, this confidence interval outperforms the post‐model‐selection confidence interval with the same nominal coverage and based on the same preliminary test. We also compare the performance of the confidence interval centred on the bootstrap smoothed estimator, in terms of expected length, to the usual confidence interval, with the same minimum coverage probability, based on the full model.     

ON THE COVERAGE PROBABILITY OF CONFIDENCE INTERVALS IN REGRESSION AFTER VARIABLE SELECTION

This paper considers a linear regression model with regression parameter vector β. The parameter of interest is θ= a^Tβ where a is specified. When, as a first step, a data‐based variable selection (e.g. minimum Akaike information criterion) is used to select a model, it is common statistical practice to then carry out inference about θ, using the same data, based on the (false) assumption that the selected model had been provided a priori. The paper considers a confidence interval for θ with nominal coverage 1 ‐ α constructed on this (false) assumption, and calls this the naive 1 ‐ α confidence interval. The minimum coverage probability of this confidence interval can be calculated for simple variable selection procedures involving only a single variable. However, the kinds of variable selection procedures used in practice are typically much more complicated. For the real‐life data presented in this paper, there are 20 variables each of which is to be either included or not, leading to 2²⁰ different models. The coverage probability at any given value of the parameters provides an upper bound on the minimum coverage probability of the naive confidence interval. This paper derives a new Monte Carlo simulation estimator of the coverage probability, which uses conditioning for variance reduction. For these real‐life data, the gain in efficiency of this Monte Carlo simulation due to conditioning ranged from 2 to 6. The paper also presents a simple one‐dimensional search strategy for parameter values at which the coverage probability is relatively small. For these real‐life data, this search leads to parameter values for which the coverage probability of the naive 0.95 confidence interval is 0.79 for variable selection using the Akaike information criterion and 0.70 for variable selection using Bayes information criterion, showing that these confidence intervals are completely inadequate.     

The large sample coverage probability of confidence intervals in general regression models after a preliminary hypothesis test

We derive a computationally convenient formula for the large sample coverage probability of a confidence interval for a scalar parameter of interest following a preliminary hypothesis test that a specified vector parameter takes a given value in a general regression model. Previously, this large sample coverage probability could only be estimated by simulation. Our formula only requires the evaluation, by numerical integration, of either a double or a triple integral, irrespective of the dimension of this specified vector parameter. We illustrate the application of this formula to a confidence interval for the odds ratio of myocardial infarction when the exposure is recent oral contraceptive use, following a preliminary test where two specified interactions in a logistic regression model are zero. For this real‐life data, we compare this large sample coverage probability with the actual coverage probability of this confidence interval, obtained by simulation.     

Confidence intervals following Box-Cox transformation

What is the interpretation of a confidence interval following estimation of a Box-Cox transformation parameter λ? Several authors have argued that confidence intervals for linear model parameters ψ can be constructed as if λ. were known in advance, rather than estimated, provided the estimand is interpreted conditionally given $\hat \lambda$. If the estimand is defined as $\psi \left( {\hat \lambda } \right)$, a function of the estimated transformation, can the nominal confidence level be regarded as a conditional coverage probability given $\hat \lambda$, where the interval is random and the estimand is fixed? Or should it be regarded as an unconditional probability, where both the interval and the estimand are random? This article investigates these questions via large-n approximations, small- σ approximations, and simulations. It is shown that, when model assumptions are satisfied and n is large, the nominal confidence level closely approximates the conditional coverage probability. When n is small, this conditional approximation is still good for regression models with small error variance. The conditional approximation can be poor for regression models with moderate error variance and single-factor ANOVA models with small to moderate error variance. In these situations the nominal confidence level still provides a good approximation for the unconditional coverage probability. This suggests that, while the estimand may be interpreted conditionally, the confidence level should sometimes be interpreted unconditionally.     

Generalized confidence interval for the slope in linear measurement error model

A generalized confidence interval for the slope parameter in linear measurement error model is proposed in this article, which is based on the relation between the slope of classical regression model and the measurement error model. The performance of the confidence interval estimation procedure is studied numerically through Monte Carlo simulation in terms of coverage probability and expected length.     

UPPER BOUNDS ON THE MINIMUM COVERAGE PROBABILITY OF CONFIDENCE INTERVALS IN REGRESSION AFTER MODEL SELECTION

We consider a linear regression model, with the parameter of interest a specified linear combination of the components of the regression parameter vector. We suppose that, as a first step, a data-based model selection (e.g. by preliminary hypothesis tests or minimizing the Akaike information criterion – AIC) is used to select a model. It is common statistical practice to then construct a confidence interval for the parameter of interest, based on the assumption that the selected model had been given to us  a priori . This assumption is false, and it can lead to a confidence interval with poor coverage properties. We provide an easily computed finite-sample upper bound (calculated by repeated numerical evaluation of a double integral) to the minimum coverage probability of this confidence interval. This bound applies for model selection by any of the following methods: minimum AIC, minimum Bayesian information criterion (BIC), maximum adjusted  R ², minimum Mallows'   C _P   and  t -tests. The importance of this upper bound is that it delineates general categories of design matrices and model selection procedures for which this confidence interval has poor coverage properties. This upper bound is shown to be a finite-sample analogue of an earlier large-sample upper bound due to Kabaila and Leeb.     

THE COVERAGE PROBABILITY OF CONFIDENCE INTERVALS IN 2r FACTORIAL EXPERIMENTS AFTER PRELIMINARY HYPOTHESIS TESTING

We consider a 2^r factorial experiment with at least two replicates. Our aim is to find a confidence interval for θ, a specified linear combination of the regression parameters (for the model written as a regression, with factor levels coded as ?1 and 1). We suppose that preliminary hypothesis tests are carried out sequentially, beginning with the rth‐order interaction. After these preliminary hypothesis tests, a confidence interval for θ with nominal coverage 1 ?α is constructed under the assumption that the selected model had been given to us a priori. We describe a new efficient Monte Carlo method, which employs conditioning for variance reduction, for estimating the minimum coverage probability of the resulting confidence interval. The application of this method is demonstrated in the context of a 2³ factorial experiment with two replicates and a particular contrast θ of interest. The preliminary hypothesis tests consist of the following two‐step procedure. We first test the null hypothesis that the third‐order interaction is zero against the alternative hypothesis that it is non‐zero. If this null hypothesis is accepted, we assume that this interaction is zero and proceed to the second step; otherwise, we stop. In the second step, for each of the second‐order interactions we test the null hypothesis that the interaction is zero against the alternative hypothesis that it is non‐zero. If this null hypothesis is accepted, we assume that this interaction is zero. The resulting confidence interval, with nominal coverage probability 0.95, has a minimum coverage probability that is, to a good approximation, 0.464. This shows that this confidence interval is completely inadequate.     

Confidence Intervals in Regression That Utilize Uncertain Prior Information About a Vector Parameter

Consider a linear regression model with independent normally distributed errors. Suppose that the scalar parameter of interest is a specified linear combination of the components of the regression parameter vector. Also suppose that we have uncertain prior information that a parameter vector, consisting of specified distinct linear combinations of these components, takes a given value. Part of our evaluation of a frequentist confidence interval for the parameter of interest is the scaled expected length, defined to be the expected length of this confidence interval divided by the expected length of the standard confidence interval for this parameter, with the same confidence coefficient. We say that a confidence interval for the parameter of interest utilizes this uncertain prior information if (a) the scaled expected length of this interval is substantially less than one when the prior information is correct, (b) the maximum value of the scaled expected length is not too large and (c) this confidence interval reverts to the standard confidence interval, with the same confidence coefficient, when the data happen to strongly contradict the prior information. We present a new confidence interval for a scalar parameter of interest, with specified confidence coefficient, that utilizes this uncertain prior information. A factorial experiment with one replicate is used to illustrate the application of this new confidence interval.     

An alternative to confidence intervals constructed after a Hausman pretest in panel data

Consider panel data modelled by a linear random intercept model that includes a time‐varying covariate. Suppose that our aim is to construct a confidence interval for the slope parameter. Commonly, a Hausman pretest is used to decide whether this confidence interval is constructed using the random effects model or the fixed effects model. This post‐model‐selection confidence interval has the attractive features that it (a) is relatively short when the random effects model is correct and (b) reduces to the confidence interval based on the fixed effects model when the data and the random effects model are highly discordant. However, this confidence interval has the drawbacks that (i) its endpoints are discontinuous functions of the data and (ii) its minimum coverage can be far below its nominal coverage probability. We construct a new confidence interval that possesses these attractive features, but does not suffer from these drawbacks. This new confidence interval provides an intermediate between the post‐model‐selection confidence interval and the confidence interval obtained by always using the fixed effects model. The endpoints of the new confidence interval are smooth functions of the Hausman test statistic, whereas the endpoints of the post‐model‐selection confidence interval are discontinuous functions of this statistic.     

A new confidence interval in measurement error model with the reliability ratio known

For the slope parameter of the measurement error model with the reliability ratio known, this article constructs a fiducial generalized confidence interval (FGCI) which is proved to have correct asymptotic coverage. Simulation results demonstrate that the FGCI often outperforms the existing intervals in terms of empirical coverage probability, average interval length, and false parameter coverage rate. Two examples are also provided to illustrate our approach.     

Asymptotic expansions for sequential confidence levels in inverse linear regression

Non-linear renewal theory is used to derive second order asymptotic expansions for the coverage probability of a fixed-width sequential confidence interval for an unknown parameter xin the inverse linear regression model. These expansions are obtained for a two-stage sequential procedure, proposed by Perng and Tong (1974) for the construction of a confidence interval for x.     

Length minimization for Poisson confidence procedures

We study Poisson confidence procedures that potentially lead to short confidence intervals, investigating the class of all minimal cardinality procedures. We consider how length minimization should be properly defined, and show that Casella and Robert's (1989) criterion for comparing Poisson confidence procedures leads to a contradiction. We provide an alternative criterion for comparing length performance, identify the unique length optimal minimal cardinality procedure by this criterion, and propose a modification that eliminates an important drawback it possesses. We focus on procedures whose coverage never falls below the nominal level and discuss the case in which the nominal level represents mean coverage.     

Performance of Interval Estimators for the Inverse Hypergeometric Distribution

The inverse hypergeometric distribution is of interest in applications of inverse sampling without replacement from a finite population where a binary observation is made on each sampling unit. Thus, sampling is performed by randomly choosing units sequentially one at a time until a specified number of one of the two types is selected for the sample. Assuming the total number of units in the population is known but the number of each type is not, we consider the problem of estimating this parameter. We use the Delta method to develop approximations for the variance of three parameter estimators. We then propose three large sample confidence intervals for the parameter. Based on these results, we selected a sampling of parameter values for the inverse hypergeometric distribution to empirically investigate performance of these estimators. We evaluate their performance in terms of expected probability of parameter coverage and confidence interval length calculated as means of possible outcomes weighted by the appropriate outcome probabilities for each parameter value considered. The unbiased estimator of the parameter is the preferred estimator relative to the maximum likelihood estimator and an estimator based on a negative binomial approximation, as evidenced by empirical estimates of closeness to the true parameter value. Confidence intervals based on the unbiased estimator tend to be shorter than the two competitors because of its relatively small variance but at a slight cost in terms of coverage probability.     

Inferences on the reliability in balanced and unbalanced one-way random models

In this article, the hypothesis testing and interval estimation for the reliability parameter are considered in balanced and unbalanced one-way random models. The tests and confidence intervals for the reliability parameter are developed using the concepts of generalized p-value and generalized confidence interval. Furthermore, some simulation results are presented to compare the performances between the proposed approach and the existing approach. For balanced models, the simulation results indicate that the proposed approach can provide satisfactory coverage probabilities and performs better than the existing approaches across the wide array of scenarios, especially for small sample sizes. For unbalanced models, the simulation results show that the two proposed approaches perform more satisfactorily than the existing approach in most cases. Finally, the proposed approaches are illustrated using two real examples.     

Confidence intervals for the ratio of two variances

In this paper we consider confidence intervals for the ratio of two population variances. We propose a confidence interval for the ratio of two variances based on the t-statistic by deriving its Edgeworth expansion and considering Hall's and Johnson's transformations. Then, we consider the coverage accuracy of suggested intervals and intervals based on the F-statistic for some distributions.     

Designs for approximately linear regression: Maximizing the minimum coverage probability of confidence ellipsoids

The classical D-optimality principle in regression design may be motivated by a desire to maximize the coverage probability of a fixed-volume confidence ellipsoid on the regression parameters. When the fitted model is exactly correct, this amounts to minimizing the determinant of the covariance matrix of the estimators. We consider an analogue of this problem, under the approximately linear model E[y|x] = θ^Tz(x) + f(x). The nonlinear disturbance f(x) is essentially unknown, and the experimenter fits only to the linear part of the response. The resulting bias affects the coverage probability of the confidence ellipsoid on θ. We study the construction of designs which maximize the minimum coverage probability as f varies over a certain class. Explicit designs are given in the case that the fitted response surface is a plane.     

On a class of improved estimators of variance and estimation under order restriction

We consider the estimation of error variance and construct a class of estimators improving upon the usual estimators uniformly under entropy loss or under squared error loss. Through a Monte Carlo simulation study, the magnitude of the risk reduction of our improved estimator as compared with the usual one is examined in a context of a nested linear hypothesis testing of a linear regression model, where substantial risk reduction can be attained. We also construct a class of confidence intervals having larger coverage probabilities and not larger interval lengths than those of the usual ones. This allows us to construct a class of estimators universally dominating the usual ones. Further, we consider the estimation of order-restricted normal variances. We give a class of isotonic regression estimators improving upon the usual ones under various types of order restrictions. We also give a class of improved confidence intervals over the usual ones, and a class of estimators universally dominating the usual ones.     

A new confidence interval in errors-in-variables model with known error variance

This paper considers constructing a new confidence interval for the slope parameter in the structural errors-in-variables model with known error variance associated with the regressors. Existing confidence intervals are so severely affected by Gleser–Hwang effect that they are subject to have poor empirical coverage probabilities and unsatisfactory lengths. Moreover, these problems get worse with decreasing reliability ratio which also result in more frequent absence of some existing intervals. To ease these issues, this paper presents a fiducial generalized confidence interval which maintains the correct asymptotic coverage. Simulation results show that this fiducial interval is slightly conservative while often having average length comparable or shorter than the other methods. Finally, we illustrate these confidence intervals with two real data examples, and in the second example some existing intervals do not exist.