Adjusted Confidence Bands in Nonparametric Regression

Suppose we have {(x _i , y _i )} i = 1, 2,…, n, a sequence of independent observations. We wish to find approximate 1 ? α simultaneous confidence bands for the regression curve. Many previous confidence bands in the literature have practical difficulties. In this article, the local linear smoother is used to estimate the regression curve. The bias of the estimator is considered. Different methods of constructing confidence bands are discussed. Finally, a possible method incorporating logistic regression in an innovative way is proposed to construct the bands for random designs. Simulations are used to study the performance or properties of the methods. The procedure for constructing confidence bands is entirely data-driven. The advantage of the proposed method is that it is simple to use and can be applied to random designs. It can be considered as a practically useful and efficient method.     

A Note on Confidence Bands for the Logistic Response Curve

A problem in logit analysis is the interval estimation of the logistic response curve. Scheffé's method is used to obtain confidence bands for the logistic response function for any number of explanatory variables. This method is computationally easier and more general than a previously reported method.     

Designs for approximately linear regression: Maximizing the minimum coverage probability of confidence ellipsoids

The classical D-optimality principle in regression design may be motivated by a desire to maximize the coverage probability of a fixed-volume confidence ellipsoid on the regression parameters. When the fitted model is exactly correct, this amounts to minimizing the determinant of the covariance matrix of the estimators. We consider an analogue of this problem, under the approximately linear model E[y|x] = θ^Tz(x) + f(x). The nonlinear disturbance f(x) is essentially unknown, and the experimenter fits only to the linear part of the response. The resulting bias affects the coverage probability of the confidence ellipsoid on θ. We study the construction of designs which maximize the minimum coverage probability as f varies over a certain class. Explicit designs are given in the case that the fitted response surface is a plane.     

Confidence bands for ROC curves

We develop two methods to construct confidence bands for the receiver operating characteristic (ROC) curve without estimating the densities of the underlying distributions. The first method is based on the smoothed bootstrap while the second method uses the Bonferroni inequality. As an illustration, we provide confidence bands for the ROC curve using data on Duchanne Muscular Dystrophy.     

Bias Reduction through First-order Mean Correction, Bootstrapping and Recursive Mean Adjustment

Standard methods of estimation for autoregressive models are known to be biased in finite samples, which has implications for estimation, hypothesis testing, confidence interval construction and forecasting. Three methods of bias reduction are considered here: first-order bias correction, FOBC, where the total bias is approximated by the O(T^-1) bias; bootstrapping; and recursive mean adjustment, RMA. In addition, we show how first-order bias correction is related to linear bias correction. The practically important case where the AR model includes an unknown linear trend is considered in detail. The fidelity of nominal to actual coverage of confidence intervals is also assessed. A simulation study covers the AR(1) model and a number of extensions based on the empirical AR(p) models fitted by Nelson & Plosser (1982). Overall, which method dominates depends on the criterion adopted: bootstrapping tends to be the best at reducing bias, recursive mean adjustment is best at reducing mean squared error, whilst FOBC does particularly well in maintaining the fidelity of confidence intervals.     

The graphical advantages of finite interval confidence band procedures

When presented as graphical illustrations, regression surface confidence bands for linear statistical models quickly convey detailed information about analysis results. A taut confidence band is a compact set of curves which are estimation candidates for the unobservable, fixed regression curve. The bounds of the band are usually plotted with the estimated regression curve and may be overlaid by a scatter-plot of the data to provide an integrated visual impression. Finite-interval confidence bands offer the advantages of clearer interpretation and improved efficiency and avoid visual ambiguities inherent to infinite-interval bands. The definitive characteristic of a finite-interval confidence band is that it is only necessary to plot it over a finite interval in order to visually communicate all its information. In contrast, visual representations of infinite-interval bands are not fully informative and can be misleading. When an infinite-interval band is plotted, and therefore truncated, substantial information given by its asymptotic behavior is lost. Many curves that are clearly within the plotted portion of the infinite interval confidence band eventually cross a boundary. In practice, a finite-interval band can always be easily obtained from any infinite-interval band. This article focuses on interpretational considerations of symmetric confidence bands as graphical devices.     

On the estimation of the influence curve

We prove the asymptotic validity of bootstrap confidence bands for the influence curve from its usual estimator (the sensitivity curve). The proof is based on the use of Gill's generalized delta method for Hadamard differentiable operators. Some statistical applications, in particular to the estimation of asymptotic variance, are given.     

Empirical Likelihood Local Polynomial Regression Analysis of Clustered Data

Abstract. In this article, a naive empirical likelihood ratio is constructed for a non‐parametric regression model with clustered data, by combining the empirical likelihood method and local polynomial fitting. The maximum empirical likelihood estimates for the regression functions and their derivatives are obtained. The asymptotic distributions for the proposed ratio and estimators are established. A bias‐corrected empirical likelihood approach to inference for the parameters of interest is developed, and the residual‐adjusted empirical log‐likelihood ratio is shown to be asymptotically chi‐squared. These results can be used to construct a class of approximate pointwise confidence intervals and simultaneous bands for the regression functions and their derivatives. Owing to our bias correction for the empirical likelihood ratio, the accuracy of the obtained confidence region is not only improved, but also a data‐driven algorithm can be used for selecting an optimal bandwidth to estimate the regression functions and their derivatives. A simulation study is conducted to compare the empirical likelihood method with the normal approximation‐based method in terms of coverage accuracies and average widths of the confidence intervals/bands. An application of this method is illustrated using a real data set.     

Secondary design considerations for minimum bias estimation

Several authors have suggested the method of minimum bias estimation for estimating response surfaces. The minimum bias estimation procedure achieves minimum average squared bias of the fitted model without depending on the values of the unknown parameters of the true surface. The only requirement is that the design satisfies a simple estimability condition. Subject to providing minimum average squared bias, the minimum bias estimator also provides minimum average variance of ?(x) where ?(x) is the estimate of the response at the point x.

To support the estimation of the parameters in the fitted model, very little has been suggested in the way of experimental designs except to say that a full rank matrix X of independent variables should be used. This paper presents a closer look at the estimability conditions that are required for minimum bias estimation, and from the form of the matrix X, a formula is derived which measures the amount of design flexibility available. The design flexibility is termed “the degrees of freedom” of the X matrix and it is shown how the degrees of freedom can be used to decide if other design optimality criteria might be considered along with minimum bias estimation. Several examples are provided.     

Confidence intervals and bands for the binormal ROC curve revisited

Two types of confidence intervals (CIs) and confidence bands (CBs) for the receiver operating characteristic (ROC) curve are studied: pointwise CIs and simultaneous CBs. An optimized version of the pointwise CI with the shortest width is developed. A new ellipse-envelope simultaneous CB for the ROC curve is suggested as an adaptation of the Working-Hotelling-type CB implemented in a paper by Ma and Hall (1993). Statistical simulations show that our ellipse-envelope CB covers the true ROC curve with a probability close to nominal while the coverage probability of the Ma and Hall CB is significantly smaller. Simulations also show that our CI for the area under the ROC curve is close to nominal while the coverage probability of the CI suggested by Hanley and McNail (1982) uniformly overestimates the nominal value. Two examples illustrate our simultaneous ROC bands: radiation dose estimation from time to vomiting and discrimination of breast cancer from benign abnormalities using electrical impedance measurements.     

Coverage probabilities of nonparametric simultaneous confidence bands for a survival function

Simultaneous confidence bands provide a useful adjunct to the popular Kaplan–Meier product limit estimator for a survival function, particularly when results are displayed graphically. They allow an assessment of the magnitude of sampling errors and provide a graphical view of a formal goodness-of-fit test. In this paper we evaluate a modified version of Nair's (1981) simultaneous confidence bands. The modification is based on a logistic transformation of the Kaplan–Meier estimator. We show that the modified bands have some important practical advantages.     

Exact Simultaneous Confidence Bands for Quadratic and Cubic Polynomial Regression with Applications in Dose Response Study

Exact simultaneous confidence bands (SCBs) for a polynomial regression model are available only in some special situations. In this paper, simultaneous confidence levels for both hyperbolic and constant width bands for a polynomial function over a given interval are expressed as multidimensional integrals. The dimension of these integrals is equal to the degree of the polynomial. Hence the values can be calculated quickly and accurately via numerical quadrature provided that the degree of the polynomial is small (e.g. 2 or 3). This allows the construction of exact SCBs for quadratic and cubic regression functions over any given interval and for any given design matrix. Quadratic and cubic regressions are frequently used to characterise dose response relationships in addition to many other applications. Comparison between the hyperbolic and constant width bands under both the average width and minimum volume confidence set criteria shows that the constant width band can be much less efficient than the hyperbolic band. For hyperbolic bands, comparison between the exact critical constant and conservative or approximate critical constants indicates that the exact critical constant can be substantially smaller than the conservative or approximate critical constants. Numerical examples from a dose response study are used to illustrate the methods.     

Simultaneous confidence bands for growth incidence curves in weighted sup-norm metrics

ABSTRACT

The so-called growth incidence curve (GIC) is a popular way to evaluate the distributional pattern of economic growth and pro-poorness of growth in development economics. The log-transformation of the the GIC is related to the sum of empirical quantile processes which allows for constructions of simultaneous confidence bands for the GIC. However, standard constructions of these bands tend to be too wide at the extreme points 0 and 1 because the estimator of the quantile function can be very volatile at the extreme points. In order to construct simultaneous confidence bands which are narrower at the ends, we consider the convergence of quantile processes with weight functions. In particular, we investigate the asymptotic convergence under specific weighted sup-norm metrics and compare different kinds of qualified weight functions. This implies simultaneous confidence bands that are narrower at the boundaries 0 and 1. We show in simulations that these bands have a more regular shape. Finally, we evaluate real data from Uganda with the improved confidence bands.     

Optimal simultaneous confidence bands in simple linear regression

A simultaneous confidence band provides useful information on the plausible range of an unknown regression model. For simple linear regression models, the most frequently quoted bands in the statistical literature include the hyperbolic band and the three-segment bands. One interesting question is whether one can construct confidence bands better than the hyperbolic and three-segment bands. The optimality criteria for confidence bands include the average width criterion considered by Gafarian (1964) and Naiman (1984) among others, and the minimum area confidence set (MACS) criterion of Liu and Hayter (2007). In this paper, two families of exact 1−α$1-\alpha$ confidence bands, the inner-hyperbolic bands and the outer-hyperbolic bands, which include the hyperbolic and three-segment bands as special cases, are introduced in simple linear regression. Under the MACS criterion, the best confidence band within each family is found by numerical search and compared with the hyperbolic band, the best three-segment band and with each other. The methodologies are illustrated with a numerical example and the Matlab programs used are available upon request.     

Bias‐reduced marginal Akaike information criteria based on a Monte Carlo method for linear mixed‐effects models

In linear mixed‐effects (LME) models, if a fitted model has more random‐effect terms than the true model, a regularity condition required in the asymptotic theory may not hold. In such cases, the marginal Akaike information criterion (AIC) is positively biased for (?2) times the expected log‐likelihood. The asymptotic bias of the maximum log‐likelihood as an estimator of the expected log‐likelihood is evaluated for LME models with balanced design in the context of parameter‐constrained models. Moreover, bias‐reduced marginal AICs for LME models based on a Monte Carlo method are proposed. The performance of the proposed criteria is compared with existing criteria by using example data and by a simulation study. It was found that the bias of the proposed criteria was smaller than that of the existing marginal AIC when a larger model was fitted and that the probability of choosing a smaller model incorrectly was decreased.     

Model-based confidence bands for survival functions

This paper focuses on a novel method of developing one-sample confidence bands for survival functions from right censored data. The approach is model-based, relying on a parametric model for the conditional expectation of the censoring indicator given the observed minimum, and derives its strength from easy access to a good-fitting model among a plethora of choices available for binary response data. The substantive methodological contribution is in exploiting a semiparametric estimator of the survival function to produce improved simultaneous confidence bands. To obtain critical values for computing the confidence bands, a two-stage bootstrap approach that combines the classical bootstrap with the more recent model-based regeneration of censoring indicators is proposed and a justification of its asymptotic validity is also provided. Several different confidence bands are studied using the proposed approach. Numerical studies, including robustness of the proposed bands to misspecification, are carried out to check efficacy. The method is illustrated using two lung cancer data sets.     

Easy-to-construct confidence bands for comparing two simple linear regression lines

This paper shows how to construct confidence bands for the difference between two simple linear regression lines. These confidence bands provide directly the information on the magnitude of the difference between the regression lines over an interval of interest and, as a by-product, can be used as a formal test of the difference between the two regression lines. Various different shapes of confidence bands are illustrated, and particular attention is paid towards confidence bands whose construction only involves critical points from standard distributions so that they are consequently easy to construct.     

Unbiased goodness-of-fit tests

Familiar distribution-free goodness-of-fit tests like the Kolmogorov–Smirnov test are all biased tests. In this paper, we show how to compute the bias of any distribution-free goodness-of-fit test that corresponds to a distribution-free confidence band for the cumulative distribution function (CDF). The bias of the Kolmogorov–Smirnov test turns out to be smaller than the biases of other distribution-free goodness-of-fit tests. We also develop a method for obtaining unbiased goodness-of-fit tests, which can then be inverted to obtain unbiased confidence bands for the CDF. Interestingly, only a discrete set of levels are available for the unbiased tests. Our power comparisons show that while removing bias improves the power of a test at some alternatives, it does not improve the overall power properties of the test.     

Comparison of bootstrap estimation intervals to forecast arithmetic mean and median air passenger demand

The aim of this paper is to compare passenger (pax) demand between airports based on the arithmetic mean (MPD) and the median pax demand (MePD). A three phases approach is applied. First phase, we use bootstrap procedures to estimate the distribution of the arithmetic MPD and the MePD for each block of routes distance; second phase, we use percentile, standard, bias corrected, and bias corrected accelerated methods to calculate bootstrap confidence bands for the MPD and the MePD; and third phase, we implement Monte Carlo (MC) experiments to analyse the finite sample performance of the applied bootstrap. Our results conclude that it is more meaningful to use the estimation of MePD rather than the estimation of MPD in the air transport industry. By carrying out MC experiments, we demonstrate that the bootstrap methods produce coverages close to the nominal for the MPD and the MePD.     

Inverse Probability Weighting with Missing Predictors of Treatment Assignment or Missingness

Inverse probability weighting (IPW) can deal with confounding in non randomized studies. The inverse weights are probabilities of treatment assignment (propensity scores), estimated by regressing assignment on predictors. Problems arise if predictors can be missing. Solutions previously proposed include assuming assignment depends only on observed predictors and multiple imputation (MI) of missing predictors. For the MI approach, it was recommended that missingness indicators be used with the other predictors. We determine when the two MI approaches, (with/without missingness indicators) yield consistent estimators and compare their efficiencies.We find that, although including indicators can reduce bias when predictors are missing not at random, it can induce bias when they are missing at random. We propose a consistent variance estimator and investigate performance of the simpler Rubin’s Rules variance estimator. In simulations we find both estimators perform well. IPW is also used to correct bias when an analysis model is fitted to incomplete data by restricting to complete cases. Here, weights are inverse probabilities of being a complete case. We explain how the same MI methods can be used in this situation to deal with missing predictors in the weight model, and illustrate this approach using data from the National Child Development Survey.