Optimal simultaneous confidence bands in simple linear regression

A simultaneous confidence band provides useful information on the plausible range of an unknown regression model. For simple linear regression models, the most frequently quoted bands in the statistical literature include the hyperbolic band and the three-segment bands. One interesting question is whether one can construct confidence bands better than the hyperbolic and three-segment bands. The optimality criteria for confidence bands include the average width criterion considered by Gafarian (1964) and Naiman (1984) among others, and the minimum area confidence set (MACS) criterion of Liu and Hayter (2007). In this paper, two families of exact 1−α$1-\alpha$ confidence bands, the inner-hyperbolic bands and the outer-hyperbolic bands, which include the hyperbolic and three-segment bands as special cases, are introduced in simple linear regression. Under the MACS criterion, the best confidence band within each family is found by numerical search and compared with the hyperbolic band, the best three-segment band and with each other. The methodologies are illustrated with a numerical example and the Matlab programs used are available upon request.     

An Independence Point Method of Confidence Band Construction for Multiple Linear Regression Models

This article addresses the problem of confidence band construction for a standard multiple linear regression model. An “independence point” method of construction is developed which generalizes the method of Gafarian (1964) for a simple linear regression model to a multiple linear regression model. Wynn (1984 Wynn , H. P. ( 1984 ). An exact confidence band for one-dimensional polynomial regression . Biometrika 71 : 375 – 9 .[Crossref], [Web of Science ®] , [Google Scholar]) pioneered the approach of basing confidence bands for a polynomial regression on a set of nodes where the function estimates are independent, and this approach is exploited in this article. This method requires only critical points from t-distributions so that the confidence bands are easy to construct. Both one-sided and two-sided confidence bands can be constructed using this method. An illustration of the new method is provided, and comparisons are made with other procedures.     

Slope modified confidence bands for a simple linear regression model

Considerable attention has been directed in the statistical literature towards the construction of confidence bands for a simple linear regression model. These confidence bands allow the experimenter to make inferences about the model over a particular region of interest. However, in practice an experimenter will usually first check the significance of the regression line before proceeding with any further inferences such as those provided by the confidence bands. From a theoretical point of view, this raises the question of what the conditional confidence level of the confidence bands might be, and from a practical point of view it is unsatisfactory if the confidence bands contain lines that are inconsistent with the directional decision on the slope. In this paper it is shown how confidence bands can be modified to alleviate these two problems.     

Simultaneous Confidence Bands for Linear Regression with Covariates Constrained in Intervals

Abstract. The focus of this article is on simultaneous confidence bands over a rectangular covariate region for a linear regression model with k>1 covariates, for which only conservative or approximate confidence bands are available in the statistical literature stretching back to Working & Hotelling (J. Amer. Statist. Assoc. 24 , 1929; 73–85). Formulas of simultaneous confidence levels of the hyperbolic and constant width bands are provided. These involve only a k‐dimensional integral; it is unlikely that the simultaneous confidence levels can be expressed as an integral of less than k‐dimension. These formulas allow the construction for the first time of exact hyperbolic and constant width confidence bands for at least a small k(>1) by using numerical quadrature. Comparison between the hyperbolic and constant width bands is then addressed under both the average width and minimum volume confidence set criteria. It is observed that the constant width band can be drastically less efficient than the hyperbolic band when k>1. Finally it is pointed out how the methods given in this article can be applied to more general regression models such as fixed‐effect or random‐effect generalized linear regression models.     

Easy-to-construct confidence bands for comparing two simple linear regression lines

This paper shows how to construct confidence bands for the difference between two simple linear regression lines. These confidence bands provide directly the information on the magnitude of the difference between the regression lines over an interval of interest and, as a by-product, can be used as a formal test of the difference between the two regression lines. Various different shapes of confidence bands are illustrated, and particular attention is paid towards confidence bands whose construction only involves critical points from standard distributions so that they are consequently easy to construct.     

Simultaneous Confidence Tubes in Multivariate Linear Regression

Simultaneous confidence bands have been shown in the statistical literature as powerful inferential tools in univariate linear regression. While the methodology of simultaneous confidence bands for univariate linear regression has been extensively researched and well developed, no published work seems available for multivariate linear regression. This paper fills this gap by studying one particular simultaneous confidence band for multivariate linear regression. Because of the shape of the band, the word ‘tube’ is more pertinent and so will be used to replace the word ‘band’. It is shown that the construction of the tube is related to the distribution of the largest eigenvalue. A simulation‐based method is proposed to compute the 1 ? α quantile of this eigenvalue. With the computation power of modern computers, the simultaneous confidence tube can be computed fast and accurately. A real‐data example is used to illustrate the method, and many potential research problems have been pointed out.     

Constant width simultaneous confidence bands in multiple linear regression with predictor variables constrained in intervals

In the last fifty years, a great deal of research effort has been made on the construction of simultaneous confidence bands for a linear regression function. Two most frequently quoted confidence bands in the statistics literature are the Scheffé type and constant width bands over a given rectangular region of the predictor variables. For the constant width bands, a method is given by Gafarian [Gafarian, A.V., 1964, Confidence bands in straight line regression. Journal of the American Statistical Association, 59, 182–213.] for the calculation of critical constants only for the special case of one predictor variable. In this article, a method is proposed to construct constant width bands when there are any number of predictor variables. A new criterion for assessing a confidence band is also proposed; it is the probability that a confidence band excludes a false regression function and can be viewed as the power function of a test associated, naturally, with a confidence band. Under this criterion, a numerical comparison between the Scheffé type and constant width bands is then carried out. It emerges from this comparison that the constant width bands can be better than the Scheffé type bands for certain designs.     

Simultaneous Small Sample Inference for Linear Combinations of Generalized Linear Model Parameters

A method is proposed to construct simultaneous confidence intervals for multiple linear combinations of generalized linear model parameters, that uses a multivariate normal- or t-distribution together with the signed likelihood root statistic. In an application to a case study simultaneous confidence bands for logistic regression are calculated. A simulation study based on the example evaluation suggests superior performance compared to the common Wald-type approaches. The proposed methods are readily implemented in the R extension package mcprofile.     

Adjusted Confidence Bands in Nonparametric Regression

Suppose we have {(x _i , y _i )} i = 1, 2,…, n, a sequence of independent observations. We wish to find approximate 1 ? α simultaneous confidence bands for the regression curve. Many previous confidence bands in the literature have practical difficulties. In this article, the local linear smoother is used to estimate the regression curve. The bias of the estimator is considered. Different methods of constructing confidence bands are discussed. Finally, a possible method incorporating logistic regression in an innovative way is proposed to construct the bands for random designs. Simulations are used to study the performance or properties of the methods. The procedure for constructing confidence bands is entirely data-driven. The advantage of the proposed method is that it is simple to use and can be applied to random designs. It can be considered as a practically useful and efficient method.     

Confidence band for percentile line in multiple linear regression

We deal with the problem of estimating constructing a confidence band for the 100γth percentile line in the multiple linear regression model with independent identically normally distributed errors. A method for computing the exact Scheffé type confidence band over a limited space of the particular covariates region is suggested. A confidence band depends on an estimator of the percentile line. The confidence bands based on the different estimators of the percentile line are compared with respect to the average bandwidth.     

Empirical Likelihood for Linear Models Under Linear Process Errors

In this article, we study the construction of confidence intervals for regression parameters in a linear model under linear process errors by using the blockwise technique. It is shown that the blockwise empirical likelihood (EL) ratio statistic is asymptotically χ²-type distributed. The result is used to obtain EL based confidence regions for regression parameters. The finite-sample performance of the method is evaluated through a simulation study.     

Empirical Likelihood Inference for the Cox Model with Time-dependent Coefficients via Local Partial Likelihood

Abstract.  The Cox model with time-dependent coefficients has been studied by a number of authors recently. In this paper, we develop empirical likelihood (EL) pointwise confidence regions for the time-dependent regression coefficients via local partial likelihood smoothing. The EL simultaneous confidence bands for a linear combination of the coefficients are also derived based on the strong approximation methods. The EL ratio is formulated through the local partial log-likelihood for the regression coefficient functions. Our numerical studies indicate that the EL pointwise/simultaneous confidence regions/bands have satisfactory finite sample performances. Compared with the confidence regions derived directly based on the asymptotic normal distribution of the local constant estimator, the EL confidence regions are overall tighter and can better capture the curvature of the underlying regression coefficient functions. Two data sets, the gastric cancer data and the Mayo Clinic primary biliary cirrhosis data, are analysed using the proposed method.     

Empirical likelihood confidence bands for distribution functions with missing responses

Distribution function estimation plays a significant role of foundation in statistics since the population distribution is always involved in statistical inference and is usually unknown. In this paper, we consider the estimation of the distribution function of a response variable Y with missing responses in the regression problems. It is proved that the augmented inverse probability weighted estimator converges weakly to a zero mean Gaussian process. A augmented inverse probability weighted empirical log-likelihood function is also defined. It is shown that the empirical log-likelihood converges weakly to the square of a Gaussian process with mean zero and variance one. We apply these results to the construction of Gaussian process approximation based confidence bands and empirical likelihood based confidence bands of the distribution function of Y. A simulation is conducted to evaluate the confidence bands.     

Confidence sets in a linear regression model for interval data

The construction of confidence sets for the parameters of a flexible simple linear regression model for interval-valued random sets is addressed. For that purpose, the asymptotic distribution of the least-squares estimators is analyzed. A simulation study is conducted to investigate the performance of those confidence sets. In particular, the empirical coverages are examined for various interval linear models. The applicability of the procedure is illustrated by means of a real-life case study.     

Simultaneous confidence bands and regions for log-location-scale distributions with censored data

In many areas of application, especially life testing and reliability, it is often of interest to estimate an unknown cumulative distribution (cdf). A simultaneous confidence band (SCB) of the cdf can be used to assess the statistical uncertainty of the estimated cdf over the entire range of the distribution. Cheng and Iles [1983. Confidence bands for cumulative distribution functions of continuous random variables. Technometrics 25 (1), 77–86] presented an approach to construct an SCB for the cdf of a continuous random variable. For the log-location-scale family of distributions, they gave explicit forms for the upper and lower boundaries of the SCB based on expected information. In this article, we extend the work of Cheng and Iles [1983. Confidence bands for cumulative distribution functions of continuous random variables. Technometrics 25 (1), 77–86] in several directions. We study the SCBs based on local information, expected information, and estimated expected information for both the “cdf method” and the “quantile method.” We also study the effects of exceptional cases where a simple SCB does not exist. We describe calibration of the bands to provide exact coverage for complete data and type II censoring and better approximate coverage for other kinds of censoring. We also discuss how to extend these procedures to regression analysis.     

Testing for a change point in linear regression models

In this paper, the Schwarz Information Criterion (SIC) is proposed to locate a change point in the simple linear regression model, as well as in the multiple linear regression model. The method is then applied to a financial data set, and a change point is successfully detected.     

Using adaptive weighted least squares to reduce the lengths of confidence intervals

The author proposes an adaptive method which produces confidence intervals that are often narrower than those obtained by the traditional procedures. The proposed methods use both a weighted least squares approach to reduce the length of the confidence interval and a permutation technique to insure that its coverage probability is near the nominal level. The author reports simulations comparing the adaptive intervals to the traditional ones for the difference between two population means, for the slope in a simple linear regression, and for the slope in a multiple linear regression having two correlated exogenous variables. He is led to recommend adaptive intervals for sample sizes superior to 40 when the error distribution is not known to be Gaussian.     

Nonlinear surface regression with dimension reduction method

This paper considers nonlinear regression analysis with a scalar response and multiple predictors. An unknown regression function is approximated by radial basis function models. The coefficients are estimated in the context of M-estimation. It is known that ordinary M-estimation leads to overfitting in nonlinear regression. The purpose of this paper is to construct a smooth estimator. The proposed method in this paper is conducted by a two-step procedure. First, the sufficient dimension reduction methods are applied to the response and radial basis functions for transforming the large number of radial bases to a small number of linear combinations of the radial bases without loss of information. In the second step, a multiple linear regression model between a response and the transformed radial bases is assumed and the ordinary M-estimation is applied. Thus, the final estimator is also obtained as a linear combination of radial bases. The validity and an asymptotic study of the proposed method are explored. A simulation and data example are addressed to confirm the behavior of the proposed method.     

The minimum coverage probability of confidence intervals in regression after a preliminary F test

Consider a linear regression model with regression parameter β=(β₁,…,β_p) and independent normal errors. Suppose the parameter of interest is θ=a^Tβ, where a is specified. Define the s-dimensional parameter vector τ=C^Tβ−t, where C and t are specified. Suppose that we carry out a preliminary F test of the null hypothesis H₀:τ=0 against the alternative hypothesis H₁:τ≠0. It is common statistical practice to then construct a confidence interval for θ with nominal coverage 1−α, using the same data, based on the assumption that the selected model had been given to us a priori (as the true model). We call this the naive 1−α confidence interval for θ. This assumption is false and it may lead to this confidence interval having minimum coverage probability far below 1−α, making it completely inadequate. We provide a new elegant method for computing the minimum coverage probability of this naive confidence interval, that works well irrespective of how large s is. A very important practical application of this method is to the analysis of covariance. In this context, τ can be defined so that H₀ expresses the hypothesis of “parallelism”. Applied statisticians commonly recommend carrying out a preliminary F test of this hypothesis. We illustrate the application of our method with a real-life analysis of covariance data set and a preliminary F test for “parallelism”. We show that the naive 0.95 confidence interval has minimum coverage probability 0.0846, showing that it is completely inadequate.     

Simultaneous bootstrap confidence bands in regression

We suggest pivotal methods for constructing simultaneous bootstrap confidence bands in regression. Most attention is given to the problem of simple linear regression, but our techniques admit trivial extension to other cases, including polynomial regression. The advantages of our bootstrap approach are twofold. Firstly, the bootstrap allows a very general distribution for the errors, and secondly, it admits a wide variety of shapes for the confidence band. In our technique the shape of each envelope of the band is determined by a general template, chosen by the experimenter, and bootstrap methods are used to select the scale of the template.