Efficient Estimation of an Additive Quantile Regression Model

Abstract. In this paper, two non‐parametric estimators are proposed for estimating the components of an additive quantile regression model. The first estimator is a computationally convenient approach which can be viewed as a more viable alternative to existing kernel‐based approaches. The second estimator involves sequential fitting by univariate local polynomial quantile regressions for each additive component with the other additive components replaced by the corresponding estimates from the first estimator. The purpose of the extra local averaging is to reduce the variance of the first estimator. We show that the second estimator achieves oracle efficiency in the sense that each estimated additive component has the same variance as in the case when all other additive components were known. Asymptotic properties are derived for both estimators under dependent processes that are strictly stationary and absolutely regular. We also provide a demonstrative empirical application of additive quantile models to ambulance travel times.     

Quantile regression estimation of partially linear additive models

In this paper, we consider the estimation of partially linear additive quantile regression models where the conditional quantile function comprises a linear parametric component and a nonparametric additive component. We propose a two-step estimation approach: in the first step, we approximate the conditional quantile function using a series estimation method. In the second step, the nonparametric additive component is recovered using either a local polynomial estimator or a weighted Nadaraya–Watson estimator. Both consistency and asymptotic normality of the proposed estimators are established. Particularly, we show that the first-stage estimator for the finite-dimensional parameters attains the semiparametric efficiency bound under homoskedasticity, and that the second-stage estimators for the nonparametric additive component have an oracle efficiency property. Monte Carlo experiments are conducted to assess the finite sample performance of the proposed estimators. An application to a real data set is also illustrated.     

Boundary and Bias Correction in Kernel Hazard Estimation

A new class of local linear hazard estimators based on weighted least square kernel estimation is considered. The class includes the kernel hazard estimator of Ramlau-Hansen (1983), which has the same boundary correction property as the local linear regression estimator (see Fan & Gijbels, 1996). It is shown that all the local linear estimators in the class have the same pointwise asymptotic properties. We derive the multiplicative bias correction of the local linear estimator. In addition we propose a new bias correction technique based on bootstrap estimation of additive bias. This latter method has excellent theoretical properties. Based on an extensive simulation study where we compare the performance of competing estimators, we also recommend the use of the additive bias correction in applied work.     

Local polynomial quasi-likelihood regression with truncated and dependent data

The local polynomial quasi-likelihood estimation has several good statistical properties such as high minimax efficiency and adaptation of edge effects. In this paper, we construct a local quasi-likelihood regression estimator for a left truncated model, and establish the asymptotic normality of the proposed estimator when the observations form a stationary and α-mixing sequence, such that the corresponding result of Fan et al. [Local polynomial kernel regression for generalized linear models and quasilikelihood functions, J. Amer. Statist. Assoc. 90 (1995), pp. 141–150] is extended from the independent and complete data to the dependent and truncated one. Finite sample behaviour of the estimator is investigated via simulations too.     

Understanding Estimators of Treatment Effects in Regression Discontinuity Designs

In this paper, we propose two new estimators of treatment effects in regression discontinuity designs. These estimators can aid understanding of the existing estimators such as the local polynomial estimator and the partially linear estimator. The first estimator is the partially polynomial estimator which extends the partially linear estimator by further incorporating derivative differences of the conditional mean of the outcome on the two sides of the discontinuity point. This estimator is related to the local polynomial estimator by a relocalization effect. Unlike the partially linear estimator, this estimator can achieve the optimal rate of convergence even under broader regularity conditions. The second estimator is an instrumental variable estimator in the fuzzy design. This estimator will reduce to the local polynomial estimator if higher order endogeneities are neglected. We study the asymptotic properties of these two estimators and conduct simulation studies to confirm the theoretical analysis.     

Nonlinear regression models with general distortion measurement errors

This paper considers nonlinear regression models when neither the response variable nor the covariates can be directly observed, but are measured with both multiplicative and additive distortion measurement errors. We propose conditional variance and conditional mean calibration estimation methods for the unobserved variables, then a nonlinear least squares estimator is proposed. For the hypothesis testing of parameter, a restricted estimator under the null hypothesis and a test statistic are proposed. The asymptotic properties for the estimator and test statistic are established. Lastly, a residual-based empirical process test statistic marked by proper functions of the regressors is proposed for the model checking problem. We further suggest a bootstrap procedure to calculate critical values. Simulation studies demonstrate the performance of the proposed procedure and a real example is analysed to illustrate its practical usage.     

Local likelihood with time-varying additive hazards model

The authors propose the local likelihood method for the time-varying coefficient additive hazards model. They use the Newton-Raphson algorithm to maximize the likelihood into which a local polynomial expansion has been incorporated. They establish the asymptotic properties for the time-varying coefficient estimators and derive explicit expressions for the variance and bias. The authors present simulation results describing the performance of their approach for finite sample sizes. Their numerical comparisons show the stability and efficiency of the local maximum likelihood estimator. They finally illustrate their proposal with data from a laryngeal cancer clinical study.     

An alternative local polynomial estimator for the error-in-variables problem

We consider the problem of estimating a regression function when a covariate is measured with error. Using the local polynomial estimator of Delaigle et al. [(2009), ‘A Design-adaptive Local Polynomial Estimator for the Errors-in-variables Problem’, Journal of the American Statistical Association, 104, 348–359] as a benchmark, we propose an alternative way of solving the problem without transforming the kernel function. The asymptotic properties of the alternative estimator are rigorously studied. A detailed implementing algorithm and a computationally efficient bandwidth selection procedure are also provided. The proposed estimator is compared with the existing local polynomial estimator via extensive simulations and an application to the motorcycle crash data. The results show that the new estimator can be less biased than the existing estimator and is numerically more stable.     

A Seemingly Unrelated Nonparametric Additive Model with Autoregressive Errors

This article considers a nonparametric additive seemingly unrelated regression model with autoregressive errors, and develops estimation and inference procedures for this model. Our proposed method first estimates the unknown functions by combining polynomial spline series approximations with least squares, and then uses the fitted residuals together with the smoothly clipped absolute deviation (SCAD) penalty to identify the error structure and estimate the unknown autoregressive coefficients. Based on the polynomial spline series estimator and the fitted error structure, a two-stage local polynomial improved estimator for the unknown functions of the mean is further developed. Our procedure applies a prewhitening transformation of the dependent variable, and also takes into account the contemporaneous correlations across equations. We show that the resulting estimator possesses an oracle property, and is asymptotically more efficient than estimators that neglect the autocorrelation and/or contemporaneous correlations of errors. We investigate the small sample properties of the proposed procedure in a simulation study.     

Efficient Estimation of the Partly Linear Additive Hazards Model with Current Status Data

This paper focuses on efficient estimation, optimal rates of convergence and effective algorithms in the partly linear additive hazards regression model with current status data. We use polynomial splines to estimate both cumulative baseline hazard function with monotonicity constraint and nonparametric regression functions with no such constraint. We propose a simultaneous sieve maximum likelihood estimation for regression parameters and nuisance parameters and show that the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound. In addition, we show that rates of convergence for the estimators of nonparametric functions are optimal. We implement the proposed estimation through a backfitting algorithm on generalized linear models. We conduct simulation studies to examine the finite‐sample performance of the proposed estimation method and present an analysis of renal function recovery data for illustration.     

Non-parametric Quantile Regression with Censored Data

Abstract.  Censored regression models have received a great deal of attention in both the theoretical and applied statistics literature. Here, we consider a model in which the response variable is censored but not the covariates. We propose a new estimator of the conditional quantiles based on the local linear method, and give an algorithm for its numerical implementation. We study its asymptotic properties and evaluate its performance on simulated data sets.     

Estimation of the error distribution in a varying coefficient regression model

This paper deals with the estimation of the error distribution function in a varying coefficient regression model. We propose two estimators and study their asymptotic properties by obtaining uniform stochastic expansions. The first estimator is a residual-based empirical distribution function. We study this estimator when the varying coefficients are estimated by under-smoothed local quadratic smoothers. Our second estimator which exploits the fact that the error distribution has mean zero is a weighted residual-based empirical distribution whose weights are chosen to achieve the mean zero property using empirical likelihood methods. The second estimator improves on the first estimator. Bootstrap confidence bands based on the two estimators are also discussed.     

Statistical inference for varying-coefficient models with error-prone covariates

Motivated by an application, we consider the statistical inference of varying-coefficient regression models in which some covariates are not observed, but ancillary variables are available to remit them. Due to the attenuation, the usual local polynomial estimation of the coefficient functions is not consistent. We propose a corrected local polynomial estimation for the unknown coefficient functions by calibrating the error-prone covariates. It is shown that the resulting estimators are consistent and asymptotically normal. In addition, we develop a wild bootstrap test for the goodness of fit of models. Some simulations are conducted to demonstrate the finite sample performances of the proposed estimation and test procedures. An example of application on a real data from Duchenne muscular dystrophy study is also illustrated.     

Bias-corrected confidence bands in nonparametric regression

Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya–Watson estimator. The results are also applicable to nonparametric autoregressive time series models.     

A Quantile Regression Model for Time-Series Data in the Presence of Additive Components

In this article, we propose a kernel-based estimator for the finite-dimensional parameter of a partially additive linear quantile regression model. For dependent processes that are strictly stationary and absolutely regular, we establish a precise convergent rate and show that the estimator is root-n consistent and asymptotically normal. To help facilitate inferential procedures, a consistent estimator for the asymptotic variance is also provided. In addition to conducting a simulation experiment to evaluate the finite sample performance of the estimator, an application to US inflation is presented. We use the real-data example to motivate how partially additive linear quantile models can offer an alternative modeling option for time-series data.     

Prediction Error Estimation Under Bregman Divergence for Non-Parametric Regression and Classification

Abstract.  Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross-validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data-driven optimal bandwidth selector for local-likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean-prediction-error-criterion. We also propose a lower-bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying-coefficient models and additive models.     

Local Linear Additive Quantile Regression

Abstract.  We consider non-parametric additive quantile regression estimation by kernel-weighted local linear fitting. The estimator is based on localizing the characterization of quantile regression as the minimizer of the appropriate 'check function'. A backfitting algorithm and a heuristic rule for selecting the smoothing parameter are explored. We also study the estimation of average-derivative quantile regression under the additive model. The techniques are illustrated by a simulated example and a real data set.     

Corrected local polynomial estimation in varying‐coefficient models with measurement errors

The authors study a varying‐coefficient regression model in which some of the covariates are measured with additive errors. They find that the usual local linear estimator (LLE) of the coefficient functions is biased and that the usual correction for attenuation fails to work. They propose a corrected LLE and show that it is consistent and asymptotically normal, and they also construct a consistent estimator for the model error variance. They then extend the generalized likelihood technique to develop a goodness of fit test for the model. They evaluate these various procedures through simulation studies and use them to analyze data from the Framingham Heart Study.     

Statistical Inference on Partially Linear Additive Models with Missing Response Variables and Error-prone Covariates

This paper considers statistical inference for the partially linear additive models, which are useful extensions of additive models and partially linear models. We focus on the case where some covariates are measured with additive errors, and the response variable is sometimes missing. We propose a profile least-squares estimator for the parametric component and show that the resulting estimator is asymptotically normal. To construct a confidence region for the parametric component, we also propose an empirical-likelihood-based statistic, which is shown to have a chi-squared distribution asymptotically. Furthermore, a simulation study is conducted to illustrate the performance of the proposed methods.     

Local CQR Smoothing: An Efficient and Safe Alternative to Local Polynomial Regression

Summary.  Local polynomial regression is a useful non-parametric regression tool to explore fine data structures and has been widely used in practice. We propose a new non-parametric regression technique called local composite quantile regression smoothing to improve local polynomial regression further. Sampling properties of the estimation procedure proposed are studied. We derive the asymptotic bias, variance and normality of the estimate proposed. The asymptotic relative efficiency of the estimate with respect to local polynomial regression is investigated. It is shown that the estimate can be much more efficient than the local polynomial regression estimate for various non-normal errors, while being almost as efficient as the local polynomial regression estimate for normal errors. Simulation is conducted to examine the performance of the estimates proposed. The simulation results are consistent with our theoretical findings. A real data example is used to illustrate the method proposed.