Symmetric regression quantile and its application to robust estimation for the nonlinear regression model

Populational conditional quantiles in terms of percentage α are useful as indices for identifying outliers. We propose a class of symmetric quantiles for estimating unknown nonlinear regression conditional quantiles. In large samples, symmetric quantiles are more efficient than regression quantiles considered by Koenker and Bassett (Econometrica 46 (1978) 33) for small or large values of α, when the underlying distribution is symmetric, in the sense that they have smaller asymptotic variances. Symmetric quantiles play a useful role in identifying outliers. In estimating nonlinear regression parameters by symmetric trimmed means constructed by symmetric quantiles, we show that their asymptotic variances can be very close to (or can even attain) the Cramer–Rao lower bound under symmetric heavy-tailed error distributions, whereas the usual robust and nonrobust estimators cannot.     

Autoregression quantiles and related rank score processes for generalized random coefficient autoregressive processes

Regression quantiles were developed by Koenker and Bassett (Econometrica 46 (1978), 33–50); they provide natural and extremely useful counterparts of the sample quantiles in general linear models. The regression rank scores were introduced by Gutenbrunner and Jurečková (Ann. Statist. 8 (1992), 305–329) as dual variables to regression quantiles. Koul and Saleh (Ann. Statist. 23 (1995), 670–689) developed the procedures based on the regression quantiles of Koenker and Bassett (Econometrica 46 (1978), 33–50) and the regression rank scores of Gutenbrunner and Jurečková Ann. Statist. 8 (1992), 305–329 in linear regression to the pth-order autoregression models. In this paper, we further develop and investigate the analogs of these procedures to a larger class of processes and derive a test for a bilinear model without estimating the bilinear coefficient and the autoregression constants.     

Nonparametric estimation of extreme conditional quantiles

The estimation of extreme conditional quantiles is an important issue in different scientific disciplines. Up to now, the extreme value literature focused mainly on estimation procedures based on independent and identically distributed samples. Our contribution is a two-step procedure for estimating extreme conditional quantiles. In a first step nonextreme conditional quantiles are estimated nonparametrically using a local version of [Koenker, R. and Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33–50.] regression quantile methodology. Next, these nonparametric quantile estimates are used as analogues of univariate order statistics in procedures for extreme quantile estimation. The performance of the method is evaluated for both heavy tailed distributions and distributions with a finite right endpoint using a small sample simulation study. A bootstrap procedure is developed to guide in the selection of an optimal local bandwidth. Finally the procedure is illustrated in two case studies.     

Asymptotic Normality of M-Estimators for Varying Coefficient Models with Longitudinal Data

This article considers a nonparametric varying coefficient regression model with longitudinal observations. The relationship between the dependent variable and the covariates is assumed to be linear at a specific time point, but the coefficients are allowed to change over time. A general formulation is used to treat mean regression, median regression, quantile regression, and robust mean regression in one setting. The local M-estimators of the unknown coefficient functions are obtained by local linear method. The asymptotic distributions of M-estimators of unknown coefficient functions at both interior and boundary points are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are derived. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

Estimation and inference for varying coefficient partially nonlinear models

In this paper, we extend the varying coefficient partially linear model to the varying coefficient partially nonlinear model in which the linear part of the varying coefficient partially linear model is replaced by a nonlinear function of the covariates. A profile nonlinear least squares estimation procedure for the parameter vector and the coefficient function vector of the varying coefficient partially nonlinear model is proposed and the asymptotic properties of the resulting estimators are established. We further propose a generalized likelihood ratio (GLR) test to check whether or not the varying coefficients in the model are constant. The asymptotic null distribution of the GLR statistic is derived and a residual-based bootstrap procedure is also suggested to derive the p-value of the GLR test. Some simulations are conducted to assess the performance of the proposed estimating and testing procedures and the results show that both the procedures perform well in finite samples. Furthermore, a real data example is given to demonstrate the application of the proposed model and its estimating and testing procedures.     

Efficient quantile regression for heteroscedastic models

Quantile regression (QR) provides estimates of a range of conditional quantiles. This stands in contrast to traditional regression techniques, which focus on a single conditional mean function. Lee et al. [Regularization of case-specific parameters for robustness and efficiency. Statist Sci. 2012;27(3):350–372] proposed efficient QR by rounding the sharp corner of the loss. The main modification generally involves an asymmetric ?₂ adjustment of the loss function around zero. We extend the idea of ?₂ adjusted QR to linear heterogeneous models. The ?₂ adjustment is constructed to diminish as sample size grows. Conditions to retain consistency properties are also provided.     

Estimation of general semi-parametric quantile regression

Quantile regression introduced by Koenker and Bassett (1978) produces a comprehensive picture of a response variable on predictors. In this paper, we propose a general semi-parametric model of which part of predictors are presented with a single-index, to model the relationship of conditional quantiles of the response on predictors. Special cases are single-index models, partially linear single-index models and varying coefficient single-index models. We propose the qOPG, a quantile regression version of outer-product gradient estimation method (OPG, Xia et al., 2002) to estimate the single-index. Large-sample properties, simulation results and a real-data analysis are provided to examine the performance of the qOPG.     

A monte carlo of plotting positions

A Monte Carlo study was made of the effects of using simple linear regression, on the appropriate probability paper, to estimate parameters, quantiles and cumulative probability for several distributions. These distributions were the Normal, Weibull (shape parameters 1, 2, and 4) and the Type I largest extreme-value distributions. The specific objective was to observe differences arising from choice of plotting positions. Plotting positions used were i/(n+l), (i?3)/(n+.04), (i?.5)/n, either (i?.375)/(n+.25) or (i?.4)/(n+.2), and either F[E(Y_i)] or F[E(£n Y)]. For each combination of 4 sample sizes (n=10(10)(40)), distribution, and plotting position, regression lines were found for each of N =9999 samples. Each regression line was used to estimate: (1) quantiles of 9 specific probabilities, (2) probabilities of 9 specific quantiles, and (3) return periods corresponding to 9 specific quantiles. Compa[rgrave]ison of the mean, variances, mean square error and medians of these estimates and of the regression coefficients confirm some results of Harter [Commun. Statist. A13(13), 1984] and provide further insight.     

Extreme Quantile Estimation for Autoregressive Models

ABSTRACT

A quantile autoregresive model is a useful extension of classical autoregresive models as it can capture the influences of conditioning variables on the location, scale, and shape of the response distribution. However, at the extreme tails, standard quantile autoregression estimator is often unstable due to data sparsity. In this article, assuming quantile autoregresive models, we develop a new estimator for extreme conditional quantiles of time series data based on extreme value theory. We build the connection between the second-order conditions for the autoregression coefficients and for the conditional quantile functions, and establish the asymptotic properties of the proposed estimator. The finite sample performance of the proposed method is illustrated through a simulation study and the analysis of U.S. retail gasoline price.     

Partially adaptive quantile estimators

This paper contrasts two approaches to estimating quantile regression models: traditional semi-parametric methods and partially adaptive estimators using flexible probability density functions (pdfs). While more general pdfs could have been used, the skewed Laplace was selected for pedagogical purposes. Monte Carlo simulations are used to compare the behavior of the semi-parametric and partially adaptive quantile estimators in the presence of possibly skewed and heteroskedastic data. Both approaches accommodate skewness and heteroskedasticity which are consistent with linear quantiles; however, the partially adaptive estimator considered allows for non linear quantiles and also provides simple tests for symmetry and heteroskedasticity. The methods are applied to the problem of estimating conditional quantile functions for wages corresponding to different levels of education.     

Sequential design for nonparametric inference

The performance of nonparametric function estimates often depends on the choice of design points. Based on the mean integrated squared error criterion, we propose a sequential design procedure that updates the model knowledge and optimal design density sequentially. The methodology is developed under a general framework covering a wide range of nonparametric inference problems, such as conditional mean and variance functions, the conditional distribution function, the conditional quantile function in quantile regression, functional coefficients in varying coefficient models and semiparametric inferences. Based on our empirical studies, nonparametric inference based on the proposed sequential design is more efficient than the uniform design and its performance is close to the true but unknown optimal design. The Canadian Journal of Statistics 40: 362–377; 2012 © 2012 Statistical Society of Canada     

Penalized least-squares estimation for regression coefficients in high-dimensional partially linear models

We consider a partially linear model with diverging number of groups of parameters in the parametric component. The variable selection and estimation of regression coefficients are achieved simultaneously by using the suitable penalty function for covariates in the parametric component. An MM-type algorithm for estimating parameters without inverting a high-dimensional matrix is proposed. The consistency and sparsity of penalized least-squares estimators of regression coefficients are discussed under the setting of some nonzero regression coefficients with very small values. It is found that the root p_n/n-consistency and sparsity of the penalized least-squares estimators of regression coefficients cannot be given consideration simultaneously when the number of nonzero regression coefficients with very small values is unknown, where p_n and n, respectively, denote the number of regression coefficients and sample size. The finite sample behaviors of penalized least-squares estimators of regression coefficients and the performance of the proposed algorithm are studied by simulation studies and a real data example.     

Shrinkage estimation of varying covariate effects based on quantile regression

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l ₁-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.     

Rank reducible varying coefficient model

We propose in this article a novel dimension reduction method for varying coefficient models. The proposed method explores the rank reducible structure of those varying coefficients, hence, can do dimension reduction and semiparametric estimation, simultaneously. As a result, the new method not only improves estimation accuracy but also facilitates practical interpretation. To determine the structure dimension, a consistent BIC criterion is developed. Numerical experiments are also presented.     

Penalized quantile regression for dynamic panel data

This paper studies penalized quantile regression for dynamic panel data with fixed effects, where the penalty involves l₁ shrinkage of the fixed effects. Using extensive Monte Carlo simulations, we present evidence that the penalty term reduces the dynamic panel bias and increases the efficiency of the estimators. The underlying intuition is that there is no need to use instrumental variables for the lagged dependent variable in the dynamic panel data model without fixed effects. This provides an additional use for the shrinkage models, other than model selection and efficiency gains. We propose a Bayesian information criterion based estimator for the parameter that controls the degree of shrinkage. We illustrate the usefulness of the novel econometric technique by estimating a “target leverage” model that includes a speed of capital structure adjustment. Using the proposed penalized quantile regression model the estimates of the adjustment speeds lie between 3% and 44% across the quantiles, showing strong evidence that there is substantial heterogeneity in the speed of adjustment among firms.     

Nonlinear recursive estimation of volatility via estimating functions

For certain volatility models, the conditional moments that depend on the parameter are of interest. Following Godambe and Heyde (1987), the combined estimating function method has been used to study inference when the conditional mean and conditional variance are functions of the parameter of interest (See Ghahramani and Thavaneswaran [Combining Estimating Functions for Volatility. Journal of Statistical Planning and Inference, 2009, 139, 1449-1461] for details). However, for application purposes, the resulting estimates are nonlinear functions of the observations and no closed form expressions of the estimates are available. As an alternative, in this paper, a recursive estimation approach based on the combined estimating function is proposed and applied to various classes of time series models, including certain volatility models.     

A note on nonparametric estimation of circular conditional densities

ABSTRACT

The conditional density offers the most informative summary of the relationship between explanatory and response variables. We need to estimate it in place of the simple conditional mean when its shape is not well-behaved. A motivation for estimating conditional densities, specific to the circular setting, lies in the fact that a natural alternative of it, like quantile regression, could be considered problematic because circular quantiles are not rotationally equivariant. We treat conditional density estimation as a local polynomial fitting problem as proposed by Fan et al. [Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika. 1996;83:189–206] in the Euclidean setting, and discuss a class of estimators in the cases when the conditioning variable is either circular or linear. Asymptotic properties for some members of the proposed class are derived. The effectiveness of the methods for finite sample sizes is illustrated by simulation experiments and an example using real data.     

Re-weighting estimation of the coefficients in the varying coefficient model with heteroscedastic errors

ABSTRACT

This article explores the estimation problem of the coefficients in the varying coefficient model with heteroscedastic errors. Specifically, we first present a method for estimating the variance function of the error term and the resulting estimator is proved to be consistent. Then, motivated by the generalized least-squares procedure for dealing with heteroscedasticity in the linear regression literature, we re-weight each squared residual term in the local linear smoother with the inverse of the corresponding estimated error variance to construct estimates of the coefficients. Simulation experiments and practical data analysis conducted demonstrate that the re-weighting approach can improve the accuracy of the coefficient estimates under a finite sample size, especially when the error heteroscedasticity is strong.     

The predictive Lasso

We propose a shrinkage procedure for simultaneous variable selection and estimation in generalized linear models (GLMs) with an explicit predictive motivation. The procedure estimates the coefficients by minimizing the Kullback-Leibler divergence of a set of predictive distributions to the corresponding predictive distributions for the full model, subject to an l ₁ constraint on the coefficient vector. This results in selection of a parsimonious model with similar predictive performance to the full model. Thanks to its similar form to the original Lasso problem for GLMs, our procedure can benefit from available l ₁-regularization path algorithms. Simulation studies and real data examples confirm the efficiency of our method in terms of predictive performance on future observations.     

SiZer inference for generalized varying coefficient models

ABSTRACT

SiZer (significant zero crossings of derivatives) is an effective tool for exploring significant features in curves from the viewpoint of the scale space theory. In this paper, a SiZer approach is developed for generalized varying coefficient models (GVCMs) in order to achieve the task of understanding dynamic characteristics of the regression relationship at multiscales. The proposed SiZer method is based on the local-linear maximum likelihood estimation of GVCMs and the one-step estimation procedure is employed to alleviate the computational cost of estimating the coefficients and their derivatives at different scales. Simulation studies are performed to assess the performance of the SiZer inference and two real-world examples are given to demonstrate its applications.