Symmetric regression quantile and its application to robust estimation for the nonlinear regression model

Populational conditional quantiles in terms of percentage α are useful as indices for identifying outliers. We propose a class of symmetric quantiles for estimating unknown nonlinear regression conditional quantiles. In large samples, symmetric quantiles are more efficient than regression quantiles considered by Koenker and Bassett (Econometrica 46 (1978) 33) for small or large values of α, when the underlying distribution is symmetric, in the sense that they have smaller asymptotic variances. Symmetric quantiles play a useful role in identifying outliers. In estimating nonlinear regression parameters by symmetric trimmed means constructed by symmetric quantiles, we show that their asymptotic variances can be very close to (or can even attain) the Cramer–Rao lower bound under symmetric heavy-tailed error distributions, whereas the usual robust and nonrobust estimators cannot.     

A monte carlo of plotting positions

A Monte Carlo study was made of the effects of using simple linear regression, on the appropriate probability paper, to estimate parameters, quantiles and cumulative probability for several distributions. These distributions were the Normal, Weibull (shape parameters 1, 2, and 4) and the Type I largest extreme-value distributions. The specific objective was to observe differences arising from choice of plotting positions. Plotting positions used were i/(n+l), (i?3)/(n+.04), (i?.5)/n, either (i?.375)/(n+.25) or (i?.4)/(n+.2), and either F[E(Y_i)] or F[E(£n Y)]. For each combination of 4 sample sizes (n=10(10)(40)), distribution, and plotting position, regression lines were found for each of N =9999 samples. Each regression line was used to estimate: (1) quantiles of 9 specific probabilities, (2) probabilities of 9 specific quantiles, and (3) return periods corresponding to 9 specific quantiles. Compa[rgrave]ison of the mean, variances, mean square error and medians of these estimates and of the regression coefficients confirm some results of Harter [Commun. Statist. A13(13), 1984] and provide further insight.     

Trimmed least squares estimator as best trimmed linear conditional estimator for linear regression model

A class of trimmed linear conditional estimators based on regression quantiles for the linear regression model is introduced. This class serves as a robust analogue of non-robust linear unbiased estimators. Asymptotic analysis then shows that the trimmed least squares estimator based on regression quantiles ( Koenker and Bassett ( 1978 ) ) is the best in this estimator class in terms of asymptotic covariance matrices. The class of trimmed linear conditional estimators contains the Mallows-type bounded influence trimmed means ( see De Jongh et al ( 1988 ) ) and trimmed instrumental variables estimators. A large sample methodology based on trimmed instrumental variables estimator for confidence ellipsoids and hypothesis testing is also provided.     

Efficient quantile regression for heteroscedastic models

Quantile regression (QR) provides estimates of a range of conditional quantiles. This stands in contrast to traditional regression techniques, which focus on a single conditional mean function. Lee et al. [Regularization of case-specific parameters for robustness and efficiency. Statist Sci. 2012;27(3):350–372] proposed efficient QR by rounding the sharp corner of the loss. The main modification generally involves an asymmetric ?₂ adjustment of the loss function around zero. We extend the idea of ?₂ adjusted QR to linear heterogeneous models. The ?₂ adjustment is constructed to diminish as sample size grows. Conditions to retain consistency properties are also provided.     

Composite quantile regression for varying-coefficient single-index models

ABSTRACT

The varying-coefficient single-index model (VCSIM) is a very general and flexible tool for exploring the relationship between a response variable and a set of predictors. Popular special cases include single-index models and varying-coefficient models. In order to estimate the index-coefficient and the non parametric varying-coefficients in the VCSIM, we propose a two-stage composite quantile regression estimation procedure, which integrates the local linear smoothing method and the information of quantile regressions at a number of conditional quantiles of the response variable. We establish the asymptotic properties of the proposed estimators for the index-coefficient and varying-coefficients when the error is heterogeneous. When compared with the existing mean-regression-based estimation method, our simulation results indicate that our proposed method has comparable performance for normal error and is more robust for error with outliers or heavy tail. We illustrate our methodologies with a real example.     

Adaptive parametric test in a semiparametric regression model

Consider the semiparametric regression model Y_i = x′_iβ +g(t_i)+e_i for i=1,2, …,n. Here the design points (x_i,t_i) are known and nonrandom and the e_i are iid random errors with Ee₁ = 0 and Ee² ₁ = α²<∞. Based on g(.) approximated by a B-spline function, we consider using atest statistic for testing H₀ : β = 0. Meanwhile, an adaptive parametric test statistic is constructed and a large sample study for this adaptive parametric test statistic is presented.     

Robust nonparametric estimation for spatial regression

In this paper, we investigate a nonparametric robust estimation for spatial regression. More precisely, given a strictly stationary random field _Zi=(_Xi,_{Yi)i∈NNN≥1}$Z_{\mathbf{i}}=(X_{\mathbf{i}},Y_{\mathbf{i}}{)}_{\mathbf{i}\in {\mathbb{N}}^{N}N\ge 1}$, we consider a family of robust nonparametric estimators for a regression function based on the kernel method. Under some general mixing assumptions, the almost complete consistency and the asymptotic normality of these estimators are obtained. A robust procedure to select the smoothing parameter adapted to the spatial data is also discussed.     

Asymptotic distribution of regression correlation coefficient for Poisson regression model

This study treats an asymptotic distribution for measures of predictive power for generalized linear models (GLMs). We focus on the regression correlation coefficient (RCC) that is one of the measures of predictive power. The RCC, proposed by Zheng and Agresti is a population value and a generalization of the population value for the coefficient of determination. Therefore, the RCC is easy to interpret and familiar. Recently, Takahashi and Kurosawa provided an explicit form of the RCC and proposed a new RCC estimator for a Poisson regression model. They also showed the validity of the new estimator compared with other estimators. This study discusses the new statistical properties of the RCC for the Poisson regression model. Furthermore, we show an asymptotic normality of the RCC estimator.     

Consistency of least-squares estimator and its jackknife variance estimator in nonlinear models

The purpose of this paper is twofold: (1) We establish the consistency of the least-squares estimator in a nonlinear modely_i = f(x_i,θ) +σ_ie_i where the range of the parameter θ is noncompact, the regression function is unbounded, and the σ_i,'s are not necessarily equal. This extends the results in Jennrich (1969) and Wu (1981). (2) Under the same model, the jackknife estimator of the asymptotic covariance matrix of the least-squares estimator is shown to be consistent, which provides a theoretical justification of the empirical results in Duncan (1978) and the use of the jackknife method in large-sample inferences.     

Shrinkage estimation of varying covariate effects based on quantile regression

Varying covariate effects often manifest meaningful heterogeneity in covariate-response associations. In this paper, we adopt a quantile regression model that assumes linearity at a continuous range of quantile levels as a tool to explore such data dynamics. The consideration of potential non-constancy of covariate effects necessitates a new perspective for variable selection, which, under the assumed quantile regression model, is to retain variables that have effects on all quantiles of interest as well as those that influence only part of quantiles considered. Current work on l ₁-penalized quantile regression either does not concern varying covariate effects or may not produce consistent variable selection in the presence of covariates with partial effects, a practical scenario of interest. In this work, we propose a shrinkage approach by adopting a novel uniform adaptive LASSO penalty. The new approach enjoys easy implementation without requiring smoothing. Moreover, it can consistently identify the true model (uniformly across quantiles) and achieve the oracle estimation efficiency. We further extend the proposed shrinkage method to the case where responses are subject to random right censoring. Numerical studies confirm the theoretical results and support the utility of our proposals.     

Multiple-output quantile regression through optimal quantization

A new nonparametric quantile regression method based on the concept of optimal quantization was developed recently and was showed to provide estimators that often dominate their classical, kernel-type, competitors. In the present work, we extend this method to multiple-output regression problems. We show how quantization allows approximating population multiple-output regression quantiles based on halfspace depth. We prove that this approximation becomes arbitrarily accurate as the size of the quantization grid goes to infinity. We also derive a weak consistency result for a sample version of the proposed regression quantiles. Through simulations, we compare the performances of our estimators with (local constant and local bilinear) kernel competitors. The results reveal that the proposed quantization-based estimators, which are local constant in nature, outperform their kernel counterparts and even often dominate their local bilinear kernel competitors. The various approaches are also compared on artificial and real data.     

On sign-based regression quantiles

A sign-based (SB) approach suggests an alternative criterion for quantile regression fit. The SB criterion is a piecewise constant function, which often leads to a non-unique solution. We compare the mid-point of this SB solution with the least absolute deviations (LAD) method and describe asymptotic properties of SB estimators under a weaker set of assumptions as compared with the assumptions often used with the generalized method of moments. Asymptotic properties of LAD and SB estimators are equivalent; however, there are finite sample differences as we show in simulation studies. At small to moderate sample sizes, the SB procedure for modelling quantiles at longer tails demonstrates a substantially lower bias, variance, and mean-squared error when compared with the LAD. In the illustrative example, we model a 0.8-level quantile of hospital charges and highlight finite sample advantage of the SB versus LAD.     

On multivariate quantile regression

To detect the dependence on the covariates in the lower and upper tails of the response distribution, regression quantiles are very useful tools in linear model problems with univariate response. We consider here a notion of regression quantiles for problems with multivariate responses. The approach is based on minimizing a loss function equivalent to that in the case of univariate response. To construct an affine equivariant notion of multivariate regression quantiles, we have considered a transformation retransformation procedure based on ‘data-driven coordinate systems’. We indicate some algorithm to compute the proposed estimates and establish asymptotic normality for them. We also, suggest an adaptive procedure to select the optimal data-driven coordinate system. We discuss the performance of our estimates with the help of a finite sample simulation study and to illustrate our methodology, we analyzed an interesting data-set on blood pressures of a group of women and another one on the dependence of sales performances on creative test scores.     

Strictly monotone and smooth nonparametric regression for two or more variables

The authors propose a new monotone nonparametric estimate for a regression function of two or more variables. Their method consists in applying successively one‐dimensional isotonization procedures on an initial, unconstrained nonparametric regression estimate. In the case of a strictly monotone regression function, they show that the new estimate and the initial one are first‐order asymptotic equivalent; they also establish asymptotic normality of an appropriate standardization of the new estimate. In addition, they show that if the regression function is not monotone in one of its arguments, the new estimate and the initial one have approximately the same L^p‐norm. They illustrate their approach by means of a simulation study, and two data examples are analyzed.     

Estimation of the linear-plateau segmented regression model in the presence of measurement error

It is well known that when the true values of the independent variable are unobservable due to measurement error, the least squares estimator for a regression model is biased and inconsistent. When repeated observations on each x_i are taken, consistent estimators for the linear-plateau model can be formed. The repeated observations are required to classify each observation to the appropriate line segment. Two cases of repeated observations are treated in detail. First, when a single value of y_i is observed with the repeated observations of x_i the least squares estimator using the mean of the repeated x_i observations is consistent and asymptotically normal. Second, when repeated observations on the pair (x_i, y_i ) are taken the least squares estimator is inconsistent, but two consistent estimators are proposed: one that consistently estimates the bias of the least squares estimator and adjusts accordingly; the second is the least squares estimator using the mean of the repeated observations on each pair.     

Automatic and location-adaptive estimation in functional single-index regression

This paper develops a new automatic and location-adaptive procedure for estimating regression in a Functional Single-Index Model (FSIM). This procedure is based on k-Nearest Neighbours (kNN) ideas. The asymptotic study includes results for automatically data-driven selected number of neighbours, making the procedure directly usable in practice. The local feature of the kNN approach insures higher predictive power compared with usual kernel estimates, as illustrated in some finite sample analysis. As by-product, we state as preliminary tools some new uniform asymptotic results for kernel estimates in the FSIM model.     

Asymptotic distribution of data-driven smoothers in density and regression estimation under dependence

We consider automatic data-driven density, regression and autoregression estimates, based on any random bandwidth selector h/_T. We show that in a first-order asymptotic approximation they behave as well as the related estimates obtained with the “optimal” bandwidth h_T as long as h_T/h_T → 1 in probability. The results are obtained for dependent observations; some of them are also new for independent observations.     

The asymptotic behavior of monotone percentile regression estimates

The least-absolute-deviation estimate of a monotone regression function on an interval has been studied in the literature. If the observation points become dense in the interval, the almost sure rate of convergence has been shown to be O(n^1/4). Applying the techniques used by Brunk (1970, Nonparametric, Techniques in Statistical Inference. Cambridge Univ. Press), the asymptotic distribution of the l₁ estimator at a point is obtained. If the underlying regression function has positive slope at the point, the rate of convergence is seen to be O(n^1/3). Monotone percentile regression estimates are also considered.     

Penalized quantile regression for dynamic panel data

This paper studies penalized quantile regression for dynamic panel data with fixed effects, where the penalty involves l₁ shrinkage of the fixed effects. Using extensive Monte Carlo simulations, we present evidence that the penalty term reduces the dynamic panel bias and increases the efficiency of the estimators. The underlying intuition is that there is no need to use instrumental variables for the lagged dependent variable in the dynamic panel data model without fixed effects. This provides an additional use for the shrinkage models, other than model selection and efficiency gains. We propose a Bayesian information criterion based estimator for the parameter that controls the degree of shrinkage. We illustrate the usefulness of the novel econometric technique by estimating a “target leverage” model that includes a speed of capital structure adjustment. Using the proposed penalized quantile regression model the estimates of the adjustment speeds lie between 3% and 44% across the quantiles, showing strong evidence that there is substantial heterogeneity in the speed of adjustment among firms.     

Bayesian non-crossing quantile regression for regularly varying distributions

Quantile regression is a very important statistical tool for predictive modelling and risk assessment. For many applications, conditional quantile at different levels are estimated separately. Consequently the monotonicity of conditional quantiles can be violated when quantile regression curves cross each other. In this paper, we propose a new Bayesian multiple quantile regression based on heavy tailed distribution for non-crossing. We consider a linear quantile regression model for simultaneous Bayesian estimation of multiple quantiles based on a regularly varying assumptions. The numerical and competitive performance of the proposed method is illustrated by simulation.