Wavelet kernel penalized estimation for non-equispaced design regression

The paper considers regression problems with univariate design points. The design points are irregular and no assumptions on their distribution are imposed. The regression function is retrieved by a wavelet based reproducing kernel Hilbert space (RKHS) technique with the penalty equal to the sum of blockwise RKHS norms. In order to simplify numerical optimization, the problem is replaced by an equivalent quadratic minimization problem with an additional penalty term. The computational algorithm is described in detail and is implemented with both the sets of simulated and real data. Comparison with existing methods showed that the technique suggested in the paper does not oversmooth the function and is superior in terms of the mean squared error. It is also demonstrated that under additional assumptions on design points the method achieves asymptotic optimality in a wide range of Besov spaces.     

Composite kernel quantile regression

The composite quantile regression (CQR) has been developed for the robust and efficient estimation of regression coefficients in a liner regression model. By employing the idea of the CQR, we propose a new regression method, called composite kernel quantile regression (CKQR), which uses the sum of multiple check functions as a loss in reproducing kernel Hilbert spaces for the robust estimation of a nonlinear regression function. The numerical results demonstrate the usefulness of the proposed CKQR in estimating both conditional nonlinear mean and quantile functions.     

Estimation and variable selection in single-index composite quantile regression

In this article, a new composite quantile regression estimation approach is proposed for estimating the parametric part of single-index model. We use local linear composite quantile regression (CQR) for estimating the nonparametric part of single-index model (SIM) when the error distribution is symmetrical. The weighted local linear CQR is proposed for estimating the nonparametric part of SIM when the error distribution is asymmetrical. Moreover, a new variable selection procedure is proposed for SIM. Under some regularity conditions, we establish the large sample properties of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the behavior of the proposed estimators.     

Hierarchically penalized quantile regression

In many regression problems, predictors are naturally grouped. For example, when a set of dummy variables is used to represent categorical variables, or a set of basis functions of continuous variables is included in the predictor set, it is important to carry out a feature selection both at the group level and at individual variable levels within the group simultaneously. To incorporate the group and variables within-group information into a regularized model fitting, several regularization methods have been developed, including the Cox regression and the conditional mean regression. Complementary to earlier works, the simultaneous group and within-group variables selection method is examined in quantile regression. We propose a hierarchically penalized quantile regression, and show that the hierarchical penalty possesses the oracle property in quantile regression, as well as in the Cox regression. The proposed method is evaluated through simulation studies and a real data application.     

On multivariate quantile regression

To detect the dependence on the covariates in the lower and upper tails of the response distribution, regression quantiles are very useful tools in linear model problems with univariate response. We consider here a notion of regression quantiles for problems with multivariate responses. The approach is based on minimizing a loss function equivalent to that in the case of univariate response. To construct an affine equivariant notion of multivariate regression quantiles, we have considered a transformation retransformation procedure based on ‘data-driven coordinate systems’. We indicate some algorithm to compute the proposed estimates and establish asymptotic normality for them. We also, suggest an adaptive procedure to select the optimal data-driven coordinate system. We discuss the performance of our estimates with the help of a finite sample simulation study and to illustrate our methodology, we analyzed an interesting data-set on blood pressures of a group of women and another one on the dependence of sales performances on creative test scores.     

Local linear kernel estimation for discontinuous nonparametric regression functions

In this paper, we consider using a local linear (LL) smoothing method to estimate a class of discontinuous regression functions. We establish the asymptotic normality of the integrated square error (ISE) of a LL-type estimator and show that the ISE has an asymptotic rate of convergence as good as for smooth functions, and the asymptotic rate of convergence of the ISE of the LL estimator is better than that of the Nadaraya-Watson (NW) and the Gasser-Miiller (GM) estimators.     

Diagnostic measures for kernel ridge regression on reproducing kernel Hilbert space

The aim of this paper is to define and develop diagnostic measures with respect to kernel ridge regression in a reproducing kernel Hilbert space (RKHS). To identify influential observations, we define a particular version of Cook’s distance for the kernel ridge regression model in RKHS, which is conceptually consistent with Cook’s distance in a classical regression model. Then, by using the perturbation formula for the regularized conditional expectation of the outcome in RKHS, we develop an approximate version of Cook”s distance in RKHS because the original definition requires intensive computations. Such an approximated Cook”s distance is represented in terms of basic building blocks such as residuals and leverages of the kernel ridge regression. The results of the simulation and real application demonstrate that our diagnostic measure successfully detects potentially influential observations on estimators in kernel ridge regression.     

The kernel regression estimation for randomly censored functional stationary ergodic data

In this paper, we investigate the asymptotic properties of the kernel estimator for non parametric regression operator when the functional stationary ergodic data with randomly censorship are considered. More precisely, we introduce the kernel-type estimator of the non parametric regression operator with the responses randomly censored and obtain the almost surely convergence with rate as well as the asymptotic normality of the estimator. As an application, the asymptotic (1 ? ζ) confidence interval of the regression operator is also presented (0 < ζ < 1). Finally, the simulation study is carried out to show the finite-sample performances of the estimator.     

Partially linear censored quantile regression

Censored regression quantile (CRQ) methods provide a powerful and flexible approach to the analysis of censored survival data when standard linear models are felt to be appropriate. In many cases however, greater flexibility is desired to go beyond the usual multiple regression paradigm. One area of common interest is that of partially linear models: one (or more) of the explanatory covariates are assumed to act on the response through a non-linear function. Here the CRQ approach of Portnoy (J Am Stat Assoc 98:1001–1012, 2003) is extended to this partially linear setting. Basic consistency results are presented. A simulation experiment and unemployment example justify the value of the partially linear approach over methods based on the Cox proportional hazards model and on methods not permitting nonlinearity.     

Modelling functional additive quantile regression using support vector machines approach

This work deals with conditional quantiles estimation when several functional covariates are involved, via a support vector machines nonparametric methodology. We establish weak consistency of this estimator. To fit the additive components, we use an ordinary backfitting procedure combined with an iterative reweighted least-squares procedure to solve the penalised minimisation problem. This procedure makes it possible to derive a split sample method for choosing the hyper-parameters of the model. The performances of the proposed technique, in terms of forecast accuracy, are evaluated through simulation and a real dataset study.     

Online learning for quantile regression and support vector regression

We consider for quantile regression and support vector regression a kernel-based online learning algorithm associated with a sequence of insensitive pinball loss functions. Our error analysis and derived learning rates show quantitatively that the statistical performance of the learning algorithm may vary with the quantile parameter τ$\tau$. In our analysis we overcome the technical difficulty caused by the varying insensitive parameter introduced with a motivation of sparsity.     

Using local linear kernel smoothers to test the lack of fit of nonlinear regression models

Herein, we propose a data-driven test that assesses the lack of fit of nonlinear regression models. The comparison of local linear kernel and parametric fits is the basis of this test, and specific boundary-corrected kernels are not needed at the boundary when local linear fitting is used. Under the parametric null model, the asymptotically optimal bandwidth can be used for bandwidth selection. This selection method leads to the data-driven test that has a limiting normal distribution under the null hypothesis and is consistent against any fixed alternative. The finite-sample property of the proposed data-driven test is illustrated, and the power of the test is compared with that of some existing tests via simulation studies. We illustrate the practicality of the proposed test by using two data sets.     

Binary quantile regression and variable selection: A new approach

In this paper, we propose a new estimation method for binary quantile regression and variable selection which can be implemented by an iteratively reweighted least square approach. In contrast to existing approaches, this method is computationally simple, guaranteed to converge to a unique solution and implemented with standard software packages. We demonstrate our methods using Monte-Carlo experiments and then we apply the proposed method to the widely used work trip mode choice dataset. The results indicate that the proposed estimators work well in finite samples.     

Functional logistic regression with fused lasso penalty

This study considers the binary classification of functional data collected in the form of curves. In particular, we assume a situation in which the curves are highly mixed over the entire domain, so that the global discriminant analysis based on the entire domain is not effective. This study proposes an interval-based classification method for functional data: the informative intervals for classification are selected and used for separating the curves into two classes. The proposed method, called functional logistic regression with fused lasso penalty, combines the functional logistic regression as a classifier and the fused lasso for selecting discriminant segments. The proposed method automatically selects the most informative segments of functional data for classification by employing the fused lasso penalty and simultaneously classifies the data based on the selected segments using the functional logistic regression. The effectiveness of the proposed method is demonstrated with simulated and real data examples.     

A weighted linear quantile regression

In this article, we introduce a new weighted quantile regression method. Traditionally, the estimation of the parameters involved in quantile regression is obtained by minimizing a loss function based on absolute distances with weights independent of explanatory variables. Specifically, we study a new estimation method using a weighted loss function with the weights associated with explanatory variables so that the performance of the resulting estimation can be improved. In full generality, we derive the asymptotic distribution of the weighted quantile regression estimators for any uniformly bounded positive weight function independent of the response. Two practical weighting schemes are proposed, each for a certain type of data. Monte Carlo simulations are carried out for comparing our proposed methods with the classical approaches. We also demonstrate the proposed methods using two real-life data sets from the literature. Both our simulation study and the results from these examples show that our proposed method outperforms the classical approaches when the relative efficiency is measured by the mean-squared errors of the estimators.     

Asymptotic results in gamma kernel regression

Abstract

Based on the Gamma kernel density estimation procedure, this article constructs a nonparametric kernel estimate for the regression functions when the covariate are nonnegative. Asymptotic normality and uniform almost sure convergence results for the new estimator are systematically studied, and the finite performance of the proposed estimate is discussed via a simulation study and a comparison study with an existing method. Finally, the proposed estimation procedure is applied to the Geyser data set.     

Wavelet-based LASSO in functional linear quantile regression

ABSTRACT

In this paper, we develop an efficient wavelet-based regularized linear quantile regression framework for coefficient estimations, where the responses are scalars and the predictors include both scalars and function. The framework consists of two important parts: wavelet transformation and regularized linear quantile regression. Wavelet transform can be used to approximate functional data through representing it by finite wavelet coefficients and effectively capturing its local features. Quantile regression is robust for response outliers and heavy-tailed errors. In addition, comparing with other methods it provides a more complete picture of how responses change conditional on covariates. Meanwhile, regularization can remove small wavelet coefficients to achieve sparsity and efficiency. A novel algorithm, Alternating Direction Method of Multipliers (ADMM) is derived to solve the optimization problems. We conduct numerical studies to investigate the finite sample performance of our method and applied it on real data from ADHD studies.     

Bayesian quantile regression for hierarchical linear models

The paper proposes a Bayesian quantile regression method for hierarchical linear models. Existing approaches of hierarchical linear quantile regression models are scarce and most of them were not from the perspective of Bayesian thoughts, which is important for hierarchical models. In this paper, based on Bayesian theories and Markov Chain Monte Carlo methods, we introduce Asymmetric Laplace distributed errors to simulate joint posterior distributions of population parameters and across-unit parameters and then derive their posterior quantile inferences. We run a simulation as the proposed method to examine the effects on parameters induced by units and quantile levels; the method is also applied to study the relationship between Chinese rural residents' family annual income and their cultivated areas. Both the simulation and real data analysis indicate that the method is effective and accurate.     

Consistency of the recursive nonparametric regression estimation for dependent functional data

We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition. Finally, the feasibility of the proposed estimator is illustrated throughout an empirical study.     

Bayesian quantile regression for skew-normal linear mixed models

Linear mixed models have been widely used to analyze repeated measures data which arise in many studies. In most applications, it is assumed that both the random effects and the within-subjects errors are normally distributed. This can be extremely restrictive, obscuring important features of within-and among-subject variations. Here, quantile regression in the Bayesian framework for the linear mixed models is described to carry out the robust inferences. We also relax the normality assumption for the random effects by using a multivariate skew-normal distribution, which includes the normal ones as a special case and provides robust estimation in the linear mixed models. For posterior inference, we propose a Gibbs sampling algorithm based on a mixture representation of the asymmetric Laplace distribution and multivariate skew-normal distribution. The procedures are demonstrated by both simulated and real data examples.