Nonstationary nonlinear quantile regression

This study examines estimation and inference based on quantile regression for parametric nonlinear models with an integrated time series covariate. We first derive the limiting distribution of the nonlinear quantile regression estimator and then consider testing for parameter restrictions, when the regression function is specified as an asymptotically homogeneous function. We also study linear-in-parameter regression models when the regression function is given by integrable regression functions as well as asymptotically homogeneous regression functions. We, furthermore, propose a fully modified estimator to reduce the bias in the original estimator under a certain set of conditions. Finally, simulation studies show that the estimators behave well, especially when the regression error term has a fat-tailed distribution.     

Semi-non parametric smooth isotonic regression

We propose a new method for smooth isotonic regression analysis. Unlike most existing methods for isotonic regression, the proposed method is akin to parametric regression without order restriction. To account for smoothness and isotonicity simultaneously, we exploit the flexible class of semi-non parametric densities to model isotonic regression functions. Under this framework, the full range of inference techniques for parametric regression models become applicable for model estimation and model validation in isotonic regression.     

Regression Kink With an Unknown Threshold

This article explores estimation and inference in a regression kink model with an unknown threshold. A regression kink model (or continuous threshold model) is a threshold regression constrained to be everywhere continuous with a kink at an unknown threshold. We present methods for estimation, to test for the presence of the threshold, for inference on the regression parameters, and for inference on the regression function. A novel finding is that inference on the regression function is nonstandard since the regression function is a nondifferentiable function of the parameters. We apply recently developed methods for inference on nondifferentiable functions. The theory is illustrated by an application to the growth and debt problem introduced by Reinhart and Rogoff, using their long-span time-series for the United States.     

Composite kernel quantile regression

The composite quantile regression (CQR) has been developed for the robust and efficient estimation of regression coefficients in a liner regression model. By employing the idea of the CQR, we propose a new regression method, called composite kernel quantile regression (CKQR), which uses the sum of multiple check functions as a loss in reproducing kernel Hilbert spaces for the robust estimation of a nonlinear regression function. The numerical results demonstrate the usefulness of the proposed CKQR in estimating both conditional nonlinear mean and quantile functions.     

缺失偏态数据下线性回归模型的统计推断

研究缺失偏态数据下线性回归模型的参数估计问题,针对缺失偏态数据,为克服样本分布扭曲缺点和提高模型的回归系数、尺度参数和偏度参数的估计效果,提出了一种适合偏态数据下线性回归模型中缺失数据的修正回归插补方法.通过随机模拟和实例研究,并与均值插补、回归插补、随机回归插补方法比较,结果表明所提出的修正回归插补方法是有效可行的.     

Ridge Regression Estimation for Survey Samples

This paper describes procedure for constructing a vector of regression weights. Under the regression superpopulation model, the ridge regression estimator that has minimum model mean squared error is derived. Through a simulation study, we compare the ridge regression weights, regression weights, quadratic programming weights, and raking ratio weights. The ridge regression procedure with weights bounded by zero performed very well.     

A hierarchical Bayesian regression model for the uncertain functional constraint using screened scale mixtures of Gaussian distributions

This paper considers a hierarchical Bayesian analysis of regression models using a class of Gaussian scale mixtures. This class provides a robust alternative to the common use of the Gaussian distribution as a prior distribution in particular for estimating the regression function subject to uncertainty about the constraint. For this purpose, we use a family of rectangular screened multivariate scale mixtures of Gaussian distribution as a prior for the regression function, which is flexible enough to reflect the degrees of uncertainty about the functional constraint. Specifically, we propose a hierarchical Bayesian regression model for the constrained regression function with uncertainty on the basis of three stages of a prior hierarchy with Gaussian scale mixtures, referred to as a hierarchical screened scale mixture of Gaussian regression models (HSMGRM). We describe distributional properties of HSMGRM and an efficient Markov chain Monte Carlo algorithm for posterior inference, and apply the proposed model to real applications with constrained regression models subject to uncertainty.     

Theory & Methods: Krige,Smooth, Both or Neither?

Both kriging and non-parametric regression smoothing can model a non-stationary regression function with spatially correlated errors. However comparisons have mainly been based on ordinary kriging and smoothing with uncorrelated errors. Ordinary kriging attributes smoothness of the response to spatial autocorrelation whereas non-parametric regression attributes trends to a smooth regression function. For spatial processes it is reasonable to suppose that the response is due to both trend and autocorrelation. This paper reviews methodology for non-parametric regression with autocorrelated errors which is a natural compromise between the two methods. Re-analysis of the one-dimensional stationary spatial data of Laslett (1994) and a clearly non-stationary time series demonstrates the rather surprising result that for these data, ordinary kriging outperforms more computationally intensive models including both universal kriging and correlated splines for spatial prediction. For estimating the regression function, non-parametric regression provides adaptive estimation, but the autocorrelation must be accounted for in selecting the smoothing parameter.     

Regression Depth with Censored and Truncated Data

Abstract

In this article, we consider the problem of estimating regression coefficients for a linear model with censored and truncated data based on regression depth. Any line can be given a rank using regression depth and the deepest regression line is the line with the maximum regression depth. We propose a method to define the regression depth of a line in the presence of censoring and truncation. We show how the proposed regression performs through analyzing Stanford heart transplant data and AIDS incubation data.     

Comment on hoerl and kennard's ridge regression simulation methodology

Ridge regression has received strong support in several Monte carlo studies, leading some authors to advocate its general use. It is argued, however, that these studies were strongly biased in favor of ridge regression by simulating regression coefficient vectors centered at the origin; a condition well suited for a shrinkage technique. Studies which modeled some non-zero regression coefficients and which showed only qualified support for ridge regression are cited in support of this argument. It is concluded that only to the extent that ridge regression type coefficient vectors actually underlie real data sets -a poorly understood phenomenon - will ridge regression be of use.     

Penalized angular regression for personalized predictions

Personalization is becoming an important aspect of many predictive applications. We introduce a penalized regression method which inherently implements personalization. Personalized angle (PAN) regression constructs regression coefficients that are specific to the covariate vector for which one is producing a prediction, thus personalizing the regression model itself. This is achieved by penalizing the normalized prediction for a given covariate vector. The method therefore penalizes the normalized regression coefficients, or the angles of the regression coefficients in a hyperspherical parametrization, introducing a new angle-based class of penalties. PAN hence combines two novel concepts: penalizing the normalized coefficients and personalization. For an orthogonal design matrix, we show that the PAN estimator is the solution to a low-dimensional eigenvector equation. Based on the hyperspherical parametrization, we construct an efficient algorithm to calculate the PAN estimator. We propose a parametric bootstrap procedure for selecting the tuning parameter, and simulations show that PAN regression can outperform ordinary least squares, ridge regression and other penalized regression methods in terms of prediction error. Finally, we demonstrate the method in a medical application.     

Confidence intervals for the regression coefficient in a simple regression model with a balanced two-fold nested error structure

ABSTRACT

In applications using a simple regression model with a balanced two-fold nested error structure, interest focuses on inferences concerning the regression coefficient. This article derives exact and approximate confidence intervals on the regression coefficient in the simple regression model with a balanced two-fold nested error structure. Eleven methods are considered for constructing the confidence intervals on the regression coefficient. Computer simulation is performed to compare the proposed confidence intervals. Recommendations are suggested for selecting an appropriate method.     

Diagnostics for non-linear regression

Sensitivity analysis in regression is concerned with assessing the sensitivity of the results of a regression model (e.g., the objective function, the regression parameters, and the fitted values) to changes in the data. Sensitivity analysis in least squares linear regression has seen a great surge of research activities over the last three decades. By contrast, sensitivity analysis in non-linear regression has received very little attention. This paper deals with the problem of local sensitivity analysis in non-linear regression. Closed-form general formulas are provided for the sensitivities of three standard methods for the estimation of the parameters of a non-linear regression model based on a set of data. These methods are the least squares, the minimax, and the least absolute value methods. The effectiveness of the proposed measures is illustrated by application to several non-linear models including the ultrasonic data and the onion yield data. The proposed sensitivity measures are shown to deal effectively with the detection of influential observations in non-linear regression models.     

Local CQR Smoothing: An Efficient and Safe Alternative to Local Polynomial Regression

Summary.  Local polynomial regression is a useful non-parametric regression tool to explore fine data structures and has been widely used in practice. We propose a new non-parametric regression technique called local composite quantile regression smoothing to improve local polynomial regression further. Sampling properties of the estimation procedure proposed are studied. We derive the asymptotic bias, variance and normality of the estimate proposed. The asymptotic relative efficiency of the estimate with respect to local polynomial regression is investigated. It is shown that the estimate can be much more efficient than the local polynomial regression estimate for various non-normal errors, while being almost as efficient as the local polynomial regression estimate for normal errors. Simulation is conducted to examine the performance of the estimates proposed. The simulation results are consistent with our theoretical findings. A real data example is used to illustrate the method proposed.     

Conditional logistic regression in cluster-specific 1: m matched treatment–control designs

Conditional logistic regression is a popular method for estimating a treatment effect while eliminating cluster-specific nuisance parameters when they are not of interest. Under a cluster-specific 1: m matched treatment–control study design, we present a new closed-form relationship between the conditional logistic regression estimator and the ordinary logistic regression estimator. In addition, we prove an equivalence between the ordinary logistic regression and the conditional logistic regression estimators, when the clusters are replicated infinitely often, which indicates that potential bias concerns when applying conditional logistic regression to complex survey samples.     

Proportional hazards and threshold regression: their theoretical and practical connections

Proportional hazards (PH) regression is a standard methodology for analyzing survival and time-to-event data. The proportional hazards assumption of PH regression, however, is not always appropriate. In addition, PH regression focuses mainly on hazard ratios and thus does not offer many insights into underlying determinants of survival. These limitations have led statistical researchers to explore alternative methodologies. Threshold regression (TR) is one of these alternative methodologies (see Lee and Whitmore, Stat Sci 21:501–513, 2006, for a review). The connection between PH regression and TR has been examined in previous published work but the investigations have been limited in scope. In this article, we study the connections between these two regression methodologies in greater depth and show that PH regression is, for most purposes, a special case of TR. We show two methods of construction by which TR models can yield PH functions for survival times, one based on altering the TR time scale and the other based on varying the TR boundary. We discuss how to estimate the TR time scale and boundary, with or without the PH assumption. A case demonstration is used to highlight the greater understanding of scientific foundations that TR can offer in comparison to PH regression. Finally, we discuss the potential benefits of positioning PH regression within the first-hitting-time context of TR regression.     

Bayesian bridge quantile regression

Regularization methods for simultaneous variable selection and coefficient estimation have been shown to be effective in quantile regression in improving the prediction accuracy. In this article, we propose the Bayesian bridge for variable selection and coefficient estimation in quantile regression. A simple and efficient Gibbs sampling algorithm was developed for posterior inference using a scale mixture of uniform representation of the Bayesian bridge prior. This is the first work to discuss regularized quantile regression with the bridge penalty. Both simulated and real data examples show that the proposed method often outperforms quantile regression without regularization, lasso quantile regression, and Bayesian lasso quantile regression.     

Outlier detection in high-dimensional regression model

An outlier is defined as an observation that is significantly different from the others in its dataset. In high-dimensional regression analysis, datasets often contain a portion of outliers. It is important to identify and eliminate the outliers for fitting a model to a dataset. In this paper, a novel outlier detection method is proposed for high-dimensional regression problems. The leave-one-out idea is utilized to construct a novel outlier detection measure based on distance correlation, and then an outlier detection procedure is proposed. The proposed method enjoys several advantages. First, the outlier detection measure can be simply calculated, and the detection procedure works efficiently even for high-dimensional regression data. Moreover, it can deal with a general regression, which does not require specification of a linear regression model. Finally, simulation studies show that the proposed method behaves well for detecting outliers in high-dimensional regression model and performs better than some other competing methods.     

On confidence intervals for semiparametric expectile regression

In regression scenarios there is a growing demand for information on the conditional distribution of the response beyond the mean. In this scenario quantile regression is an established method of tail analysis. It is well understood in terms of asymptotic properties and estimation quality. Another way to look at the tail of a distribution is via expectiles. They provide a valuable alternative since they come with a combination of preferable attributes. The easy weighted least squares estimation of expectiles and the quadratic penalties often used in flexible regression models are natural partners. Also, in a similar way as quantiles can be seen as a generalisation of median regression, expectiles offer a generalisation of mean regression. In addition to regression estimates, confidence intervals are essential for interpretational purposes and to assess the variability of the estimate, but there is a lack of knowledge regarding the asymptotic properties of a semiparametric expectile regression estimate. Therefore confidence intervals for expectiles based on an asymptotic normal distribution are introduced. Their properties are investigated by a simulation study and compared to a boostrap-based gold standard method. Finally the introduced confidence intervals help to evaluate a geoadditive expectile regression model on childhood malnutrition data from India.     

TREE-BASED REGRESSION FOR A CIRCULAR RESPONSE

ABSTRACT

The application of conventional statistical methods to directional data generally produces erroneous results. Various regression models for a circular response have been presented in the literature, however these are unsatisfactory either in the limited relationships that can be modeled, or the limitations on the number or type of covariates admissible. One difficulty with circular regression is devising a meaningful regression function. This problem is exacerbated when trying to incorporate both linear and circular variables as covariates. Due to these complexities, circular regression is ripe for exploration via tree-based methods, in which a formal regression function is not needed, but where insight into the general structure and relationship between predictors and the response may be obtained. A basic framework for regression trees, predicting a circular response from a combination of circular and linear predictors, will be presented.