Finite mixture of regression models for a stratified sample

Despite the popularity and importance, there is limited work on modelling data which come from complex survey design using finite mixture models. In this work, we explored the use of finite mixture regression models when the samples were drawn using a complex survey design. In particular, we considered modelling data collected based on stratified sampling design. We developed a new design-based inference where we integrated sampling weights in the complete-data log-likelihood function. The expectation–maximisation algorithm was developed accordingly. A simulation study was conducted to compare the new methodology with the usual finite mixture of a regression model. The comparison was done using bias-variance components of mean square error. Additionally, a simulation study was conducted to assess the ability of the Bayesian information criterion to select the optimal number of components under the proposed modelling approach. The methodology was implemented on real data with good results.     

Randomized quantile regression estimation for heteroskedastic non parametric model

In this paper, we propose robust randomized quantile regression estimators for the mean and (condition) variance functions of the popular heteroskedastic non parametric regression model. Unlike classical approaches which consider quantile as a fixed quantity, our method treats quantile as a uniformly distributed random variable. Our proposed method can be employed to estimate the error distribution, which could significantly improve prediction results. An automatic bandwidth selection scheme will be discussed. Asymptotic properties and relative efficiencies of the proposed estimators are investigated. Our empirical results show that the proposed estimators work well even for random errors with infinite variances. Various numerical simulations and two real data examples are used to demonstrate our methodologies.     

Mixtures of Linear Regression with Measurement Errors

Existing research on mixtures of regression models are limited to directly observed predictors. The estimation of mixtures of regression for measurement error data imposes challenges for statisticians. For linear regression models with measurement error data, the naive ordinary least squares method, which directly substitutes the observed surrogates for the unobserved error-prone variables, yields an inconsistent estimate for the regression coefficients. The same inconsistency also happens to the naive mixtures of regression estimate, which is based on the traditional maximum likelihood estimator and simply ignores the measurement error. To solve this inconsistency, we propose to use the deconvolution method to estimate the mixture likelihood of the observed surrogates. Then our proposed estimate is found by maximizing the estimated mixture likelihood. In addition, a generalized EM algorithm is also developed to find the estimate. The simulation results demonstrate that the proposed estimation procedures work well and perform much better than the naive estimates.     

Influence diagnostics for Student-t censored linear regression models

In this paper, we extend the censored linear regression model with normal errors to Student-t errors. A simple EM-type algorithm for iteratively computing maximum-likelihood estimates of the parameters is presented. To examine the performance of the proposed model, case-deletion and local influence techniques are developed to show its robust aspect against outlying and influential observations. This is done by the analysis of the sensitivity of the EM estimates under some usual perturbation schemes in the model or data and by inspecting some proposed diagnostic graphics. The efficacy of the method is verified through the analysis of simulated data sets and modelling a real data set first analysed under normal errors. The proposed algorithm and methods are implemented in the R package CensRegMod.     

Multiple-output quantile regression through optimal quantization

A new nonparametric quantile regression method based on the concept of optimal quantization was developed recently and was showed to provide estimators that often dominate their classical, kernel-type, competitors. In the present work, we extend this method to multiple-output regression problems. We show how quantization allows approximating population multiple-output regression quantiles based on halfspace depth. We prove that this approximation becomes arbitrarily accurate as the size of the quantization grid goes to infinity. We also derive a weak consistency result for a sample version of the proposed regression quantiles. Through simulations, we compare the performances of our estimators with (local constant and local bilinear) kernel competitors. The results reveal that the proposed quantization-based estimators, which are local constant in nature, outperform their kernel counterparts and even often dominate their local bilinear kernel competitors. The various approaches are also compared on artificial and real data.     

Variable screening for ultrahigh dimensional censored quantile regression

Quantile regression is a flexible approach to assessing covariate effects on failure time, which has attracted considerable interest in survival analysis. When the dimension of covariates is much larger than the sample size, feature screening and variable selection become extremely important and indispensable. In this article, we introduce a new feature screening method for ultrahigh dimensional censored quantile regression. The proposed method can work for a general class of survival models, allow for heterogeneity of data and enjoy desirable properties including the sure screening property and the ranking consistency property. Moreover, an iterative version of screening algorithm has also been proposed to accommodate more complex situations. Monte Carlo simulation studies are designed to evaluate the finite sample performance under different model settings. We also illustrate the proposed methods through an empirical analysis.     

Testing the equality of two independent regression models

In some situations, for example in agriculture, biology, hydrology, and psychology, researchers wish to determine whether the relationship between response variable and predictor variables differs in two populations. In other words, we are interested in comparing two regression models for two independent datasets. In this work, we will use the parametric and nonparametric methods to establish hypothesis testing for the equality of two independent regression models. Then the simulation study is provided to investigate the performance of the proposed method.     

Non-iterative Estimation and Variable Selection in the Single-index Quantile Regression Model

A new estimation procedure is proposed for the single-index quantile regression model. Compared to existing work, this approach is non-iterative and hence, computationally efficient. The proposed method not only estimates the index parameter and the link function but also selects variables simultaneously. The performance of the variable selection is enhanced by a fully adaptive penalty function motivated by the sliced inverse regression technique. Finite sample performance is studied through a simulation study that compares the proposed method with existing work under several criteria. A data analysis is given that highlights the usefulness of the proposed methodology.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

The single-index support vector regression model to address the problem of high dimensionality

ABSTRACT

The last few years, the applications of Support Vector Machine (SVM) for solving classification and regression problems have been increasing, due to its high performance and ability to transform the non-linear relationships among variables to linear form by employing the kernel idea (kernel function). In this work, we develop a semi-parametric approach to fit single-index models to deal with high-dimensional problems. To achieve this goal, we use support vector regression (SVR) for estimating the unknown nonparametric link function, while the single-index is determined by using the semi-parametric least squares method (Ichimura 1993). This development enhances the ability of SVR to solve high-dimensional problem. We design a three simulation examples with high-dimensional problems (linear and nonlinear). The simulations demonstrate the superior performance of the proposed method versus the standard SVR method. This is further illustrated by applying the real data.     

Pruning regression trees for censored survival data: the recpam approach

RECPAM is a methodology, implemented in a computer program of the same name, for the construction of tree-structured models in Biostatistics. In this work we present algorithms for pruning and amalgamating terminal nodes of a tree, within the RECPAM approach. These algorithms construct sequences of nested models and calculate at each step the AIC of the corresponding model and correct significance levels, according to Gabriel's theory of Simultaneous Test Procedures. As an example, the analysis of data from clinical trials involving patients with Small Cell Carcinoma of the Lung is presented.     

Bayesian bridge quantile regression

Regularization methods for simultaneous variable selection and coefficient estimation have been shown to be effective in quantile regression in improving the prediction accuracy. In this article, we propose the Bayesian bridge for variable selection and coefficient estimation in quantile regression. A simple and efficient Gibbs sampling algorithm was developed for posterior inference using a scale mixture of uniform representation of the Bayesian bridge prior. This is the first work to discuss regularized quantile regression with the bridge penalty. Both simulated and real data examples show that the proposed method often outperforms quantile regression without regularization, lasso quantile regression, and Bayesian lasso quantile regression.     

An improved meta-analysis for analyzing cylindrical-type time series data with applications to forecasting problem in environmental study

In this paper, we propose an improved generalized least square (GLS) meta-analysis in a linear-circular regression, and show its utility in the analysis of a certain environmental issue. The existing GLS meta-analysis proposed in Becker and Wu has a serious flaw since information about the covariance among coefficients across studies is not utilized. In our proposed meta-analysis, we take the correlations between adjacent studies into account, and improve the existing GLS meta-analysis. We provide numerical examples to compare the proposed method with several other existing methods by using Akaike's Information Criterion, Bayesian Information Criterion and mean square prediction errors with applications to forecasting problem in Environmental study.     

Guided Censored Regression

Parametrically guided non‐parametric regression is an appealing method that can reduce the bias of a non‐parametric regression function estimator without increasing the variance. In this paper, we adapt this method to the censored data case using an unbiased transformation of the data and a local linear fit. The asymptotic properties of the proposed estimator are established, and its performance is evaluated via finite sample simulations.     

Doubly robust weighted composite quantile regression based on SCAD-L2

In this article, a robust variable selection procedure based on the weighted composite quantile regression (WCQR) is proposed. Compared with the composite quantile regression (CQR), WCQR is robust to heavy-tailed errors and outliers in the explanatory variables. For the choice of the weights in the WCQR, we employ a weighting scheme based on the principal component method. To select variables with grouping effect, we consider WCQR with SCAD-L₂ penalization. Furthermore, under some suitable assumptions, the theoretical properties, including the consistency and oracle property of the estimator, are established with a diverging number of parameters. In addition, we study the numerical performance of the proposed method in the case of ultrahigh-dimensional data. Simulation studies and real examples are provided to demonstrate the superiority of our method over the CQR method when there are outliers in the explanatory variables and/or the random error is from a heavy-tailed distribution.     

Comparison of Nonparametric Regression Curves by Spline Smoothing

In this article, procedures are proposed to test the hypothesis of equality of two or more regression functions. Tests are proposed by p-values, first under homoscedastic regression model, which are derived using fiducial method based on cubic spline interpolation. Then, we construct a test in the heteroscedastic case based on Fisher's method of combining independent tests. We study the behaviors of the tests by simulation experiments, in which comparisons with other tests are also given. The proposed tests have good performances. Finally, an application to a data set are given to illustrate the usefulness of the proposed test in practice.     

Variance estimation in nonparametric regression with jump discontinuities

Variance estimation is an important topic in nonparametric regression. In this paper, we propose a pairwise regression method for estimating the residual variance. Specifically, we regress the squared difference between observations on the squared distance between design points, and then estimate the residual variance as the intercept. Unlike most existing difference-based estimators that require a smooth regression function, our method applies to regression models with jump discontinuities. Our method also applies to the situations where the design points are unequally spaced. Finally, we conduct extensive simulation studies to evaluate the finite-sample performance of the proposed method and compare it with some existing competitors.     

Fast Bayesian variable screenings for binary response regressions with small sample size

Screening procedures play an important role in data analysis, especially in high-throughput biological studies where the datasets consist of more covariates than independent subjects. In this article, a Bayesian screening procedure is introduced for the binary response models with logit and probit links. In contrast to many screening rules based on marginal information involving one or a few covariates, the proposed Bayesian procedure simultaneously models all covariates and uses closed-form screening statistics. Specifically, we use the posterior means of the regression coefficients as screening statistics; by imposing a generalized g-prior on the regression coefficients, we derive the analytical form of their posterior means and compute the screening statistics without Markov chain Monte Carlo implementation. We evaluate the utility of the proposed Bayesian screening method using simulations and real data analysis. When the sample size is small, the simulation results suggest improved performance with comparable computational cost.     

Focused Information Criterion and Model Averaging in Quantile Regression

In this article, we study model selection and model averaging in quantile regression. Under general conditions, we develop a focused information criterion and a frequentist model average estimator for the parameters in quantile regression model, and examine their theoretical properties. The new procedures provide a robust alternative to the least squares method or likelihood method, and a major advantage of the proposed procedures is that when the variance of random error is infinite, the proposed procedure works beautifully while the least squares method breaks down. A simulation study and a real data example are presented to show that the proposed method performs well with a finite sample and is easy to use in practice.     

Quantile regression and variable selection for the single-index model

In this paper, we propose a new full iteration estimation method for quantile regression (QR) of the single-index model (SIM). The asymptotic properties of the proposed estimator are derived. Furthermore, we propose a variable selection procedure for the QR of SIM by combining the estimation method with the adaptive LASSO penalized method to get sparse estimation of the index parameter. The oracle properties of the variable selection method are established. Simulations with various non-normal errors are conducted to demonstrate the finite sample performance of the estimation method and the variable selection procedure. Furthermore, we illustrate the proposed method by analyzing a real data set.