Local CQR Smoothing: An Efficient and Safe Alternative to Local Polynomial Regression

Summary.  Local polynomial regression is a useful non-parametric regression tool to explore fine data structures and has been widely used in practice. We propose a new non-parametric regression technique called local composite quantile regression smoothing to improve local polynomial regression further. Sampling properties of the estimation procedure proposed are studied. We derive the asymptotic bias, variance and normality of the estimate proposed. The asymptotic relative efficiency of the estimate with respect to local polynomial regression is investigated. It is shown that the estimate can be much more efficient than the local polynomial regression estimate for various non-normal errors, while being almost as efficient as the local polynomial regression estimate for normal errors. Simulation is conducted to examine the performance of the estimates proposed. The simulation results are consistent with our theoretical findings. A real data example is used to illustrate the method proposed.     

Parametrically Guided Non-parametric Regression

We present a new approach to regression function estimation in which a non-parametric regression estimator is guided by a parametric pilot estimate with the aim of reducing the bias. New classes of parametrically guided kernel weighted local polynomial estimators are introduced and formulae for asymptotic expectation and variance, hence approximated mean squared error and mean integrated squared error, are derived. It is shown that the new classes of estimators have the very same large sample variance as the estimators in the standard non-parametric setting, while there is substantial room for reducing the bias if the chosen parametric pilot function belongs to a wide neighbourhood around the true regression line. Bias reduction is discussed in light of examples and simulations.     

Robust fitting of hidden Markov regression models under a longitudinal setting

We propose a robust estimation procedure for the analysis of longitudinal data including a hidden process to account for unobserved heterogeneity between subjects in a dynamic fashion. We show how to perform estimation by an expectation–maximization-type algorithm in the hidden Markov regression literature. We show that the proposed robust approaches work comparably to the maximum-likelihood estimator when there are no outliers and the error is normal and outperform it when there are outliers or the error is heavy tailed. A real data application is used to illustrate our proposal. We also provide details on a simple criterion to choose the number of hidden states.     

A new information criterion-based bandwidth selection method for non-parametric regressions

ABSTRACT

Local linear estimator is a popularly used method to estimate the non-parametric regression functions, and many methods have been derived to estimate the smoothing parameter, or the bandwidth in this case. In this article, we propose an information criterion-based bandwidth selection method, with the degrees of freedom originally derived for non-parametric inferences. Unlike the plug-in method, the new method does not require preliminary parameters to be chosen in advance, and is computationally efficient compared to the cross-validation (CV) method. Numerical study shows that the new method performs better or comparable to existing plug-in method or CV method in terms of the estimation of the mean functions, and has lower variability than CV selectors. Real data applications are also provided to illustrate the effectiveness of the new method.     

Testing discontinuities in nonparametric regression

In nonparametric regression, it is often needed to detect whether there are jump discontinuities in the mean function. In this paper, we revisit the difference-based method in [13] and propose to further improve it. To achieve the goal, we first reveal that their method is less efficient due to the inappropriate choice of the response variable in their linear regression model. We then propose a new regression model for estimating the residual variance and the total amount of discontinuities simultaneously. In both theory and simulation, we show that the proposed variance estimator has a smaller mean-squared error compared to the existing estimator, whereas the estimation efficiency for the total amount of discontinuities remains unchanged. Finally, we construct a new test procedure for detection of discontinuities using the proposed method; and via simulation studies, we demonstrate that our new test procedure outperforms the existing one in most settings.     

Quasi-Likelihood Regression with Multiple Indices and Smooth Link and Variance Functions

Abstract.  A flexible semi-parametric regression model is proposed for modelling the relationship between a response and multivariate predictor variables. The proposed multiple-index model includes smooth unknown link and variance functions that are estimated non-parametrically. Data-adaptive methods for automatic smoothing parameter selection and for the choice of the number of indices M are considered. This model adapts to complex data structures and provides efficient adaptive estimation through the variance function component in the sense that the asymptotic distribution is the same as if the non-parametric components are known. We develop iterative estimation schemes, which include a constrained projection method for the case where the regression parameter vectors are mutually orthogonal. The proposed methods are illustrated with the analysis of data from a growth bioassay and a reproduction experiment with medflies. Asymptotic properties of the estimated model components are also obtained.     

Composite kernel quantile regression

The composite quantile regression (CQR) has been developed for the robust and efficient estimation of regression coefficients in a liner regression model. By employing the idea of the CQR, we propose a new regression method, called composite kernel quantile regression (CKQR), which uses the sum of multiple check functions as a loss in reproducing kernel Hilbert spaces for the robust estimation of a nonlinear regression function. The numerical results demonstrate the usefulness of the proposed CKQR in estimating both conditional nonlinear mean and quantile functions.     

Model misspecification in parametric dual modeling

In typical normal theory regression, the assumption of homogeneity of variances is often not appropriate. Instead of treating the variances as a nuisance and transforming away the heterogeneity, the structure of the variances may be of interest and it is desirable to model the variances. Simultaneous modeling of the mean and variance of a response is known as dual modeling. When parametric models for the mean and variance are prescribed, estimation of the mean and variance parameters are interrelated. One commonly used dual model assumes a linear model for the mean and a log-linear variance model (Aitkin, 1987). This paper considers the impact of model misspecification (mean and variance) on the dual model estimation procedure. Asymptotic expressions for the mean and variance estimates, graphical illustrations of the impact of model misspecification, and simulation results are presented.     

Efficient Robust Estimation for Linear Models with Missing Response at Random

Coefficient estimation in linear regression models with missing data is routinely carried out in the mean regression framework. However, the mean regression theory breaks down if the error variance is infinite. In addition, correct specification of the likelihood function for existing imputation approach is often challenging in practice, especially for skewed data. In this paper, we develop a novel composite quantile regression and a weighted quantile average estimation procedure for parameter estimation in linear regression models when some responses are missing at random. Instead of imputing the missing response by randomly drawing from its conditional distribution, we propose to impute both missing and observed responses by their estimated conditional quantiles given the observed data and to use the parametrically estimated propensity scores to weigh check functions that define a regression parameter. Both estimation procedures are resistant to heavy‐tailed errors or outliers in the response and can achieve nice robustness and efficiency. Moreover, we propose adaptive penalization methods to simultaneously select significant variables and estimate unknown parameters. Asymptotic properties of the proposed estimators are carefully investigated. An efficient algorithm is developed for fast implementation of the proposed methodologies. We also discuss a model selection criterion, which is based on an IC_Q ‐type statistic, to select the penalty parameters. The performance of the proposed methods is illustrated via simulated and real data sets.     

Bayesian composite quantile regression for linear mixed-effects models

Longitudinal data are commonly modeled with the normal mixed-effects models. Most modeling methods are based on traditional mean regression, which results in non robust estimation when suffering extreme values or outliers. Median regression is also not a best choice to estimation especially for non normal errors. Compared to conventional modeling methods, composite quantile regression can provide robust estimation results even for non normal errors. In this paper, based on a so-called pseudo composite asymmetric Laplace distribution (PCALD), we develop a Bayesian treatment to composite quantile regression for mixed-effects models. Furthermore, with the location-scale mixture representation of the PCALD, we establish a Bayesian hierarchical model and achieve the posterior inference of all unknown parameters and latent variables using Markov Chain Monte Carlo (MCMC) method. Finally, this newly developed procedure is illustrated by some Monte Carlo simulations and a case analysis of HIV/AIDS clinical data set.     

A semi-parametric approach to robust parameter design

Parameter design or robust parameter design (RPD) is an engineering methodology intended as a cost-effective approach for improving the quality of products and processes. The goal of parameter design is to choose the levels of the control variables that optimize a defined quality characteristic. An essential component of RPD involves the assumption of well estimated models for the process mean and variance. Traditionally, the modeling of the mean and variance has been done parametrically. It is often the case, particularly when modeling the variance, that nonparametric techniques are more appropriate due to the nature of the curvature in the underlying function. Most response surface experiments involve sparse data. In sparse data situations with unusual curvature in the underlying function, nonparametric techniques often result in estimates with problematic variation whereas their parametric counterparts may result in estimates with problematic bias. We propose the use of semi-parametric modeling within the robust design setting, combining parametric and nonparametric functions to improve the quality of both mean and variance model estimation. The proposed method will be illustrated with an example and simulations.     

Variance Estimation in Heteroscedastic Models by Undecimated Haar Transform

We propose a method in order to maximize the accuracy in the estimation of piecewise constant and piecewise smooth variance functions in a nonparametric heteroscedastic fixed design regression model. The difference-based initial estimates are obtained from the given observations. Then an estimator is constructed by using iterative regularization method with the analysis-prior undecimated three-level Haar transform as regularizer term. We notice that this method shows better results in the mean square sense over an existing adaptive estimation procedure considering all the standard test functions used in addition to the functions that we target. Some simulations and comparisons with other methods are conducted to assess the performance of the proposed method.     

A semiparametric approach to hidden Markov models under longitudinal observations

We propose a hidden Markov model for longitudinal count data where sources of unobserved heterogeneity arise, making data overdispersed. The observed process, conditionally on the hidden states, is assumed to follow an inhomogeneous Poisson kernel, where the unobserved heterogeneity is modeled in a generalized linear model (GLM) framework by adding individual-specific random effects in the link function. Due to the complexity of the likelihood within the GLM framework, model parameters may be estimated by numerical maximization of the log-likelihood function or by simulation methods; we propose a more flexible approach based on the Expectation Maximization (EM) algorithm. Parameter estimation is carried out using a non-parametric maximum likelihood (NPML) approach in a finite mixture context. Simulation results and two empirical examples are provided.     

Comparison of Separable Components in Different Samples

Abstract.  Imagine we have two different samples and are interested in doing semi- or non-parametric regression analysis in each of them, possibly on the same model. In this paper, we consider the problem of testing whether a specific covariate has different impacts on the regression curve in these two samples. We compare the regression curves of different samples but are interested in specific differences instead of testing for equality of the whole regression function. Our procedure does allow for random designs, different sample sizes, different variance functions, different sets of regressors with different impact functions, etc. As we use the marginal integration approach, this method can be applied to any strong, weak or latent separable model as well as to additive interaction models to compare the lower dimensional separable components between the different samples. Thus, in the case of having separable models, our procedure includes the possibility of comparing the whole regression curves, thereby avoiding the curse of dimensionality. It is shown that bootstrap fails in theory and practice. Therefore, we propose a subsampling procedure with automatic choice of subsample size. We present a complete asymptotic theory and an extensive simulation study.     

Local Linear Kernel Regression with Long-Range Dependent Errors

This paper considers the use of a local linear kernel regression method to test whether the mean function of a sequence of long-range dependent processes has discontinuities or change-points. It proposes a non-parametric estimation procedure and then establishes an asymptotic theory for the estimation procedure. Examples, simulated and real, illustrate the estimation procedure.     

Non-parametric Analysis of Covariance – The Case of Inhomogeneous and Heteroscedastic Noise

Abstract.  The purpose of this paper was to propose a procedure for testing the equality of several regression curves f _i in non-parametric regression models when the noise is inhomogeneous and heteroscedastic, i.e. when the variances depend on the regressor and may vary between groups. The presented approach is very natural because it transfers the maximum likelihood statistic from a heteroscedastic one-way analysis of variance to the context of non-parametric regression. The maximum likelihood estimators will be replaced by kernel estimators of the regression functions f _i . It is shown that the asymptotic distribution of the obtained test-statistic is nuisance parameter free. Asymptotic efficiency is compared with a test of Dette & Neumeyer [Annals of Statistics (2001) Vol. 29, 1361–1400] and it is shown that the new test is asymptotically uniformly more powerful. For practical purposes, a bootstrap variant is suggested. In a simulation study, level and power of this test will be briefly investigated and compared with other procedures. In summary, our theoretical findings are supported by this study. Finally, a crop yield experiment is reanalysed.     

Efficient regression modeling for correlated and overdispersed count data

Abstract

The objective of this paper is to propose an efficient estimation procedure in a marginal mean regression model for longitudinal count data and to develop a hypothesis test for detecting the presence of overdispersion. We extend the matrix expansion idea of quadratic inference functions to the negative binomial regression framework that entails accommodating both the within-subject correlation and overdispersion issue. Theoretical and numerical results show that the proposed procedure yields a more efficient estimator asymptotically than the one ignoring either the within-subject correlation or overdispersion. When the overdispersion is absent in data, the proposed method might hinder the estimation efficiency in practice, yet the Poisson regression based regression model is fitted to the data sufficiently well. Therefore, we construct the hypothesis test that recommends an appropriate model for the analysis of the correlated count data. Extensive simulation studies indicate that the proposed test can identify the effective model consistently. The proposed procedure is also applied to a transportation safety study and recommends the proposed negative binomial regression model.     

Penalized Pseudolikelihood Inference in Spatial Interaction Models with Covariates

Given spatially located observed random variables ( x , z = {( x _i , z _i )} _i , we propose a new method for non-parametric estimation of the potential functions of a Markov random field p ( x | z ), based on a roughness penalty approach. The new estimator maximizes the penalized log-pseudolikelihood function and is a natural cubic spline. The calculations involved do not rely on Monte Carlo simulation. We suggest the use of B-splines to stabilize the numerical procedure. An application in Bayesian image reconstruction is described.     

Estimating Mixture of Gaussian Processes by Kernel Smoothing

When functional data are not homogenous, for example, when there are multiple classes of functional curves in the dataset, traditional estimation methods may fail. In this article, we propose a new estimation procedure for the mixture of Gaussian processes, to incorporate both functional and inhomogenous properties of the data. Our method can be viewed as a natural extension of high-dimensional normal mixtures. However, the key difference is that smoothed structures are imposed for both the mean and covariance functions. The model is shown to be identifiable, and can be estimated efficiently by a combination of the ideas from expectation-maximization (EM) algorithm, kernel regression, and functional principal component analysis. Our methodology is empirically justified by Monte Carlo simulations and illustrated by an analysis of a supermarket dataset.