Semiparametric multiple kernel estimators and model diagnostics for count regression functions

Abstract

This study concerns semiparametric approaches to estimate discrete multivariate count regression functions. The semiparametric approaches investigated consist of combining discrete multivariate nonparametric kernel and parametric estimations such that (i) a prior knowledge of the conditional distribution of model response may be incorporated and (ii) the bias of the traditional nonparametric kernel regression estimator of Nadaraya-Watson may be reduced. We are precisely interested in combination of the two estimations approaches with some asymptotic properties of the resulting estimators. Asymptotic normality results were showed for nonparametric correction terms of parametric start function of the estimators. The performance of discrete semiparametric multivariate kernel estimators studied is illustrated using simulations and real count data. In addition, diagnostic checks are performed to test the adequacy of the parametric start model to the true discrete regression model. Finally, using discrete semiparametric multivariate kernel estimators provides a bias reduction when the parametric multivariate regression model used as start regression function belongs to a neighborhood of the true regression model.     

A Partially Linear Model Using a Gaussian Process Prior

A partially linear model is a semiparametric regression model that consists of parametric and nonparametric regression components in an additive form. In this article, we propose a partially linear model using a Gaussian process regression approach and consider statistical inference of the proposed model. Based on the proposed model, the estimation procedure is described by posterior distributions of the unknown parameters and model comparisons between parametric representation and semi- and nonparametric representation are explored. Empirical analysis of the proposed model is performed with synthetic data and real data applications.     

NEW EFFICIENT ESTIMATION AND VARIABLE SELECTION METHODS FOR SEMIPARAMETRIC VARYING-COEFFICIENT PARTIALLY LINEAR MODELS

The complexity of semiparametric models poses new challenges to statistical inference and model selection that frequently arise from real applications. In this work, we propose new estimation and variable selection procedures for the semiparametric varying-coefficient partially linear model. We first study quantile regression estimates for the nonparametric varying-coefficient functions and the parametric regression coefficients. To achieve nice efficiency properties, we further develop a semiparametric composite quantile regression procedure. We establish the asymptotic normality of proposed estimators for both the parametric and nonparametric parts and show that the estimators achieve the best convergence rate. Moreover, we show that the proposed method is much more efficient than the least-squares-based method for many non-normal errors and that it only loses a small amount of efficiency for normal errors. In addition, it is shown that the loss in efficiency is at most 11.1% for estimating varying coefficient functions and is no greater than 13.6% for estimating parametric components. To achieve sparsity with high-dimensional covariates, we propose adaptive penalization methods for variable selection in the semiparametric varying-coefficient partially linear model and prove that the methods possess the oracle property. Extensive Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. Finally, we apply the new methods to analyze the plasma beta-carotene level data.     

Estimation of regression parameters in a semiparametric transformation model

A semiparametric approach to model skewed/heteroscedastic regression data is discussed. We work with a semiparametric transform-both-sides regression model, which contains a parametric regression function and a nonparametric transformation. This model is adequate when the relationship between the median response and the explanatory variable has been specified by a theoretical result or a previous empirical study. The transform-both-sides model with a parametric transformation has been studied extensively and applied successfully to a number data sets. Allowing a nonparametric transformation function increases the flexibility of the model. In this article, we estimate the nonparametric transformation function by the conditional kernel density approach developed by Wang and Ruppert (1995), and then use a pseudo-maximum likelihood estimator to estimate the regression parameters. This estimate of the regression parameters has not been studied previously. In this article, the asymptotic distribution of this pseudo-MLE is derived. We also show that when σ, the standard deviation of the error, goes to zero (small σ asymptotics), this estimator is adaptive. Adaptive means that the regression parameters are estimated as precisely as when the transformation is known exactly. A similar result holds in the parametric approaches of Carroll and Ruppert (1984) and Ruppert and Aldershof (1989). Simulated and real examples are provided to illustrate the performance of the proposed estimator for finite sample size.     

On Semiparametric Mode Regression Estimation

It has been found that, for a variety of probability distributions, there is a surprising linear relation between mode, mean, and median. In this article, the relation between mode, mean, and median regression functions is assumed to follow a simple parametric model. We propose a semiparametric conditional mode (mode regression) estimation for an unknown (unimodal) conditional distribution function in the context of regression model, so that any m-step-ahead mean and median forecasts can then be substituted into the resultant model to deliver m-step-ahead mode prediction. In the semiparametric model, Least Squared Estimator (LSEs) for the model parameters and the simultaneous estimation of the unknown mean and median regression functions by the local linear kernel method are combined to infer about the parametric and nonparametric components of the proposed model. The asymptotic normality of these estimators is derived, and the asymptotic distribution of the parameter estimates is also given and is shown to follow usual parametric rates in spite of the presence of the nonparametric component in the model. These results are applied to obtain a data-based test for the dependence of mode regression over mean and median regression under a regression model.     

Bayesian spectral analysis models for quantile regression with Dirichlet process mixtures

This paper presents a Bayesian analysis of partially linear additive models for quantile regression. We develop a semiparametric Bayesian approach to quantile regression models using a spectral representation of the nonparametric regression functions and the Dirichlet process (DP) mixture for error distribution. We also consider Bayesian variable selection procedures for both parametric and nonparametric components in a partially linear additive model structure based on the Bayesian shrinkage priors via a stochastic search algorithm. Based on the proposed Bayesian semiparametric additive quantile regression model referred to as BSAQ, the Bayesian inference is considered for estimation and model selection. For the posterior computation, we design a simple and efficient Gibbs sampler based on a location-scale mixture of exponential and normal distributions for an asymmetric Laplace distribution, which facilitates the commonly used collapsed Gibbs sampling algorithms for the DP mixture models. Additionally, we discuss the asymptotic property of the sempiparametric quantile regression model in terms of consistency of posterior distribution. Simulation studies and real data application examples illustrate the proposed method and compare it with Bayesian quantile regression methods in the literature.     

Generalized partially linear varying coefficient models with multiple smoothing variables

This paper is concerned with semiparametric efficient estimation of a generalized partially linear varying coefficient model. The model studied in this paper is very flexible, accommodating various nonlinear relations between the response variable and a set of predictor variables. It is a structured regression model and is particularly useful in dealing with a discrete response variable. We apply the smooth backfitting technique to estimate the nonparametric part of the model and employ the profiling approach to obtain a semiparametric efficient estimator of the parametric part.     

Semiparametric regression under long-range dependent errors

In this paper, we consider a semiparametric regression model under long-range dependent errors. By approximating the nonparametric component by a finite series sum, we construct consistent estimators for both parametric and nonparametric components. Meanwhile, convergence rates for the consistent estimators are also investigated. Additionally, an optimal truncation parameter selection procedure is proposed.     

On inference for a semiparametric partially linear regression model with serially correlated errors

The authors consider a semiparametric partially linear regression model with serially correlated errors. They propose a new way of estimating the error structure which has the advantage that it does not involve any nonparametric estimation. This allows them to develop an inference procedure consisting of a bandwidth selection method, an efficient semiparametric generalized least squares estimator of the parametric component, a goodness‐of‐fit test based on the bootstrap, and a technique for selecting significant covariates in the parametric component. They assess their approach through simulation studies and illustrate it with a concrete application.     

Bayesian semiparametric inference for the accelerated failure-time model

Bayesian semiparametric inference is considered for a loglinear model. This model consists of a parametric component for the regression coefficients and a nonparametric component for the unknown error distribution. Bayesian analysis is studied for the case of a parametric prior on the regression coefficients and a mixture-of-Dirichlet-processes prior on the unknown error distribution. A Markov-chain Monte Carlo (MCMC) method is developed to compute the features of the posterior distribution. A model selection method for obtaining a more parsimonious set of predictors is studied. The method adds indicator variables to the regression equation. The set of indicator variables represents all the possible subsets to be considered. A MCMC method is developed to search stochastically for the best subset. These procedures are applied to two examples, one with censored data.     

Influence Diagnostics of Semiparametric Nonlinear Reproductive Dispersion Models

This article proposes a semiparametric nonlinear reproductive dispersion model (SNRDM) which is an extension of nonlinear reproductive dispersion model and semiparametric regression model. Maximum penalized likelihood estimators (MPLEs) of unknown parameters and nonparametric functions in SNRDMs are presented. Some novel diagnostic statistics such as Cook distance and difference deviance for parametric and nonparametric parts are developed to identify influence observations in SNRDMs on the basis of case-deletion method, and some formulae readily computed with the MPLEs algorithm for diagnostic measures are given. The equivalency of case-deletion models and mean-shift outlier models in SNRDM is investigated. A simulation study and a real example are used to illustrate the proposed diagnostic measures.     

Smoothing for small samples with model misspecification: nonparametric and semiparametric concerns

Our goal is to find a regression technique that can be used in a small-sample situation with possible model misspecification. The development of a new bandwidth selector allows nonparametric regression (in conjunction with least squares) to be used in this small-sample problem, where nonparametric procedures have previously proven to be inadequate. Considered here are two new semiparametric (model-robust) regression techniques that combine parametric and nonparametric techniques when there is partial information present about the underlying model. A general overview is given of how typical concerns for bandwidth selection in nonparametric regression extend to the model-robust procedures. A new penalized PRESS criterion (with a graphical selection strategy for applications) is developed that overcomes these concerns and is able to maintain the beneficial mean squared error properties of the new model-robust methods. It is shown that this new selector outperforms standard and recently improved bandwidth selectors. Comparisons of the selectors are made via numerous generated data examples and a small simulation study.     

Restricted Estimation in Partially Linear Errors-in-Variables Models

As a compromise between parametric regression and nonparametric regression, partially linear models are frequently used in statistical modelling. This article considers statistical inference for this semiparametric model when the linear covariate is measured with additive error and some additional linear restrictions on the parametric component are assumed to hold. We propose a restricted corrected profile least-squares estimator for the parametric component, and study the asymptotic normality of the estimator. To test hypothesis on the parametric component, we construct a Wald test statistic and obtain its limiting distribution. Some simulation studies are conducted to illustrate our approaches.     

Dimension reduced kernel estimation for distribution function with incomplete data

This work focuses on the estimation of distribution functions with incomplete data, where the variable of interest Y has ignorable missingness but the covariate X is always observed. When X is high dimensional, parametric approaches to incorporate X—information is encumbered by the risk of model misspecification and nonparametric approaches by the curse of dimensionality. We propose a semiparametric approach, which is developed under a nonparametric kernel regression framework, but with a parametric working index to condense the high dimensional X—information for reduced dimension. This kernel dimension reduction estimator has double robustness to model misspecification and is most efficient if the working index adequately conveys the X—information about the distribution of Y. Numerical studies indicate better performance of the semiparametric estimator over its parametric and nonparametric counterparts. We apply the kernel dimension reduction estimation to an HIV study for the effect of antiretroviral therapy on HIV virologic suppression.     

Estimating the error distribution function in semiparametric additive regression models

We consider semiparametric additive regression models with a linear parametric part and a nonparametric part, both involving multivariate covariates. For the nonparametric part we assume two models. In the first, the regression function is unspecified and smooth; in the second, the regression function is additive with smooth components. Depending on the model, the regression curve is estimated by suitable least squares methods. The resulting residual-based empirical distribution function is shown to differ from the error-based empirical distribution function by an additive expression, up to a uniformly negligible remainder term. This result implies a functional central limit theorem for the residual-based empirical distribution function. It is used to test for normal errors.     

Semiparametric estimation of the cumulative incidence function under competing risks and left-truncated sampling

In this article, we propose semiparametric methods to estimate the cumulative incidence function of two dependent competing risks for left-truncated and right-censored data. The proposed method is based on work by Huang and Wang (1995). We extend previous model by allowing for a general parametric truncation distribution and a third competing risk before recruitment. Based on work by Vardi (1989), several iterative algorithms are proposed to obtain the semiparametric estimates of cumulative incidence functions. The asymptotic properties of the semiparametric estimators are derived. Simulation results show that a semiparametric approach assuming the parametric truncation distribution is correctly specified produces estimates with smaller mean squared error than those obtained in a fully nonparametric model.     

Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

Semiparametric inference for the recurrent events process by means of a single-index model

In this paper, we introduce new parametric and semiparametric regression techniques for a recurrent event process subject to random right censoring. We develop models for the cumulative mean function and provide asymptotically normal estimators. Our semiparametric model which relies on a single-index assumption can be seen as a dimension reduction technique that, contrary to a fully nonparametric approach, is not stroke by the curse of dimensionality when the number of covariates is high. We discuss data-driven techniques to choose the parameters involved in the estimation procedures and provide a simulation study to support our theoretical results.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

A gradient-based algorithm for semiparametric models with missing covariates

In the parametric regression model, the covariate missing problem under missing at random is considered. It is often desirable to use flexible parametric or semiparametric models for the covariate distribution, which can reduce a potential misspecification problem. Recently, a completely nonparametric approach was developed by [H.Y. Chen, Nonparametric and semiparametric models for missing covariates in parameter regression, J. Amer. Statist. Assoc. 99 (2004), pp. 1176–1189; Z. Zhang and H.E. Rockette, On maximum likelihood estimation in parametric regression with missing covariates, J. Statist. Plann. Inference 47 (2005), pp. 206–223]. Although it does not require a model for the covariate distribution or the missing data mechanism, the proposed method assumes that the covariate distribution is supported only by observed values. Consequently, their estimator is a restricted maximum likelihood estimator (MLE) rather than the global MLE. In this article, we show the restricted semiparametric MLE could be very misleading in some cases. We discuss why this problem occurs and suggest an algorithm to obtain the global MLE. Then, we assess the performance of the proposed method via some simulation experiments.