Estimating the error variance in nonparametric regression by a covariate-matched u-statistic

For nonparametric regression models with fixed and random design, two classes of estimators for the error variance have been introduced: second sample moments based on residuals from a nonparametric fit, and difference-based estimators. The former are asymptotically optimal but require estimating the regression function; the latter are simple but have larger asymptotic variance. For nonparametric regression models with random covariates, we introduce a class of estimators for the error variance that are related to difference-based estimators: covariate-matched U-statistics. We give conditions on the random weights involved that lead to asymptotically optimal estimators of the error variance. Our explicit construction of the weights uses a kernel estimator for the covariate density.     

Empirical smoothing lack-of-fit tests for variance function

This paper discusses a nonparametric empirical smoothing lack-of-fit test for the functional form of the variance in regression models. The proposed test can be treated as a nontrivial modification of Zheng's nonparametric smoothing test, Koul and Ni's minimum distance test for the mean function in the classic regression models. The paper establishes the asymptotic normality of the proposed test under the null hypothesis. Consistency at some fixed alternatives and asymptotic power under some local alternatives are also discussed. A simulation study is conducted to assess the finite sample performance of the proposed test. Simulation study also shows that the proposed test is more powerful and computationally more efficient than some existing tests.     

Nonparametric quasi-likelihood for right censored data

Quasi-likelihood was extended to right censored data to handle heteroscedasticity in the frame of the accelerated failure time (AFT) model. However, the assumption of known variance function in the quasi-likelihood for right censored data is usually unrealistic. In this paper, we propose a nonparametric quasi-likelihood by replacing the specified variance function with a nonparametric variance function estimator. This nonparametric variance function estimator is obtained by smoothing a function of squared residuals via local polynomial regression. The rate of convergence of the nonparametric variance function estimator and the asymptotic limiting distributions of the regression coefficient estimators are derived. It is demonstrated in simulations that for finite samples the proposed nonparametric quasi-likelihood method performs well. The new method is illustrated with one real dataset.     

Weighted quantile regression and testing for varying-coefficient models with randomly truncated data

This paper develops a varying-coefficient approach to the estimation and testing of regression quantiles under randomly truncated data. In order to handle the truncated data, the random weights are introduced and the weighted quantile regression (WQR) estimators for nonparametric functions are proposed. To achieve nice efficiency properties, we further develop a weighted composite quantile regression (WCQR) estimation method for nonparametric functions in varying-coefficient models. The asymptotic properties both for the proposed WQR and WCQR estimators are established. In addition, we propose a novel bootstrap-based test procedure to test whether the nonparametric functions in varying-coefficient quantile models can be specified by some function forms. The performance of the proposed estimators and test procedure are investigated through simulation studies and a real data example.     

Estimation and variable selection in nonparametric heteroscedastic regression

The article considers a Gaussian model with the mean and the variance modeled flexibly as functions of the independent variables. The estimation is carried out using a Bayesian approach that allows the identification of significant variables in the variance function, as well as averaging over all possible models in both the mean and the variance functions. The computation is carried out by a simulation method that is carefully constructed to ensure that it converges quickly and produces iterates from the posterior distribution that have low correlation. Real and simulated examples demonstrate that the proposed method works well. The method in this paper is important because (a) it produces more realistic prediction intervals than nonparametric regression estimators that assume a constant variance; (b) variable selection identifies the variables in the variance function that are important; (c) variable selection and model averaging produce more efficient prediction intervals than those obtained by regular nonparametric regression.     

A nonparametric hypothesis test for heteroscedasticity

In this paper, a hypothesis test for heteroscedasticity is proposed in a nonparametric regression model. The test statistic, which uses the residuals from a nonparametric fit of the mean function, is based on an adaptation of the well-known Levene's test. Using the recent theory for analysis of variance when the number of factor levels goes to infinity, the asymptotic distribution of the test statistic is established under the null hypothesis of homocedasticity and under local alternatives. Simulations suggest that the proposed test performs well in several situations, especially when the variance is a nonlinear function of the predictor.     

Empirical likelihood based diagnostics for heteroscedasticity in partially linear errors-in-variables models

A standard assumption in regression analysis is homogeneity of the error variance. Violation of this assumption can have adverse consequences for the efficiency of estimators. In this paper, we propose an empirical likelihood based diagnostic technique for heteroscedasticity in the partially linear errors-in-variables models. Under mild conditions, a nonparametric version of Wilk's theorem is derived. Simulation results reveal that our test performs well in both size and power.     

Variance estimation in nonparametric regression with jump discontinuities

Variance estimation is an important topic in nonparametric regression. In this paper, we propose a pairwise regression method for estimating the residual variance. Specifically, we regress the squared difference between observations on the squared distance between design points, and then estimate the residual variance as the intercept. Unlike most existing difference-based estimators that require a smooth regression function, our method applies to regression models with jump discontinuities. Our method also applies to the situations where the design points are unequally spaced. Finally, we conduct extensive simulation studies to evaluate the finite-sample performance of the proposed method and compare it with some existing competitors.     

Consistency of Posterior Distributions for Heteroscedastic Nonparametric Regression Models

In this article, we consider Bayesian inferences for the heteroscedastic nonparametric regression models, when both the mean function and variance function are unknown. We demonstrated consistency of posterior distributions for this model using priors induced by B-splines expansion, treating both random and deterministic covariates in a uniform manner.     

Efficient estimation of variance components in nonparametric mixed-effects models with large samples

Linear mixed-effects (LME) regression models are a popular approach for analyzing correlated data. Nonparametric extensions of the LME regression model have been proposed, but the heavy computational cost makes these extensions impractical for analyzing large samples. In particular, simultaneous estimation of the variance components and smoothing parameters poses a computational challenge when working with large samples. To overcome this computational burden, we propose a two-stage estimation procedure for fitting nonparametric mixed-effects regression models. Our results reveal that, compared to currently popular approaches, our two-stage approach produces more accurate estimates that can be computed in a fraction of the time.     

On completely data-driven bandwidth selection for single-index models

The class of single-index models (SIMs) has become an important tool for nonparametric regression analysis. As with any other nonparametric regression models, the selection of bandwidth plays an important role in the inferences of the SIMs. However, most results in the literature either take the bandwidths as externally given, or require unpractical assumptions or very restrictive conditions for data-driven bandwidths. We examine the asymptotic properties of a popular bandwidth selection method based on cross-validation that is completely data-driven, under much weaker conditions than those assumed in the literature. And we show that the same bandwidth that is optimal for estimating the index vector, can be used for nearly optimal error variance estimation through the method of varying cross-validation. A simulation study is presented to demonstrate the finite sample performance of the proposed procedures, based on which we recommend a simple 2-step procedure for bandwidth selection, index vector estimation, as well as error variance estimation.     

On conditional variance estimation in nonparametric regression

In this paper we consider a nonparametric regression model in which the conditional variance function is assumed to vary smoothly with the predictor. We offer an easily implemented and fully Bayesian approach that involves the Markov chain Monte Carlo sampling of standard distributions. This method is based on a technique utilized by Kim, Shephard, and Chib (in Rev. Econ. Stud. 65:361–393, 1998) for the stochastic volatility model. Although the (parametric or nonparametric) heteroscedastic regression and stochastic volatility models are quite different, they share the same structure as far as the estimation of the conditional variance function is concerned, a point that has been previously overlooked. Our method can be employed in the frequentist context and in Bayesian models more general than those considered in this paper. Illustrations of the method are provided.     

Local Linear Least Squares Kernel Regression for Linear and Circular Predictors

We extend nonparametric regression models with local linear least squares fitting using kernel weights to the case of linear and circular predictors. We derive the asymptotic properties of the conditional bias and variance of bivariate local linear least squares kernel estimators. A small simulation study and a real experiment are given.     

Testing for structural changes in linear regressions with time-varying variance

Abstract

This article proposes a nonparametric test for structural changes in linear regression models that allows for serial correlation, autoregressive conditional heteroskedasticity and time-varying variance in error terms. The test requires no trimming of the boundary region near the end points of the sample period, and requires no prior information on the alternative, what it requires is the transformed OLS residuals under the null hypothesis. We show that the test has a limiting standard normal distribution under the null hypothesis, and is powerful against single break, multiple breaks and smooth structural changes. The Monte Carlo experiment is conducted to highlight the merits of the proposed test relative to other popular tests for structural changes.     

Nonparametric survival regression using the beta-Stacy process

A novel class of hierarchical nonparametric Bayesian survival regression models for time-to-event data with uninformative right censoring is introduced. The survival curve is modeled as a random function whose prior distribution is defined using the beta-Stacy (BS) process. The prior mean of each survival probability and its prior variance are linked to a standard parametric survival regression model. This nonparametric survival regression can thus be anchored to any reference parametric form, such as a proportional hazards or an accelerated failure time model, allowing substantial departures of the predictive survival probabilities when the reference model is not supported by the data. Also, under this formulation the predictive survival probabilities will be close to the empirical survival distribution near the mode of the reference model and they will be shrunken towards its probability density in the tails of the empirical distribution.     

A class of semiparametric transformation models for survival data with a cured proportion

We propose a new class of semiparametric regression models based on a multiplicative frailty assumption with a discrete frailty, which may account for cured subgroup in population. The cure model framework is then recast as a problem with a transformation model. The proposed models can explain a broad range of nonproportional hazards structures along with a cured proportion. An efficient and simple algorithm based on the martingale process is developed to locate the nonparametric maximum likelihood estimator. Unlike existing expectation-maximization based methods, our approach directly maximizes a nonparametric likelihood function, and the calculation of consistent variance estimates is immediate. The proposed method is useful for resolving identifiability features embedded in semiparametric cure models. Simulation studies are presented to demonstrate the finite sample properties of the proposed method. A case study of stage III soft-tissue sarcoma is given as an illustration.     

A nonparametric method for analysis of variance

A nonparametric method for analyzing analysis of variance models is introduced which is highly resistant to outliers, computationally simple, and comprehensible to anyone with a rudimentary knowledge of classical analysis of variance. The methodology is based on Mood's median test and is highly useful as an exploratory technique.     

Conditional variance estimation via nonparametric generalized additive models

Generalized additive models provide a way of circumventing curse of dimension in a wide range of nonparametric regression problem. In this paper, we present a multiplicative model for conditional variance functions where one can apply a generalized additive regression method. This approach extends Fan and Yao (1998) to multivariate cases with a multiplicative structure. In this approach, we use squared residuals instead of using log-transformed squared residuals. This idea gives a smaller variance than Yu (2017) when the variance of squared error is smaller than the variance of log-transformed squared error. We provide estimators based on quasi-likelihood and an iterative algorithm based on smooth backfitting for generalized additive models. We also provide some asymptotic properties of estimators and the convergence of proposed algorithm. A numerical study shows the empirical evidence of the theory.     

Testing heteroscedasticity in nonparametric regression

The importance of being able to detect heteroscedasticity in regression is widely recognized because efficient inference for the regression function requires that heteroscedasticity is taken into account. In this paper a simple consistent test for heteroscedasticity is proposed in a nonparametric regression set-up. The test is based on an estimator for the best L ²-approximation of the variance function by a constant. Under mild assumptions asymptotic normality of the corresponding test statistic is established even under arbitrary fixed alternatives. Confidence intervals are obtained for a corresponding measure of heteroscedasticity. The finite sample performance and robustness of these procedures are investigated in a simulation study and Box-type corrections are suggested for small sample sizes.