Adaptive asymptotically efficient estimation in heteroscedastic nonparametric regression

The paper considers some asymptotic properties of the adaptive procedure proposed in authors’ paper, 2007, for estimating an unknown nonparametric regression. We show that the procedure is asymptotically efficient for quadratic risk, i.e. the asymptotic quadratic risk of the procedure coincides with the corresponding Pinsker constant provided the sharp lower bound for the quadratic risk over all possible estimators.     

Quantile inference for heteroscedastic regression models

Consider the nonparametric heteroscedastic regression model Y=m(X)+σ(X)?, where m(·) is an unknown conditional mean function and σ(·) is an unknown conditional scale function. In this paper, the limit distribution of the quantile estimate for the scale function σ(X) is derived. Since the limit distribution depends on the unknown density of the errors, an empirical likelihood ratio statistic based on quantile estimator is proposed. This statistics is used to construct confidence intervals for the variance function. Under certain regularity conditions, it is shown that the quantile estimate of the scale function converges to a Brownian motion and the empirical likelihood ratio statistic converges to a chi-squared random variable. Simulation results demonstrate the superiority of the proposed method over the least squares procedure when the underlying errors have heavy tails.     

Efficient quantile regression for heteroscedastic models

Quantile regression (QR) provides estimates of a range of conditional quantiles. This stands in contrast to traditional regression techniques, which focus on a single conditional mean function. Lee et al. [Regularization of case-specific parameters for robustness and efficiency. Statist Sci. 2012;27(3):350–372] proposed efficient QR by rounding the sharp corner of the loss. The main modification generally involves an asymmetric ?₂ adjustment of the loss function around zero. We extend the idea of ?₂ adjusted QR to linear heterogeneous models. The ?₂ adjustment is constructed to diminish as sample size grows. Conditions to retain consistency properties are also provided.     

Estimation and variable selection in nonparametric heteroscedastic regression

The article considers a Gaussian model with the mean and the variance modeled flexibly as functions of the independent variables. The estimation is carried out using a Bayesian approach that allows the identification of significant variables in the variance function, as well as averaging over all possible models in both the mean and the variance functions. The computation is carried out by a simulation method that is carefully constructed to ensure that it converges quickly and produces iterates from the posterior distribution that have low correlation. Real and simulated examples demonstrate that the proposed method works well. The method in this paper is important because (a) it produces more realistic prediction intervals than nonparametric regression estimators that assume a constant variance; (b) variable selection identifies the variables in the variance function that are important; (c) variable selection and model averaging produce more efficient prediction intervals than those obtained by regular nonparametric regression.     

On nonparametric regression estimators based on regression quantiles

In the ciassical regression model Y_i=h(x_i) + ? _i, i=1,…,n, Cheng (1984) introduced linear combinations of regression quantiles as a new class of estimators for the unknown regression function h(x). The asymptotic properties studied in Cheng (1984) are reconsidered. We obtain a sharper scrong consistency rate and we improve on the conditions for asymptotic normality by proving a new result on the remainder term in the Bahadur representation for regression quantiles.     

A-optimal designs for heteroscedastic multifactor regression models

Abstract

This paper searches for A-optimal designs for Kronecker product and additive regression models when the errors are heteroscedastic. Sufficient conditions are given so that A-optimal designs for the multifactor models can be built from A-optimal designs for their sub-models with a single factor. The results of an efficiency study carried out to check the adequacy of the products of optimal designs for uni-factor marginal models when these are used to estimate different multi-factor models are also reported.     

Jackknife inference for heteroscedastic linear regression models

Inference on the regression parameters in a heteroscedastic linear regression model with replication is considered, using either the ordinary least-squares (OLS) or the weighted least-squares (WLS) estimator. A delete-group jackknife method is shown to produce consistent variance estimators irrespective of within-group correlations, unlike the delete-one jackknife variance estimators or those based on the customary δ-method assuming within-group independence. Finite-sample properties of the delete-group variance estimators and associated confidence intervals are also studied through simulation.     

Robust plug-in bandwidth estimators in nonparametric regression

In this paper, we propose a robust bandwidth selection method for local M-estimates used in nonparametric regression. We study the asymptotic behavior of the resulting estimates. We use the results of a Monte Carlo study to compare the performance of various competitors for moderate samples sizes. It appears that the robust plug-in bandwidth selector we propose compares favorably to its competitors, despite the need to select a pilot bandwidth. The Monte Carlo study shows that the robust plug-in bandwidth selector is very stable and relatively insensitive to the choice of the pilot.     

Switching nonparametric regression models

We propose a methodology to analyse data arising from a curve that, over its domain, switches among J states. We consider a sequence of response variables, where each response y depends on a covariate x according to an unobserved state z. The states form a stochastic process and their possible values are j=1,?…?, J. If z equals j the expected response of y is one of J unknown smooth functions evaluated at x. We call this model a switching nonparametric regression model. We develop an Expectation–Maximisation algorithm to estimate the parameters of the latent state process and the functions corresponding to the J states. We also obtain standard errors for the parameter estimates of the state process. We conduct simulation studies to analyse the frequentist properties of our estimates. We also apply the proposed methodology to the well-known motorcycle dataset treating the data as coming from more than one simulated accident run with unobserved run labels.     

Corrected maximum likelihood estimators in heteroscedastic symmetric nonlinear models

In this article, we derive general matrix formulae for second-order biases of maximum likelihood estimators (MLEs) in a class of heteroscedastic symmetric nonlinear regression models, thus generalizing some results in the literature. This class of regression models includes all symmetric continuous distributions, and has a wide range of practical applications in various fields such as engineering, biology, medicine and economics, among others. The variety of distributions with different kurtosis coefficients than the normal may give more flexibility in the choice of an appropriate distribution, particularly to accommodate outlying and influential observations. We derive a joint iterative process for estimating the mean and dispersion parameters. We also present simulation studies for the biases of the MLEs.     

A fiducial p-value approach for comparing heteroscedastic regression models

To study the equality of regression coefficients in several heteroscedastic regression models, we propose a fiducial-based test, and theoretically examine the frequency property of the proposed test. We numerically compare the performance of the proposed approach with the parametric bootstrap (PB) approach. Simulation results indicate that the fiducial approach controls the Type I error rates satisfactorily regardless of the number of regression models and sample sizes, whereas the PB approach tends to be a little of liberal in some scenarios. Finally, the proposed approach is applied to an analysis of a real dataset for illustration.     

Conditional variance estimation in heteroscedastic regression models

First, we propose a new method for estimating the conditional variance in heteroscedasticity regression models. For heavy tailed innovations, this method is in general more efficient than either of the local linear and local likelihood estimators. Secondly, we apply a variance reduction technique to improve the inference for the conditional variance. The proposed methods are investigated through their asymptotic distributions and numerical performances.     

Bayesian neural tree models for nonparametric regression

Frequentist and Bayesian methods differ in many aspects but share some basic optimal properties. In real-life prediction problems, situations exist in which a model based on one of the above paradigms is preferable depending on some subjective criteria. Nonparametric classification and regression techniques, such as decision trees and neural networks, have both frequentist (classification and regression trees (CARTs) and artificial neural networks) as well as Bayesian counterparts (Bayesian CART and Bayesian neural networks) to learning from data. In this paper, we present two hybrid models combining the Bayesian and frequentist versions of CART and neural networks, which we call the Bayesian neural tree (BNT) models. BNT models can simultaneously perform feature selection and prediction, are highly flexible, and generalise well in settings with limited training observations. We study the statistical consistency of the proposed approaches and derive the optimal value of a vital model parameter. The excellent performance of the newly proposed BNT models is shown using simulation studies. We also provide some illustrative examples using a wide variety of standard regression datasets from a public available machine learning repository to show the superiority of the proposed models in comparison to popularly used Bayesian CART and Bayesian neural network models.     

Quantile regression in heteroscedastic varying coefficient models

Varying coefficient models are flexible models to describe the dynamic structure in longitudinal data. Quantile regression, more than mean regression, gives partial information on the conditional distribution of the response given the covariates. In the literature, the focus has been so far mostly on homoscedastic quantile regression models, whereas there is an interest in looking into heteroscedastic modelling. This paper contributes to the area by modelling the heteroscedastic structure and estimating it from the data, together with estimating the quantile functions. The use of the proposed methods is illustrated on real-data applications. The finite-sample behaviour of the methods is investigated via a simulation study, which includes a comparison with an existing method.     

Bayesian and maximin optimal designs for heteroscedastic regression models

The authors consider the problem of constructing standardized maximin D‐optimal designs for weighted polynomial regression models. In particular they show that by following the approach to the construction of maximin designs introduced recently by Dette, Haines & Imhof (2003), such designs can be obtained as weak limits of the corresponding Bayesian _q‐optimal designs. They further demonstrate that the results are more broadly applicable to certain families of nonlinear models. The authors examine two specific weighted polynomial models in some detail and illustrate their results by means of a weighted quadratic regression model and the Bleasdale–Nelder model. They also present a capstone example involving a generalized exponential growth model.     

Robust adaptive estimators for binary regression models

This article introduces adaptive weighted maximum likelihood estimators for binary regression models. The asymptotic distribution under the model is established, and asymptotic confidence intervals are derived. Finite-sample properties are studied by simulation. For clean datasets, the proposed adaptive estimators are more efficient than the non-adaptive ones even for moderate sample sizes, and for outlier-contaminated datasets they show a comparable robustness. As for the asymptotic confidence intervals, the actual coverage levels under the model are very close to the nominal levels (even for moderate sample sizes), and they are reasonably stable under contamination.     

Testing equality of regression coefficients in heteroscedastic normal regression models

This article addresses the problem of testing whether the vectors of regression coefficients are equal for two independent normal regression models when the error variances are unknown. This problem poses severe difficulties both to the frequentist and Bayesian approaches to statistical inference. In the former approach, normal hypothesis testing theory does not apply because of the unrelated variances. In the latter, the prior distributions typically used for the parameters are improper and hence the Bayes factor-based solution cannot be used.We propose a Bayesian solution to this problem in which no subjective input is considered. We first generate “objective” proper prior distributions (intrinsic priors) for which the Bayes factor and model posterior probabilities are well defined. The posterior probability of each model is used as a model selection tool. This consistent procedure of testing hypotheses is compared with some of the frequentist approximate tests proposed in the literature.     

MOSUM monitoring for variance change in nonparametric regression models

ABSTRACT

This article considers the monitoring for variance change in nonparametric regression models. First, the local linear estimator of the regression function is given. A moving square cumulative sum procedure is proposed based on residuals of the estimator. And the asymptotic results of the statistic under the null hypothesis and the alternative hypothesis are obtained. Simulations and Application support our procedure.