A simple test for comparing regression curves versus one-sided alternatives

In this article we present a simple procedure to test for the null hypothesis of equality of two regression curves versus one-sided alternatives in a general nonparametric and heteroscedastic setup. The test is based on the comparison of the sample averages of the estimated residuals in each regression model under the null hypothesis. The test statistic has asymptotic normal distribution and can detect any local alternative of rate n^-1/2. Some simulations and an application to a data set are included.     

Heteroscedastic log-exponentiated Weibull regression model

We introduce a new class of heteroscedastic log-exponentiated Weibull (LEW) regression models. The class of regression models can be applied to censored data and be used more effectively in survival analysis. Maximum likelihood estimation of the model parameters with censored data as well as influence diagnostics for the new regression model is investigated. For different parameter settings, sample sizes and censoring percentages, various simulation studies are performed and compared to the performance of the heteroscedastic LEW regression model. The normal curvatures for studying local influence are derived under various perturbation schemes. An empirical application to a real data set is provided to illustrate the usefulness of the new class of heteroscedastic regression models.     

The heteroscedastic odd log-logistic generalized gamma regression model for censored data

We propose a four-parameter extended generalized gamma model, which includes as special cases some important distributions and it is very useful for modeling lifetime data. A advantage is that it can represent the error distribution for a new heteroscedastic log-odd log-logistic generalized gamma regression model. The proposed heteroscedastic regression model can be used more effectively in the analysis of survival data since it includes as special models several widely-known regression models. Further, for different parameter settings, sample sizes and censoring percentages, various simulations are performed. Overall, the new regression model is very useful to the analysis of real data.     

Estimation of the Variance Function in Heteroscedastic Linear Regression Models

Heteroscedasticity generally exists when a linear regression model is applied to analyzing some real-world problems. Therefore, how to accurately estimate the variance functions of the error term in a heteroscedastic linear regression model is of great importance for obtaining efficient estimates of the regression parameters and making valid statistical inferences. A method for estimating the variance function of heteroscedastic linear regression models is proposed in this article based on the variance-reduced local linear smoothing technique. Some simulations and comparisons with other method are conducted to assess the performance of the proposed method. The results demonstrate that the proposed method can accurately estimate the variance functions and therefore produce more efficient estimates of the regression parameters.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Non-parametric Analysis of Covariance – The Case of Inhomogeneous and Heteroscedastic Noise

Abstract.  The purpose of this paper was to propose a procedure for testing the equality of several regression curves f _i in non-parametric regression models when the noise is inhomogeneous and heteroscedastic, i.e. when the variances depend on the regressor and may vary between groups. The presented approach is very natural because it transfers the maximum likelihood statistic from a heteroscedastic one-way analysis of variance to the context of non-parametric regression. The maximum likelihood estimators will be replaced by kernel estimators of the regression functions f _i . It is shown that the asymptotic distribution of the obtained test-statistic is nuisance parameter free. Asymptotic efficiency is compared with a test of Dette & Neumeyer [Annals of Statistics (2001) Vol. 29, 1361–1400] and it is shown that the new test is asymptotically uniformly more powerful. For practical purposes, a bootstrap variant is suggested. In a simulation study, level and power of this test will be briefly investigated and compared with other procedures. In summary, our theoretical findings are supported by this study. Finally, a crop yield experiment is reanalysed.     

The Performance of a Robust Multistage Estimator in Nonlinear Regression with Heteroscedastic Errors

In this article, a robust multistage parameter estimator is proposed for nonlinear regression with heteroscedastic variance, where the residual variances are considered as a general parametric function of predictors. The motivation is based on considering the chi-square distribution for the calculated sample variance of the data. It is shown that outliers that are influential in nonlinear regression parameter estimates are not necessarily influential in calculating the sample variance. This matter persuades us, not only to robustify the estimate of the parameters of the models for both the regression function and the variance, but also to replace the sample variance of the data by a robust scale estimate.     

Non-parametric Estimation of the Residual Distribution

Consider a heteroscedastic regression model Y = m ( X ) +σ( X )ε, where the functions m and σ are "smooth", and ε is independent of X . An estimator of the distribution of ε based on non-parametric regression residuals is proposed and its weak convergence is obtained. Applications to prediction intervals and goodness-of-fit tests are discussed.     

Local M-estimation for Conditional Variance in Heteroscedastic Regression Models

In this article, we develop a local M-estimation for the conditional variance in heteroscedastic regression models. The estimator is based on the local linear smoothing technique and the M-estimation technique, and it is shown to be not only asymptotically equivalent to the local linear estimator but also robust. The consistency and asymptotic normality of the local M-estimator for the conditional variance in heteroscedastic regression models are obtained under mild conditions. The simulation studies demonstrate that the proposed estimators perform well in robustness.     

Asymptotics for L1-estimators of regression parameters under heteroscedasticityY

We consider the asymptotic behaviour of L₁ -estimators in a linear regression under a very general form of heteroscedasticity. The limiting distributions of the estimators are derived under standard conditions on the design. We also consider the asymptotic behaviour of the bootstrap in the heteroscedastic model and show that it is consistent to first order only if the limiting distribution is normal.     

A flexible semiparametric regression model for bimodal,asymmetric and censored data

In this paper, we propose a new semiparametric heteroscedastic regression model allowing for positive and negative skewness and bimodal shapes using the B-spline basis for nonlinear effects. The proposed distribution is based on the generalized additive models for location, scale and shape framework in order to model any or all parameters of the distribution using parametric linear and/or nonparametric smooth functions of explanatory variables. We motivate the new model by means of Monte Carlo simulations, thus ignoring the skewness and bimodality of the random errors in semiparametric regression models, which may introduce biases on the parameter estimates and/or on the estimation of the associated variability measures. An iterative estimation process and some diagnostic methods are investigated. Applications to two real data sets are presented and the method is compared to the usual regression methods.     

Optimal Designs for Random Coefficient Regression Models with Heteroscedastic Errors

The purpose of this article is to present the optimal designs based on D-, G-, A-, I-, and D _β-optimality criteria for random coefficient regression (RCR) models with heteroscedastic errors. A sufficient condition for the heteroscedastic structure is given to make sure that the search of optimal designs can be confined at extreme settings of the design region when the criteria satisfy the assumption of the real valued monotone design criteria. Analytical solutions of D-, G-, A-, I-, and D _β-optimal designs for the RCR models are derived. Two examples are presented for random slope models with specific heteroscedastic errors.     

Nonparametric confidence regions via the analytic wild bootstrap

The wild bootstrap is a nonparametric tool that can be used to estimate a sampling distribution in the presence of heteroscedastic errors. In particular, the wild bootstrap enables us to compute confidence regions for regression parameters under non-i.i.d. models. While the wild bootstrap may perform well in these settings, its obvious drawback is a lack of computational efficiency. The wild bootstrap requires a large number of bootstrap replications, making the use of this tool impractical when dealing with big data. We introduce the analytic wild bootstrap (ANWB), which provides a nonparametric alternative way of constructing confidence regions for regression parameters. The ANWB is superior to the wild bootstrap from a computational standpoint while exhibiting similar finite-sample performance. We report simulation results for both least squares and ridge regression. Additionally, we test the ANWB on a real dataset and compare its performance with that of other standard approaches.     

Quantile regression in heteroscedastic varying coefficient models

Varying coefficient models are flexible models to describe the dynamic structure in longitudinal data. Quantile regression, more than mean regression, gives partial information on the conditional distribution of the response given the covariates. In the literature, the focus has been so far mostly on homoscedastic quantile regression models, whereas there is an interest in looking into heteroscedastic modelling. This paper contributes to the area by modelling the heteroscedastic structure and estimating it from the data, together with estimating the quantile functions. The use of the proposed methods is illustrated on real-data applications. The finite-sample behaviour of the methods is investigated via a simulation study, which includes a comparison with an existing method.     

Bayesian inference in a heteroscedastic replicated measurement error model using heavy-tailed distributions

We introduce a multivariate heteroscedastic measurement error model for replications under scale mixtures of normal distribution. The model can provide a robust analysis and can be viewed as a generalization of multiple linear regression from both model structure and distribution assumption. An efficient method based on Markov Chain Monte Carlo is developed for parameter estimation. The deviance information criterion and the conditional predictive ordinates are used as model selection criteria. Simulation studies show robust inference behaviours of the model against both misspecification of distributions and outliers. We work out an illustrative example with a real data set on measurements of plant root decomposition.     

Corrected maximum likelihood estimators in heteroscedastic symmetric nonlinear models

In this article, we derive general matrix formulae for second-order biases of maximum likelihood estimators (MLEs) in a class of heteroscedastic symmetric nonlinear regression models, thus generalizing some results in the literature. This class of regression models includes all symmetric continuous distributions, and has a wide range of practical applications in various fields such as engineering, biology, medicine and economics, among others. The variety of distributions with different kurtosis coefficients than the normal may give more flexibility in the choice of an appropriate distribution, particularly to accommodate outlying and influential observations. We derive a joint iterative process for estimating the mean and dispersion parameters. We also present simulation studies for the biases of the MLEs.     

Comparison of Nonparametric Regression Curves by Spline Smoothing

In this article, procedures are proposed to test the hypothesis of equality of two or more regression functions. Tests are proposed by p-values, first under homoscedastic regression model, which are derived using fiducial method based on cubic spline interpolation. Then, we construct a test in the heteroscedastic case based on Fisher's method of combining independent tests. We study the behaviors of the tests by simulation experiments, in which comparisons with other tests are also given. The proposed tests have good performances. Finally, an application to a data set are given to illustrate the usefulness of the proposed test in practice.     

Adaptive Estimation of the Integral of Squared Regression Derivatives

A problem of estimating the integral of a squared regression function and of its squared derivatives has been addressed in a number of papers. For the case of a heteroscedastic model where smoothness of the underlying regression function, the design density, and the variance of errors are known, the asymptotically sharp minimax lower bound and a sharp estimator were found in Pastuchova & Khasminski (1989). However, there are apparently no results on the either rate optimal or sharp optimal adaptive, or data-driven, estimation when neither the degree of regression function smoothness nor design density, scale function and distribution of errors are known. After a brief review of main developments in non-parametric estimation of non-linear functionals, we suggest a simple adaptive estimator for the integral of a squared regression function and its derivatives and prove that it is sharp-optimal whenever the estimated derivative is sufficiently smooth.     

Bayesian likelihood robustness in linear models

This paper deals with the problem of robustness of Bayesian regression with respect to the data. We first give a formal definition of Bayesian robustness to data contamination, prove that robustness according to the definition cannot be obtained by using heavy-tailed error distributions in linear regression models and propose a heteroscedastic approach to achieve the desired Bayesian robustness.     

DO NOT WEIGHT FOR HETEROSCEDASTICITY IN NONPARAMETRIC REGRESSION

The potential role of weighting in kernel regression is examined. The concept that weighting has something to do with heteroscedastic errors is shown to be false. However, weighting does affect bias, and ways in which this might be exploited are indicated.