Robust extrapolation designs and weights for biased regression models with heteroscedastic errors

We consider the construction of designs for the extrapolation of regression responses, allowing both for possible heteroscedasticity in the errors and for imprecision in the specification of the response function. We find minimax designs and correspondingly optimal estimation weights in the context of the following problems: (1) for ordinary least squares estimation, determine a design to minimize the maximum value of the integrated mean squared prediction error (IMSPE), with the maximum being evaluated over both types of departure; (2) for weighted least squares estimation, determine both weights and a design to minimize the maximum IMSPE; (3) choose weights and design points to minimize the maximum IMSPE, subject to a side condition of unbiasedness. Solutions to (1) and (2) are given for multiple linear regression with no interactions, a spherical design space and an annular extrapolation space. For (3) the solution is given in complete generality; as one example we consider polynomial regression. Applications to a dose-response problem for bioassays are discussed. Numerical comparisons, including a simulation study, indicate that, as well as being easily implemented, the designs and weights for (3) perform as well as those for (1) and (2) and outperform some common competitors for moderate but undetectable amounts of model bias.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.     

General m-esttmators and applications to bounded influence estimation for non-linear regression

In this paper, we study the M-estimators in the case that λ_F:(β)=E_F:(φ(Z,β))=0 has more than one solution, We show that the numerical iterative procedures converge and that the resulting estimators are consistent and asymptotically normal. We apply them to the non-linear regression models, and then, we find an optimal M-estimate among those that have bounded gross error sensitivity.     

Model averaging for multivariate multiple regression models

The theories and applications of model averaging have been developed comprehensively in the past two decades. In this paper, we consider model averaging for multivariate multiple regression models. In order to make use of the correlation information of the dependent variables sufficiently, we propose a model averaging method based on Mahalanobis distance which is related to the correlation of the dependent variables. We prove the asymptotic optimality of the resulting Mahalanobis Mallows model averaging (MMMA) estimators under certain assumptions. In the simulation study, we show that the proposed MMMA estimators compare favourably with model averaging estimators based on AIC and BIC weights and the Mallows model averaging estimators from the single dependent variable regression models. We further apply our method to the real data on urbanization rate and the proportion of non-agricultural population in ethnic minority areas of China.     

Quantile regression for robust estimation and variable selection in partially linear varying-coefficient models

In this paper, we develop a new estimation procedure based on quantile regression for semiparametric partially linear varying-coefficient models. The proposed estimation approach is empirically shown to be much more efficient than the popular least squares estimation method for non-normal error distributions, and almost not lose any efficiency for normal errors. Asymptotic normalities of the proposed estimators for both the parametric and nonparametric parts are established. To achieve sparsity when there exist irrelevant variables in the model, two variable selection procedures based on adaptive penalty are developed to select important parametric covariates as well as significant nonparametric functions. Moreover, both these two variable selection procedures are demonstrated to enjoy the oracle property under some regularity conditions. Some Monte Carlo simulations are conducted to assess the finite sample performance of the proposed estimators, and a real-data example is used to illustrate the application of the proposed methods.     

Minimum hellinger distance estimation for normal models

A robust estimator introduced by Beran (1977a, 1977b), which is based on the minimum Hellinger distance between a projection model density and a nonparametric sample density, is studied empirically. An extensive simulation provides an estimate of the small sample distribution and supplies empirical evidence of the estimator performance for a normal location-scale model. While the performance of the minimum Hellinger distance estimator is seen to be competitive with the maximum likelihood estimator at the true model, its robustness to deviations from normality is shown to be competitive in this setting with that obtained from the M-estimator and the Cramér-von Mises minimum distance estimator. Beran also introduced a goodness-of-fit statisticH ₂, based on the minimized Hellinger distance between a member of a parametric family of densities and a nonparametric density estimate. We investigate the statistic H (the square root of H ₂) as a test for normality when both location and scale are unspecified. Empirically derived critical values are given which do not require extensive tables. The power of the statistic H compares favorably with four other widely used tests for normality.     

LASSO and shrinkage estimation in Weibull censored regression models

In this paper we address the problem of estimating a vector of regression parameters in the Weibull censored regression model. Our main objective is to provide natural adaptive estimators that significantly improve upon the classical procedures in the situation where some of the predictors may or may not be associated with the response. In the context of two competing Weibull censored regression models (full model and candidate submodel), we consider an adaptive shrinkage estimation strategy that shrinks the full model maximum likelihood estimate in the direction of the submodel maximum likelihood estimate. We develop the properties of these estimators using the notion of asymptotic distributional risk. The shrinkage estimators are shown to have higher efficiency than the classical estimators for a wide class of models. Further, we consider a LASSO type estimation strategy and compare the relative performance with the shrinkage estimators. Monte Carlo simulations reveal that when the true model is close to the candidate submodel, the shrinkage strategy performs better than the LASSO strategy when, and only when, there are many inactive predictors in the model. Shrinkage and LASSO strategies are applied to a real data set from Veteran's administration (VA) lung cancer study to illustrate the usefulness of the procedures in practice.     

Joint estimation of multiple high‐dimensional Gaussian copula graphical models

A joint estimation approach for multiple high‐dimensional Gaussian copula graphical models is proposed, which achieves estimation robustness by exploiting non‐parametric rank‐based correlation coefficient estimators. Although we focus on continuous data in this paper, the proposed method can be extended to deal with binary or mixed data. Based on a weighted minimisation problem, the estimators can be obtained by implementing second‐order cone programming. Theoretical properties of the procedure are investigated. We show that the proposed joint estimation procedure leads to a faster convergence rate than estimating the graphs individually. It is also shown that the proposed procedure achieves an exact graph structure recovery with probability tending to 1 under certain regularity conditions. Besides theoretical analysis, we conduct numerical simulations to compare the estimation performance and graph recovery performance of some state‐of‐the‐art methods including both joint estimation methods and estimation methods for individuals. The proposed method is then applied to a gene expression data set, which illustrates its practical usefulness.     

M‐estimation for general ARMA Processes with Infinite Variance

Abstract. General autoregressive moving average (ARMA) models extend the traditional ARMA models by removing the assumptions of causality and invertibility. The assumptions are not required under a non‐Gaussian setting for the identifiability of the model parameters in contrast to the Gaussian setting. We study M‐estimation for general ARMA processes with infinite variance, where the distribution of innovations is in the domain of attraction of a non‐Gaussian stable law. Following the approach taken by Davis et al. (1992) and Davis (1996) , we derive a functional limit theorem for random processes based on the objective function, and establish asymptotic properties of the M‐estimator. We also consider bootstrapping the M‐estimator and extend the results of Davis & Wu (1997) to the present setting so that statistical inferences are readily implemented. Simulation studies are conducted to evaluate the finite sample performance of the M‐estimation and bootstrap procedures. An empirical example of financial time series is also provided.     

Developing ridge estimation method for median regression

In this paper, the ridge estimation method is generalized to the median regression. Though the least absolute deviation (LAD) estimation method is robust in the presence of non-Gaussian or asymmetric error terms, it can still deteriorate into a severe multicollinearity problem when non-orthogonal explanatory variables are involved. The proposed method increases the efficiency of the LAD estimators by reducing the variance inflation and giving more room for the bias to get a smaller mean squared error of the LAD estimators. This paper includes an application of the new methodology and a simulation study as well.     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

Minimax A‐ and D‐optimal integer‐valued wavelet designs for estimation

The author discusses integer‐valued designs for wavelet estimation of nonparametric response curves in the possible presence of heteroscedastic noise using a modified wavelet version of the Gasser‐Müller kernel estimator or weighted least squares estimation. The designs are constructed using a minimax treatment and the simulated annealing algorithm. The author presents designs for three case studies in experiments for investigating Gompertz's theory on mortality rates, nitrite utilization in bush beans and the impact of crash helmets in motorcycle accidents.     

Deviance residuals in generalised log-gamma regression models with censored observations

In this article, we compare three residuals based on the deviance component in generalised log-gamma regression models with censored observations. For different parameter settings, sample sizes and censoring percentages, various simulation studies are performed and the empirical distribution of each residual is displayed and compared with the standard normal distribution. For all cases studied, the empirical distributions of the proposed residuals are in general symmetric around zero, but only a martingale-type residual presented negligible kurtosis for the majority of the cases studied. These studies suggest that the residual analysis usually performed in normal linear regression models can be straightforwardly extended for the martingale-type residual in generalised log-gamma regression models with censored data. A lifetime data set is analysed under log-gamma regression models and a model checking based on the martingale-type residual is performed.     

A hybrid Newton-type method for censored survival data using double weights in linear models

As an alternative to the Cox model, the rank-based estimating method for censored survival data has been studied extensively since it was proposed by Tsiatis [Tsiatis AA (1990) Ann Stat 18:354–372] among others. Due to the discontinuity feature of the estimating function, a significant amount of work in the literature has been focused on numerical issues. In this article, we consider the computational aspects of a family of doubly weighted rank-based estimating functions. This family is rich enough to include both estimating functions of Tsiatis (1990) for the randomly observed data and of Nan et al. [Nan B, Yu M, Kalbfleisch JD (2006) Biometrika (to appear)] for the case-cohort data as special examples. The latter belongs to the biased sampling problems. We show that the doubly weighted rank-based discontinuous estimating functions are monotone, a property established for the randomly observed data in the literature, when the generalized Gehan-type weights are used. Though the estimating problem can be formulated to a linear programming problem as that for the randomly observed data, due to its easily uncontrollable large scale even for a moderate sample size, we instead propose a Newton-type iterated method to search for an approximate solution of the (system of) discontinuous monotone estimating equation(s). Simulation results provide a good demonstration of the proposed method. We also apply our method to a real data example.     

Moment Consistency of the Exchangeably Weighted Bootstrap for Semiparametric M‐estimation

The bootstrap variance estimate is widely used in semiparametric inferences. However, its theoretical validity is a well‐known open problem. In this paper, we provide a first theoretical study on the bootstrap moment estimates in semiparametric models. Specifically, we establish the bootstrap moment consistency of the Euclidean parameter, which immediately implies the consistency of t‐type bootstrap confidence set. It is worth pointing out that the only additional cost to achieve the bootstrap moment consistency in contrast with the distribution consistency is to simply strengthen the L₁ maximal inequality condition required in the latter to the L_p maximal inequality condition for p≥1. The general L_p multiplier inequality developed in this paper is also of independent interest. These general conclusions hold for the bootstrap methods with exchangeable bootstrap weights, for example, non‐parametric bootstrap and Bayesian bootstrap. Our general theory is illustrated in the celebrated Cox regression model.     

Goodness‐of‐fit tests for parametric models based on biased samples

The authors study the problem of checking the adequacy of a parametric model for a distribution using several possibly censored weight biased samples. They discuss identifiability problems related to the underlying distribution and the distributions of the biased samples. They propose a test statistic based on the supremum of the weighted aggregated martingale residual processes from a number of such samples. Both numerical and graphical procedures are discussed, which the authors apply to do model checking for oil exploration drilling data.     

Robust sampling strategies for regression estimation

For probability linear regression estimation, conditions are derived where sampling will be robust against violations of the commonly assumed heterogeneous variance model. Stratified pps (spps) and stratified random sampling (spscx) are shown to satisfy these conditions approximately and are more efficient generally than restricted simple random sampling (RSRS) for some real populations and for artificial populations with weights of k = 0, 0.5, 1.0, 1.5 and 2.0. The criteria needs some additional refinement to better predict relative efficiency of spps and spscx.     

An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.