Residuals from deletion in added variable plots

An added variable plot is a commonly used plot in regression diagnostics. The rationale for this plot is to provide information about the addition of a further explanatory variable to the model. In addition, an added variable plot is most often used for detecting high leverage points and influential data. So far as we know, this type of plot involves the least squares residuals which, we suspect, could produce a confusing picture when a group of unusual cases are present in the data. In this situation, added variable plots may not only fail to detect the unusual cases but also may fail to focus on the need for adding a further regressor to the model. We suggest that residuals from deletion should be more convincing and reliable in this type of plot. The usefulness of an added variable plot based on residuals from deletion is investigated through a few examples and a Monte Carlo simulation experiment in a variety of situations.     

Diagnostic plots in beta-regression models

Two diagnostic plots for selecting explanatory variables are introduced to assess the accuracy of a generalized beta-linear model. The added variable plot is developed to examine the need for adding a new explanatory variable to the model. The constructed variable plot is developed to identify the nonlinearity of the explanatory variable in the model. The two diagnostic procedures are also useful for detecting unusual observations that may affect the regression much. Simulation studies and analysis of two practical examples are conducted to illustrate the performances of the proposed plots.     

Properties of Added Variable Plots in Cox's Regression Model

The added variable plot is useful for examining the effect of a covariate in regression models. The plot provides information regarding the inclusion of a covariate, and is useful in identifying influential observations on the parameter estimates. Hall et al. (1996) proposed a plot for Cox's proportional hazards model derived by regarding the Cox model as a generalized linear model. This paper proves and discusses properties of this plot. These properties make the plot a valuable tool in model evaluation. Quantities considered include parameter estimates, residuals, leverage, case influence measures and correspondence to previously proposed residuals and diagnostics.     

Robust estimation for ordinal regression

Ordinal regression is used for modelling an ordinal response variable as a function of some explanatory variables. The classical technique for estimating the unknown parameters of this model is Maximum Likelihood (ML). The lack of robustness of this estimator is formally shown by deriving its breakdown point and its influence function. To robustify the procedure, a weighting step is added to the Maximum Likelihood estimator, yielding an estimator with bounded influence function. We also show that the loss in efficiency due to the weighting step remains limited. A diagnostic plot based on the Weighted Maximum Likelihood estimator allows to detect outliers of different types in a single plot.     

The Added Variable Plot for a Time Series of Counts

The issue of modelling non-Gaussian time series data is one that has been examined by several authors in recent years. Zeger (1988) introduced a parameter-driven model for a time series of counts as well as a more general observation-driven model for non-Gaussian data (Zeger & Qaqish, 1988). This paper examines the application of the added variable plot to these two models. This plot is useful for determining the strength of relationships and the detection of influential or outlying observations.     

The SSR Plot: A Graphical Representation for Regression

An alternative graphical method, called the SSR plot, is proposed for use with a multiple regression model. The new method uses the fact that the sum of squares for regression (SSR) of two explanatory variables can be partitioned into the SSR of one variable and the increment in SSR due to the addition of the second variable. The SSR plot represents each explanatory variable as a vector in a half circle. Our proposed SSR plot explains that the explanatory variables corresponding to the vectors located closer to the horizontal axis have stronger effects on the response variable. Furthermore, for a regression model with two explanatory variables, the magnitude of the angle between two vectors can be used to identify suppression.     

Added variable plots for linear regression with censored data

In linear regression the structure of the hat matrix plays an important part in regression diagnostics. In this note we investigate the properties of the hat matrix for regression with censored responses in the presence of one or more explanatory variables observed without censoring. The censored points in the scatterplot are renovated to positions had they been observed without censoring in a renovation process based on Buckley-James censored regression estimators. This allows natural links to be established with the structure of ordinary least squares estimators. In particular, we show that the renovated hat matrix may be partitioned in a manner which assists in deciding whether further explanatory variables should be added to the linear model. The added variable plot for regression with censored data is developed as a diagnostic tool for this decision process.     

Adjusted variable plots for Cox's proportional hazards regression model

Adjusted variable plots are useful in linear regression for outlier detection and for qualitative evaluation of the fit of a model. In this paper, we extend adjusted variable plots to Cox's proportional hazards model for possibly censored survival data. We propose three different plots: a risk level adjusted variable (RLAV) plot in which each observation in each risk set appears, a subject level adjusted variable (SLAV) plot in which each subject is represented by one point, and an event level adjusted variable (ELAV) plot in which the entire risk set at each failure event is represented by a single point. The latter two plots are derived from the RLAV by combining multiple points. In each point, the regression coefficient and standard error from a Cox proportional hazards regression is obtained by a simple linear regression through the origin fit to the coordinates of the pictured points. The plots are illustrated with a reanalysis of a dataset of 65 patients with multiple myeloma.     

Robust variable selection in finite mixture of regression models using the t distribution

Abstract

Variable selection in finite mixture of regression (FMR) models is frequently used in statistical modeling. The majority of applications of variable selection in FMR models use a normal distribution for regression error. Such assumptions are unsuitable for a set of data containing a group or groups of observations with heavy tails and outliers. In this paper, we introduce a robust variable selection procedure for FMR models using the t distribution. With appropriate selection of the tuning parameters, the consistency and the oracle property of the regularized estimators are established. To estimate the parameters of the model, we develop an EM algorithm for numerical computations and a method for selecting tuning parameters adaptively. The parameter estimation performance of the proposed model is evaluated through simulation studies. The application of the proposed model is illustrated by analyzing a real data set.     

A robust biplot

This paper introduces a robust biplot which is related to multivariate M-estimates. The n × p data matrix is first considered as a sample of size n from some p-variate population, and robust M-estimates of the population location vector and scatter matrix are calculated. In the construction of the biplot, each row of the data matrix is assigned a weight determined in the preliminary robust estimation. In a robust biplot, one can plot the variables in order to represent characteristics of the robust variance-covariance matrix: the length of the vector representing a variable is proportional to its robust standard deviation, while the cosine of the angle between two variables is approximately equal to their robust correlation. The proposed biplot also permits a meaningful representation of the variables in a robust principal-component analysis. The discrepancies between least-squares and robust biplots are illustrated in an example.     

Quantile plots for mean effects in the presence of variance effects for 2k-p designs

In robust parameter design, variance effects and mean effects in a factorial experiment are modelled simultaneously. If variance effects are present in a model, correlations are induced among the naive estimators of the mean effects. A simple normal quantile plot of the mean effects may be misleading because the mean effects are no longer iid under the null hypothesis that they are zero. Adjusted quantiles are computed for the case when one variance effect is significant and examples of 8-run and 16-run fractional factorial designs are examined in detail. We find that the usual normal quantiles are similar to adjusted quantiles for all but the largest and smallest ordered effects for which they are conservative. Graphically, the qualitative difference between the two sets of quantiles is negligible (even in the presence of large variance effects) and we conclude that normal probability plots are robust in the presence of variance effects.     

Testing the information matrix equality with robust estimators

The information matrix (IM) equality can be used to test for misspecification of a parametric model. We study the behavior of the IM test when the maximum-likelihood (ML) estimators used in the construction of this test are replaced with robust estimators. The latter do not suffer from the masking effect in the presence of outliers and can improve the power of the IM test. At the normal location-scale model, the IM test using the ML estimators is known as the Jarque–Bera test, and uses skewness and kurtosis to detect deviations from normality. When robust estimators are employed to test the IM equality, a robust version of the Jarque–Bera test emerges. We investigate in detail the local asymptotic power of the IM test, for various estimators and under a variety of local alternatives. For the normal regression model, it is shown by simulations under fixed alternatives that in many cases the use of robust estimators substantially increases the power of the IM test.     

A formula for a missing plot in a general incomplete block design,when recovery of interblock information is used

In this paper, Yate's missing plot technique is used to derive the formula for substitution in a missing plot in a general incomplete block design, where blocks are assumed to be independent normal. The use of penalized normal equations, using BLUPS, makes this task simpler.     

M-Estimation for partially functional linear regression model based on splines

ABSTRACT

M-estimation is a widely used technique for robust statistical inference. In this paper, we study robust partially functional linear regression model in which a scale response variable is explained by a function-valued variable and a finite number of real-valued variables. For the estimation of the regression parameters, which include the infinite dimensional function as well as the slope parameters for the real-valued variables, we use polynomial splines to approximate the slop parameter. The estimation procedure is easy to implement, and it is resistant to heavy-tailederrors or outliers in the response. The asymptotic properties of the proposed estimators are established. Finally, we assess the finite sample performance of the proposed method by Monte Carlo simulation studies.     

Assessing the adequacy of Weibull survival models: a simulated envelope approach

The Weibull proportional hazards model is commonly used for analysing survival data. However, formal tests of model adequacy are still lacking. It is well known that residual-based goodness-of-fit measures are inappropriate for censored data. In this paper, a graphical diagnostic plot of Cox–Snell residuals with a simulated envelope added is proposed to assess the adequacy of Weibull survival models. Both single component and two-component mixture models with random effects are considered for recurrent failure time data. The effectiveness of the diagnostic method is illustrated using simulated data sets and data on recurrent urinary tract infections of elderly women.     

Two-stage hierarchical modeling for analysis of subpopulations in conditional distributions

In this work, we develop modeling and estimation approach for the analysis of cross-sectional clustered data with multimodal conditional distributions where the main interest is in analysis of subpopulations. It is proposed to model such data in a hierarchical model with conditional distributions viewed as finite mixtures of normal components. With a large number of observations in the lowest level clusters, a two-stage estimation approach is used. In the first stage, the normal mixture parameters in each lowest level cluster are estimated using robust methods. Robust alternatives to the maximum likelihood estimation are used to provide stable results even for data with conditional distributions such that their components may not quite meet normality assumptions. Then the lowest level cluster-specific means and standard deviations are modeled in a mixed effects model in the second stage. A small simulation study was conducted to compare performance of finite normal mixture population parameter estimates based on robust and maximum likelihood estimation in stage 1. The proposed modeling approach is illustrated through the analysis of mice tendon fibril diameters data. Analyses results address genotype differences between corresponding components in the mixtures and demonstrate advantages of robust estimation in stage 1.     

Penalized inverse probability weighted estimators for weighted rank regression with missing covariates

Abstract

In this article, we study the variable selection and estimation for linear regression models with missing covariates. The proposed estimation method is almost as efficient as the popular least-squares-based estimation method for normal random errors and empirically shown to be much more efficient and robust with respect to heavy tailed errors or outliers in the responses and covariates. To achieve sparsity, a variable selection procedure based on SCAD is proposed to conduct estimation and variable selection simultaneously. The procedure is shown to possess the oracle property. To deal with the covariates missing, we consider the inverse probability weighted estimators for the linear model when the selection probability is known or unknown. It is shown that the estimator by using estimated selection probability has a smaller asymptotic variance than that with true selection probability, thus is more efficient. Therefore, the important Horvitz-Thompson property is verified for penalized rank estimator with the covariates missing in the linear model. Some numerical examples are provided to demonstrate the performance of the estimators.     

RDELA—a Delaunay-triangulation-based, location and covariance estimator with high breakdown point

We propose an approach that utilizes the Delaunay triangulation to identify a robust/outlier-free subsample. Given that the data structure of the non-outlying points is convex (e.g. of elliptical shape), this subsample can then be used to give a robust estimation of location and scatter (by applying the classical mean and covariance). The estimators derived from our approach are shown to have a high breakdown point. In addition, we provide a diagnostic plot to expand the initial subset in a data-driven way, further increasing the estimators’ efficiency.     

A monte carlo study of robust and least squares response surface methods

Response surface methodology is useful for exploring a response over a region of factor space and in searching for extrema. Its generality, makes it applicable to a variety of areas. Classical response surface methodology for a continuous response variable is generally based on least squares fitting. The sensitivity of least squares to outlying observations carries over to the surface procedures. To overcome this sensitivity, we propose response surface methodology based on robust procedures for continuous response variables. This robust methodology is analogous to the methodology based on least squares, while being much less sensitive to outlying observations. The results of a Monte Carlo study comparing it and classical surface methodologies for normal and contaminated normal errors are presented. The results show that as the proportion of contamination increases, the robust methodology correctly identifies a higher proportion of extrema than the least squares methods and that the robust estimates of extrema tend to be closer to the true extrema than the least squares methods.     

Robust multivariate mixture regression models with incomplete data

Multivariate mixture regression models can be used to investigate the relationships between two or more response variables and a set of predictor variables by taking into consideration unobserved population heterogeneity. It is common to take multivariate normal distributions as mixing components, but this mixing model is sensitive to heavy-tailed errors and outliers. Although normal mixture models can approximate any distribution in principle, the number of components needed to account for heavy-tailed distributions can be very large. Mixture regression models based on the multivariate t distributions can be considered as a robust alternative approach. Missing data are inevitable in many situations and parameter estimates could be biased if the missing values are not handled properly. In this paper, we propose a multivariate t mixture regression model with missing information to model heterogeneity in regression function in the presence of outliers and missing values. Along with the robust parameter estimation, our proposed method can be used for (i) visualization of the partial correlation between response variables across latent classes and heterogeneous regressions, and (ii) outlier detection and robust clustering even under the presence of missing values. We also propose a multivariate t mixture regression model using MM-estimation with missing information that is robust to high-leverage outliers. The proposed methodologies are illustrated through simulation studies and real data analysis.