Deletion diagnostics for marginal mean and correlation model parameters in estimating equations

Regression diagnostics are introduced for parameters in marginal association models for clustered binary outcomes in an implementation of generalized estimating equations. Estimating equations for intracluster correlations facilitate computational formulae for one-step deletion diagnostics in an extension of earlier work on diagnostics for parameters in the marginal mean model. The proposed diagnostics measure the influence of an observation or a cluster of observations on the estimated regression parameters and on the overall fit of the model. The diagnostics are applied to data from four research studies from public health and medicine.     

Leverage and Influence Diagnostics for Spatial Point Processes

Abstract. For a spatial point process model fitted to spatial point pattern data, we develop diagnostics for model validation, analogous to the classical measures of leverage and influence in a generalized linear model. The diagnostics can be characterized as derivatives of basic functionals of the model. They can also be derived heuristically (and computed in practice) as the limits of classical diagnostics under increasingly fine discretizations of the spatial domain. We apply the diagnostics to two example datasets where there are concerns about model validity.     

Local influence in binary regression models,and its correspondence with global influence

We develop local influence diagnostics for a general binary regression model,and apply these methods to case-weight perturbations in four examples. In addition, we illustrate the correspondence between case-deletion diagnostics and local case-weight perturbation slopes and curvatures. We demonstrate that local influence diagnostics can provide a more computationally efficient means for obtaining analogous information to that yielded by case-deletion diagnostics, which can be thought of as global influence perturbations. We also assess the global consistency of patterns of local influence using these data examples.     

Efficient Computing of Regression Diagnostics

Multiple regression diagnostic methods have recently been developed to help data analysts identify failures of data to adhere to the assumptions that customarily accompany regression models. However, the mathematical development of regression diagnostics has not generally led to efficient computing formulas. Conflicting terminology and the use of closely related but subtly different statistics has caused confusion. This article attempts to make regression diagnostics more readily available to those who compute regressions with packaged statistics programs. We review regression diagnostic methodology, highlighting ambiguities of terminology and relationships among similar methods. We present new formulas for efficient computing of regression diagnostics. Finally, we offer specific advice on obtaining regression diagnostics from existing statistics programs, with examples drawn from Minitab and SAS.     

A useful approach to identify the multicollinearity in the presence of outliers

The presence of outliers in the data sets affects the structure of multicollinearity which arises from a high degree of correlation between explanatory variables in a linear regression analysis. This affect could be seen as an increase or decrease in the diagnostics used to determine multicollinearity. Thus, the cases of outliers reduce the reliability of diagnostics such as variance inflation factors, condition numbers and variance decomposition proportions. In this study, we propose to use a robust estimation of the correlation matrix obtained by the minimum covariance determinant method to determine the diagnostics of multicollinearity in the presence of outliers. As a result, the present paper demonstrates that the diagnostics of multicollinearity obtained by the robust estimation of the correlation matrix are more reliable in the presence of outliers.     

Improved approximation for transformation diagnostics

The suitability of a normal linear regression model may require transformation of the original response, and transformation diagnostics are designed to detect the need for such transformation. A common approach to transformation diagnostics is to construct an artificial explanatory variable, which is then tested in the augmented linear regression model for the original response. This paper describes corresponding diagnostics based directly on score statistics with accurate approximations for their standard errors. Several transformation models are covered. Some numerical illustrations are given.     

DELETE-2 AND DELETE-3 JACKKNIFE PROCEDURES FOR UNMASKING IN REGRESSION

Single-case deletion regression diagnostics have been used widely to discover unusual data points, but such approaches can fail in the presence of multiple unusual data points and as a result of masking. We propose a new approach to the use of single-case deletion diagnostics that involves applying these diagnostics to delete-2 and delete-3 jackknife replicates of the data, and considering the percentage of times among these replicates that points are flagged as unusual as an indicator of their influence. By considering replicates that exclude certain collections of points, subtle masking effects can be uncovered.     

Categorical data regression diagnostics for remote access servers

Owing to the growing concerns over data confidentiality, many national statistical agencies are considering remote access servers to disseminate data to the public. With remote servers, users submit requests for output from statistical models fit using the collected data, but they are not allowed access to the data. Remote servers also should enable users to check the fit of their models; however, standard diagnostics like residuals or influence statistics can disclose individual data values. In this article, we present diagnostics for categorical data regressions that can be safely and usefully employed in remote servers. We illustrate the diagnostics with simulation studies.     

Why Not Try a Robust Regression?

We introduce and discuss three important regression diagnostics: leverage, Studentized residuals, and DFFITS. We then develop two approaches to bounded-influence robust regression based on these diagnostics. The methods are illustrated on a data set using a simple MINITAB program.     

Influence diagnostics in a vector autoregressive model

In this paper, we use a likelihood approach and the local influence method introduced by Cook [Assessment of local influence (with discussion). J Roy Statist Soc Ser B. 1986;48:133–149] to study a vector autoregressive (VAR) model. We present the maximum likelihood estimators and the information matrix. We establish the normal curvature and slope diagnostics for the VAR model under several perturbation schemes and use the Monte Carlo method to obtain benchmark values for determining the influence of directional diagnostics and possible influential observations. An empirical study using the VAR model to fit real data of monthly returns of IBM and S&P500 index illustrates the effectiveness of our proposed diagnostics.     

Robust multivariate diagnostics for PLSR and application on high dimensional spectrally overlapped drug systems

ABSTRACT

Statistical methods are effectively used in the evaluation of pharmaceutical formulations instead of laborious liquid chromatography. However, signal overlapping, nonlinearity, multicollinearity and presence of outliers deteriorate the performance of statistical methods. The Partial Least Squares Regression (PLSR) is a very popular method in the quantification of high dimensional spectrally overlapped drug formulations. The SIMPLS is the mostly used PLSR algorithm, but it is highly sensitive to outliers that also effect the diagnostics. In this paper, we propose new robust multivariate diagnostics to identify outliers, influential observations and points causing non-normality for a PLSR model. We study performances of the proposed diagnostics on two everyday use highly overlapping drug systems: Paracetamol–Caffeine and Doxylamine Succinate–Pyridoxine Hydrochloride.     

Model Diagnostics for Remote Access Regression Servers

To protect public-use microdata, one approach is not to allow users access to the microdata. Instead, users submit analyses to a remote computer that reports back basic output from the fitted model, such as coefficients and standard errors. To be most useful, this remote server also should provide some way for users to check the fit of their models, without disclosing actual data values. This paper discusses regression diagnostics for remote servers. The proposal is to release synthetic diagnostics—i.e. simulated values of residuals and dependent and independent variables–constructed to mimic the relationships among the real-data residuals and independent variables. Using simulations, it is shown that the proposed synthetic diagnostics can reveal model inadequacies without substantial increase in the risk of disclosures. This approach also can be used to develop remote server diagnostics for generalized linear models.     

Influence Diagnostics for Simultaneous Equations Models

This paper presents influence diagnostics for simultaneous equations models. It proposes residuals, leverage and other influence measures. A missing data method is adopted to minimize the masking effect due to case deletions. The assessment of local influence is also considered. The paper shows how to evaluate the effects that perturbations to the endogenous variables, predetermined variables and case weights may have on the parameter estimates. The diagnostics are illustrated with two examples.     

Influence Diagnostics and Estimation Algorithms for Powell's SCLS

This article studies influence diagnostics and estimation algorithms for Powell's symmetrically censored least squares estimator. The proposed measures of influence are based on one-step approximations to the analogous deletion diagnostics used in least squares regression and can be conveniently constructed using a Newton-type algorithm. Additionally, it is found that this algorithm can be used to substantially reduce the computational burden of the estimator. The results of the article are illustrated with an application.     

Model diagnostics for smoothing spline ANOVA models

The author proposes some simple diagnostics for assessing the necessity of selected terms in smoothing spline ANOVA models. The elimination of practically insignificant terms generally enhances the interpretability of the estimates and sometimes may also have inferential implications. The diagnostics are derived from Kullback‐Leibler geometry and are illustrated in the settings of regression, probability density estimation, and hazard rate estimation.     

Regression diagnostics for removing an observation with animating graphics

Recently, Cook and Weisberg (1989) presented dynamic graphics for regression diagnostics. They suggested animating graphics which could aid to understanding the effects of adding a variable to a model. In this paper, using the Cook and Weisberg's idea of animation, we propose a dynamic graphical method for residuals to display the effects of removing an observation from a model. Based on the information obtained from these animating graphics, it is possible to see the influence of observations for regression diagnostics.     

Influence Measures Based on Cressie-Read Divergence Measures in Multivariate Linear Model

We define a new family of influence measures based on the divergence measures, in the multivariate general linear model. Influence measures are obtained by quantifying the divergence between the sample distribution of an estimate obtained with all the observations and the sample distribution of the same estimate obtained without any observation. This approach is applied to best linear unbiased estimates of estimable functions. Therefore, these diagnostics can be applied to every statistical multivariate technique that can be formulated like this kind of model. Some examples are considered to clarify the applicability of the introduced diagnostics.     

Some algebra and geometry for hierarchical models, applied to diagnostics

Recent advances in computing make it practical to use complex hierarchical models. However, the complexity makes it difficult to see how features of the data determine the fitted model. This paper describes an approach to diagnostics for hierarchical models, specifically linear hierarchical models with additive normal or t -errors. The key is to express hierarchical models in the form of ordinary linear models by adding artificial `cases' to the data set corresponding to the higher levels of the hierarchy. The error term of this linear model is not homoscedastic, but its covariance structure is much simpler than that usually used in variance component or random effects models. The re-expression has several advantages. First, it is extremely general, covering dynamic linear models, random effect and mixed effect models, and pairwise difference models, among others. Second, it makes more explicit the geometry of hierarchical models, by analogy with the geometry of linear models. Third, the analogy with linear models provides a rich source of ideas for diagnostics for all the parts of hierarchical models. This paper gives diagnostics to examine candidate added variables, transformations, collinearity, case influence and residuals.     

Residuals for the linear model with general covariance structure

A general theory is presented for residuals from the general linear model with correlated errors. It is demonstrated that there are two fundamental types of residual associated with this model, referred to here as the marginal and the conditional residual. These measure respectively the distance to the global aspects of the model as represented by the expected value and the local aspects as represented by the conditional expected value. These residuals may be multivariate. Some important dualities are developed which have simple implications for diagnostics. The results are illustrated by reference to model diagnostics in time series and in classical multivariate analysis with independent cases.