Residuals and leverages in the linear mixed measurement error models

Residuals are frequently used to evaluate the validity of the assumptions of statistical models and may also be employed as tools for model selection. In this paper, we consider residuals and their limiting properties in the linear mixed measurement error models. Also, we develop types of residuals for these models and then review some of the residual analysis techniques. Further, by using the definition of generalized leverage, we derive generalized leverage matrices for identification of high-leverage points for these models. Finally, we analyse a real data set.     

Outlier detection and accommodation in general spatial models

This paper studies outlier detection and accommodation in general spatial models including spatial autoregressive models and spatial error model as special cases. Using mean-shift and variance-weight models respectively, test statistics for multiple outliers are derived and the detecting procedures are proposed. In addition, several key diagnostic measures such as standardized residuals and leverage measure are defined in general spatial models. Outlier modified models are proposed to accommodate outliers in the data set. The performance of test statistics, including size and power, are examined via simulation studies. Three real examples are analyzed and the results show that the proposed methodology is useful for identifying and accommodating outliers in general spatial models.     

Properties of Added Variable Plots in Cox's Regression Model

The added variable plot is useful for examining the effect of a covariate in regression models. The plot provides information regarding the inclusion of a covariate, and is useful in identifying influential observations on the parameter estimates. Hall et al. (1996) proposed a plot for Cox's proportional hazards model derived by regarding the Cox model as a generalized linear model. This paper proves and discusses properties of this plot. These properties make the plot a valuable tool in model evaluation. Quantities considered include parameter estimates, residuals, leverage, case influence measures and correspondence to previously proposed residuals and diagnostics.     

Residuals from deletion in added variable plots

An added variable plot is a commonly used plot in regression diagnostics. The rationale for this plot is to provide information about the addition of a further explanatory variable to the model. In addition, an added variable plot is most often used for detecting high leverage points and influential data. So far as we know, this type of plot involves the least squares residuals which, we suspect, could produce a confusing picture when a group of unusual cases are present in the data. In this situation, added variable plots may not only fail to detect the unusual cases but also may fail to focus on the need for adding a further regressor to the model. We suggest that residuals from deletion should be more convincing and reliable in this type of plot. The usefulness of an added variable plot based on residuals from deletion is investigated through a few examples and a Monte Carlo simulation experiment in a variety of situations.     

Local Influence Under Parameter Constraints

Calculations of local influence curvatures and leverage have been well developed when the parameters are unrestricted. In this article, we discuss the assessment of local influence and leverage under linear equality parameter constraints with extensions to inequality constraints. Using a penalized quadratic function we express the normal curvature of local influence for arbitrary perturbation schemes and the generalized leverage matrix in interpretable forms, which depend on restricted and unrestricted components. The results are quite general and can be applied in various statistical models. In particular, we derive the normal curvature under three useful perturbation schemes for generalized linear models. Four illustrative examples are analyzed by the methodology developed in the article.     

Efficient Estimation and Robust Inference of Linear Regression Models in the Presence of Heteroscedastic Errors and High Leverage Points

It is common for linear regression models that the error variances are not the same for all observations and there are some high leverage data points. In such situations, the available literature advocates the use of heteroscedasticity consistent covariance matrix estimators (HCCME) for the testing of regression coefficients. Primarily, such estimators are based on the residuals derived from the ordinary least squares (OLS) estimator that itself can be seriously inefficient in the presence of heteroscedasticity. To get efficient estimation, many efficient estimators, namely the adaptive estimators are available but their performance has not been evaluated yet when the problem of heteroscedasticity is accompanied with the presence of high leverage data. In this article, the presence of high leverage data is taken into account to evaluate the performance of the adaptive estimator in terms of efficiency. Furthermore, our numerical work also evaluates the performance of the robust standard errors based on this efficient estimator in terms of interval estimation and null rejection rate (NRR).     

Influence Diagnostics on Testing Linear Hypothesis in Linear Models with Correlated Errors

To assess the influence of observations on the parameter estimates, case deletion diagnostics are commonly used in linear regression models. For linear models with correlated errors we study the influence of observations on testing a linear hypothesis using single and multiple case deletions. The change in likelihood ratio test and F test theoretically is derived and it is shown these tests to be completely determined by two proposed generalized externally studentized residuals. An illustrative example of a real data set is also reported.     

Deletion residuals in the detection of heterogeneity of variances in linear regression

The heterogeneity of error variance often causes a huge interpretive problem in linear regression analysis. Before taking any remedial measures we first need to detect this problem. A large number of diagnostic plots are now available in the literature for detecting heteroscedasticity of error variances. Among them the ‘residuals’ and ‘fits’ (R–F) plot is very popular and commonly used. In the R–F plot residuals are plotted against the fitted responses, where both these components are obtained using the ordinary least squares (OLS) method. It is now evident that the OLS fits and residuals suffer a huge setback in the presence of unusual observations and hence the R–F plot may not exhibit the real scenario. The deletion residuals based on a data set free from all unusual cases should estimate the true errors in a better way than the OLS residuals. In this paper we propose ‘deletion residuals’ and the ‘deletion fits’ (DR–DF) plot for the detection of the heterogeneity of error variances in a linear regression model to get a more convincing and reliable graphical display. Examples show that this plot locates unusual observations more clearly than the R–F plot. The advantage of using deletion residuals in the detection of heteroscedasticity of error variance is investigated through Monte Carlo simulations under a variety of situations.     

Detection of outliers in simple circular regression models using the mean circular error statistic

The investigation on the identification of outliers in linear regression models can be extended to those for circular regression case. In this paper, we propose a new numerical statistic called mean circular error to identify possible outliers in circular regression models by using a row deletion approach. Through intensive simulation studies, the cut-off points of the statistic are obtained and its power of performance investigated. It is found that the performance improves as the concentration parameter of circular residuals becomes larger or the sample size becomes smaller. As an illustration, the statistic is applied to a wind direction data set.     

Leverage,influence and residuals in regression models when observations are correlated

This paper considers the concepts of leverage and influence in the linear regression model with correlated errors when the error covariance structure is completely specified. Generalizations of the usual measures are given. Extensions of residuals also naturally arise. The theory is illustrated using two examples     

Pseudo latent models: Goodness of fit measures, residuals, estimation, testing, and simulation

Binary response models consider pseudo-R ² measures which are not based on residuals while several concepts of residuals were developed for tests. In this paper the endogenous variable of the latent model corresponding to the binary observable model is substituted by a pseudo variable. Then goodness of fit measures and tests can be based on a joint concept of residuals as for linear models. Different kinds of residuals based on probit ML estimates are employed. The analytical investigations and the simulation results lead to the recommendation to use standardized residuals where there is no difference between observed and generalized residuals. In none of the investigated situations this estimator is far away from the best result. While in large samples all considered estimators are very similar, small sample properties speak in favour of residuals which are modifications of those suggested in the literature. An empirical application demonstrates that it is not necessary to develop new testing procedures for the observable models with dichotomous regressands. Well-know approaches for linear models with continuous endogenous variables which are implemented in usual econometric packages can be used for pseudo latent models. An erratum to this article is available at .     

Residual analysis for spatial point processes (with discussion)

Summary.  We define residuals for point process models fitted to spatial point pattern data, and we propose diagnostic plots based on them. The residuals apply to any point process model that has a conditional intensity; the model may exhibit spatial heterogeneity, interpoint interaction and dependence on spatial covariates. Some existing ad hoc methods for model checking (quadrat counts, scan statistic, kernel smoothed intensity and Berman's diagnostic) are recovered as special cases. Diagnostic tools are developed systematically, by using an analogy between our spatial residuals and the usual residuals for (non-spatial) generalized linear models. The conditional intensity λ plays the role of the mean response. This makes it possible to adapt existing knowledge about model validation for generalized linear models to the spatial point process context, giving recommendations for diagnostic plots. A plot of smoothed residuals against spatial location, or against a spatial covariate, is effective in diagnosing spatial trend or co-variate effects. Q – Q -plots of the residuals are effective in diagnosing interpoint interaction.     

Deviance residuals in generalised log-gamma regression models with censored observations

In this article, we compare three residuals based on the deviance component in generalised log-gamma regression models with censored observations. For different parameter settings, sample sizes and censoring percentages, various simulation studies are performed and the empirical distribution of each residual is displayed and compared with the standard normal distribution. For all cases studied, the empirical distributions of the proposed residuals are in general symmetric around zero, but only a martingale-type residual presented negligible kurtosis for the majority of the cases studied. These studies suggest that the residual analysis usually performed in normal linear regression models can be straightforwardly extended for the martingale-type residual in generalised log-gamma regression models with censored data. A lifetime data set is analysed under log-gamma regression models and a model checking based on the martingale-type residual is performed.     

Recursive residuals in generalised linear models

Recursive estimation and recursive residuals are introduced for generalised linear models (GLIM). Their definitions parallel those of normal theory regression models and relate to one of the outlier model definitions of GLIM residuals. An example illustrates their use.     

Truncated location-scale non linear regression models

We present a class of truncated non linear regression models for location and scale where the truncated nature of the data is incorporated into the statistical model by assuming that the response variable follows a truncated distribution. The location parameter of the response variable is assumed to be modeled by a continuous non linear function of covariates and unknown parameters. In addition, the proposed model also allows for the scale parameter of the responses to be characterized by a continuous function of the covariates and unknown parameters. Three particular cases of the proposed models are presented by considering the response variable to follow a truncated normal, truncated skew normal, and truncated beta distribution. These truncated non linear regression models are constructed assuming fixed known truncation limits and model parameters are estimated by direct maximization of the log-likelihood using a non linear optimization algorithm. Standardized residuals and diagnostic metrics based on the cases deletion are considered to verify the adequacy of the model and to detect outliers and influential observations. Results based on simulated data are presented to assess the frequentist properties of estimates, and a real data set on soil-water retention from the Buriti Vermelho River Basin database is analyzed using the proposed methodology.     

Influence Diagnostics for Simultaneous Equations Models

This paper presents influence diagnostics for simultaneous equations models. It proposes residuals, leverage and other influence measures. A missing data method is adopted to minimize the masking effect due to case deletions. The assessment of local influence is also considered. The paper shows how to evaluate the effects that perturbations to the endogenous variables, predetermined variables and case weights may have on the parameter estimates. The diagnostics are illustrated with two examples.     

Generalized Leverage and its Applications

The generalized leverage of an estimator is defined in regression models as a measure of the importance of individual observations. We derive a simple but powerful result, developing an explicit expression for leverage in a general M -estimation problem, of which the maximum likelihood problems are special cases. A variety of applications are considered, most notably to the exponential family non-linear models. The relationship between leverage and local influence is also discussed. Numerical examples are given to illustrate our results     

Diagnostic checks for discrete data regression models using posterior predictive simulations

Model checking with discrete data regressions can be difficult because the usual methods such as residual plots have complicated reference distributions that depend on the parameters in the model. Posterior predictive checks have been proposed as a Bayesian way to average the results of goodness-of-fit tests in the presence of uncertainty in estimation of the parameters. We try this approach using a variety of discrepancy variables for generalized linear models fitted to a historical data set on behavioural learning. We then discuss the general applicability of our findings in the context of a recent applied example on which we have worked. We find that the following discrepancy variables work well, in the sense of being easy to interpret and sensitive to important model failures: structured displays of the entire data set, general discrepancy variables based on plots of binned or smoothed residuals versus predictors and specific discrepancy variables created on the basis of the particular concerns arising in an application. Plots of binned residuals are especially easy to use because their predictive distributions under the model are sufficiently simple that model checks can often be made implicitly. The following discrepancy variables did not work well: scatterplots of latent residuals defined from an underlying continuous model and quantile–quantile plots of these residuals.     

Influence on tests with focus on linear models

To assess the influence of single observations on the parameter estimates, case-deletion diagnostics are commonly used in linear regression models; one example is Cook's distance. For nested parametric models we consider a deletion diagnostic for evaluating the influence of a single observation on the likelihood ratio (LR) test. In order to have a common scale as reference, the asymptotic distribution of the diagnostic is derived and the values of the diagnostic are converted to percentiles. We focus on linear models and general linear models, and in these cases explicit results are derived. The performance of the diagnostic is explored in two small bench mark examples from linear regression and in a larger linear mixed model example.     

Influence and residuals in restricte generalized linear models

A particular influence measure for restricted regression models is reviewed in this paper. We give em- phasis on establishing regularity conditions to apply the proposed influence measure in restricted gen- eralized linear models. The development of conditional residuals is also discussed. In particular, a sim- ulation study was conducted in order to compare the distributions of the proposed residuals for various generalized linear models. Finally, an application is given.