Influence diagnostics in gamma ridge regression model

In this article, we proposed some influence diagnostics for the gamma regression model (GRM) and the gamma ridge regression model (GRRM). We assess the impact of influential observations on the GRM and GRRM estimates by extending the work of Pregibon [Logistic regression diagnostics. Ann Stat. 1981;9:705–724] and Walker and Birch [Influence measures in ridge regression. Technometrics. 1988;30:221–227]. Comparison of both models is made and demonstrated with the help of a simulation study and a real data set. We report some momentous results in detecting the influential observations and their effects on the GRM and GRRM estimates.     

Some properties of,and relationships among,several uncorrelated and homoscedastic residual vectors

In this paper we examine the properties of four types of residual vectors, arising from fitting a linear regression model to a set of data by least squares. The four types of residuals are (i) the Stepwise residuals (Hedayat and Robson, 1970), (ii) the Recursive residuals (Brown, Durbin, and Evans, 1975), (iii) the Sequentially Adjusted residuals (to be defined herein), and (iv) the BLUS residuals (Theil, 1965, 1971). We also study the relationships among the four residual vectors. It is found that, for any given sequence of observations, (i) the first three sets of residuals are identical, (ii) each of the first three sets, being identical, is a member of Thei’rs (1965, 1971) family of residuals; specifically, they are Linear Unbiased with a Scalar covariance matrix (LUS) but not Best Linear Unbiased with a Scalar covariance matrix (BLUS). We find the explicit form of the transformation matrix and show that the first three sets of residual vectors can be written as an orthogonal transformation of the BLUS residual vector. These and other properties may prove to be useful in the statistical analysis of residuals.     

Simple correspondence analysis using adjusted residuals

Correspondence analysis is a versatile statistical technique that allows the user to graphically identify the association that may exist between variables of a contingency table. For two categorical variables, the classical approach involves applying singular value decomposition to the Pearson residuals of the table. These residuals allow for one to use a simple test to determine those cells that deviate from what is expected under independence. However, the assumptions concerning these residuals are not always satisfied and so such results can lead to questionable conclusions.One may consider instead, an adjustment of the Pearson residual, which is known to have properties associated with the standard normal distribution. This paper explores the application of these adjusted residuals to correspondence analysis and determines how they impact upon the configuration of points in the graphical display.     

Bootstrapping regression models with BLUS residuals

To bootstrap a regression problem, pairs of response and explanatory variables or residuals can be resam‐pled, according to whether we believe that the explanatory variables are random or fixed. In the latter case, different residuals have been proposed in the literature, including the ordinary residuals (Efron 1979), standardized residuals (Bickel & Freedman 1983) and Studentized residuals (Weber 1984). Freedman (1981) has shown that the bootstrap from ordinary residuals is asymptotically valid when the number of cases increases and the number of variables is fixed. Bickel & Freedman (1983) have shown the asymptotic validity for ordinary residuals when the number of variables and the number of cases both increase, provided that the ratio of the two converges to zero at an appropriate rate. In this paper, the authors introduce the use of BLUS (Best Linear Unbiased with Scalar covariance matrix) residuals in bootstrapping regression models. The main advantage of the BLUS residuals, introduced in Theil (1965), is that they are uncorrelated. The main disadvantage is that only n —p residuals can be computed for a regression problem with n cases and p variables. The asymptotic results of Freedman (1981) and Bickel & Freedman (1983) for the ordinary (and standardized) residuals are generalized to the BLUS residuals. A small simulation study shows that even though only n — p residuals are available, in small samples bootstrapping BLUS residuals can be as good as, and sometimes better than, bootstrapping from standardized or Studentized residuals.     

GEEs for repeated categorical responses based on generalized residuals

Clustered or correlated samples of categorical response data arise frequently in many fields of application. The method of generalized estimating equations (GEEs) introduced in Liang and Zeger [Longitudinal data analysis using generalized linear models, Biometrika 73 (1986), pp. 13–22] is often used to analyse this type of data. GEEs give consistent estimates of the regression parameters and their variance based upon the Pearson residuals. Park et al. [Alternative GEE estimation procedures for discrete longitudinal data, Comput. Stat. Data Anal. 28 (1998), pp. 243–256] considered a modification of the GEE approach using the Anscombe residual and the deviance residual. In this work, we propose to extend this idea to a family of generalized residuals. A wide simulation study is conducted for binary and Poisson correlated outcomes and also two numerical illustrations are presented.     

Theory & Methods: Residual Diagnostic Plots for Checking for Model Mis‐specification in Time Series Regression

This paper considers residuals for time series regression. Despite much literature on visual diagnostics for uncorrelated data, there is little on the autocorrelated case. To examine various aspects of the fitted time series regression model, three residuals are considered. The fitted regression model can be checked using orthogonal residuals; the time series error model can be analysed using marginal residuals; and the white noise error component can be tested using conditional residuals. When used together, these residuals allow identification of outliers, model mis‐specification and mean shifts. Due to the sensitivity of conditional residuals to model mis‐specification, it is suggested that the orthogonal and marginal residuals be examined first.     

Bias in Multinomial Models

This paper investigates bias in parameter estimates and residual diagnostics for parametric multinomial models by considering the effect of deleting a cell. In particular, it describes the average changes in the standardized residuals and maximum likelihood estimates resulting from conditioning on the given cells. These changes suggest how individual cell observations affect biases. Emphasis is placed on the role of individual cell observations in determining bias and on how bias affects standard diagnostic methods. Examples from genetics and log–linear models are considered. Numerical results show that conditioning on an influential cell results in substantial changes in biases.     

Identification of Multiple Outliers in Logistic Regression

The use of logistic regression modeling has seen a great deal of attention in the literature in recent years. This includes all aspects of the logistic regression model including the identification of outliers. A variety of methods for the identification of outliers, such as the standardized Pearson residuals, are now available in the literature. These methods, however, are successful only if the data contain a single outlier. In the presence of multiple outliers in the data, which is often the case in practice, these methods fail to detect the outliers. This is due to the well-known problems of masking (false negative) and swamping (false positive) effects. In this article, we propose a new method for the identification of multiple outliers in logistic regression. We develop a generalized version of standardized Pearson residuals based on group deletion and then propose a technique for identifying multiple outliers. The performance of the proposed method is then investigated through several examples.     

A log-linear regression model for the odd Weibull distribution with censored data

We introduce the log-odd Weibull regression model based on the odd Weibull distribution (Cooray, 2006). We derive some mathematical properties of the log-transformed distribution. The new regression model represents a parametric family of models that includes as sub-models some widely known regression models that can be applied to censored survival data. We employ a frequentist analysis and a parametric bootstrap for the parameters of the proposed model. We derive the appropriate matrices for assessing local influence on the parameter estimates under different perturbation schemes and present some ways to assess global influence. Further, for different parameter settings, sample sizes and censoring percentages, some simulations are performed. In addition, the empirical distribution of some modified residuals are given and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be extended to a modified deviance residual in the proposed regression model applied to censored data. We define martingale and deviance residuals to check the model assumptions. The extended regression model is very useful for the analysis of real data.     

Estimation in regression models with stationary,dependent errors

Often the unknown covariance structure of a stationary, dependent, Gaussian error sequence can be simply parametrised. The error sequence can either be directly observed or observed only through a random sequence containing a deterministic regression model. The method of scoring is used here, in conjunction with recursive estimation techniques, to effect the maximum likelihood estimation of the covariance parameters. Sequences of recursive residuals, useful in model diagnostics and data analysis, are obtained in the estimation procedure.     

An alternative method correcting BDR type of heteroskedasticity by the weighting re-estimated absolute residuals

The aim of this paper is to introduce a new method which corrects residual variances for the butterfly distributed residuals (BDR). Distribution theory, confidence intervals, and tests of hypotheses are valid and meaningful only if the standard regression assumptions are satisfied. Heteroskedasticity is one of the violations of these assumptions and BDR is another type of heteroskedasticity. This study reveals an alternative approach to correct the BDR type of heteroskedasticity by the weighting re-estimated absolute residuals (WRAR). After giving brief information about heteroskedasticity and BDR type of heteroskedasticity, WRAR is introduced. WRAR and the usual variance stabilizing techniques are compared on multiple and simple regression models.     

Influence Diagnostics for Simultaneous Equations Models

This paper presents influence diagnostics for simultaneous equations models. It proposes residuals, leverage and other influence measures. A missing data method is adopted to minimize the masking effect due to case deletions. The assessment of local influence is also considered. The paper shows how to evaluate the effects that perturbations to the endogenous variables, predetermined variables and case weights may have on the parameter estimates. The diagnostics are illustrated with two examples.     

Diagnostic Measures for Generalized Linear Models with Missing Covariates

Abstract.  In this paper, we carry out an in-depth investigation of diagnostic measures for assessing the influence of observations and model misspecification in the presence of missing covariate data for generalized linear models. Our diagnostic measures include case-deletion measures and conditional residuals. We use the conditional residuals to construct goodness-of-fit statistics for testing possible misspecifications in model assumptions, including the sampling distribution. We develop specific strategies for incorporating missing data into goodness-of-fit statistics in order to increase the power of detecting model misspecification. A resampling method is proposed to approximate the p -value of the goodness-of-fit statistics. Simulation studies are conducted to evaluate our methods and a real data set is analysed to illustrate the use of our various diagnostic measures.     

Comments on “bootstrapping time series models”

We take issue with the main suggestion in Li and Maddala (LiMa) that bootstrapping residuals is always the preferred approach and question some of their guidelines. We show that it can be potentially misleading to mimic the autocovariance structure of residuals, since it can be very different from that of true errors. We emphasize that the residuals are sensitive to model misspecification and generally not a part of the information set. We make constructive suggestions and propose a semiparametric method.     

Monitoring Structural Changes in Generalized Linear Models

In this article, we introduce two monitoring schemes to (sequentially) detect structural changes in generalized linear models and develop asymptotic theories for them. The first method is based on cumulative sums (CUSUM) of weighted residuals, in which the unknown in-control parameters have been replaced by its maximum likelihood (ML) estimate from the training sample, whereas the second scheme makes use of moving sums (MOSUM) of weighted residuals. We characterize the limit distribution of the test statistic and show that these tests are consistent. Moreover, we also obtain and tabulate the asymptotic critical values of the tests. Finally, we study the speed of detection under different conditions. The methods are illustrated and compared in several simulations.     

Properties of Added Variable Plots in Cox's Regression Model

The added variable plot is useful for examining the effect of a covariate in regression models. The plot provides information regarding the inclusion of a covariate, and is useful in identifying influential observations on the parameter estimates. Hall et al. (1996) proposed a plot for Cox's proportional hazards model derived by regarding the Cox model as a generalized linear model. This paper proves and discusses properties of this plot. These properties make the plot a valuable tool in model evaluation. Quantities considered include parameter estimates, residuals, leverage, case influence measures and correspondence to previously proposed residuals and diagnostics.     

Regression diagnostics for removing an observation with animating graphics

Recently, Cook and Weisberg (1989) presented dynamic graphics for regression diagnostics. They suggested animating graphics which could aid to understanding the effects of adding a variable to a model. In this paper, using the Cook and Weisberg's idea of animation, we propose a dynamic graphical method for residuals to display the effects of removing an observation from a model. Based on the information obtained from these animating graphics, it is possible to see the influence of observations for regression diagnostics.     

Deviance Residuals for an Angular Response

This paper discusses deviance residual approximations in von Mises regression models. By using a relationship between the von Mises and the wrapped normal distributions, the paper shows that the deviance component of the von Mises distribution is approximately a linear function of the standard normal distribution. Two standardized forms are proposed for the deviance residual, and a simulation study is performed to compare the approximation of the proposed residuals to the standard normal distribution. An illustrative example is given.     

A scale-location adjustment for proportional hazards deviance residuals

We show that deviance residuals derived using the proportional hazards assumption (including Cox regression) are not asymptotically standard normal, but that a scale-location adjustment makes them nearly standard normal, even for moderate sample sizes. This adjustment should aid in outlier detection, as it allows a more exact assessment of when a deviance residual is unusually large.     

Standardized Martingale Residuals Applied to Grouped Left Truncated Observations of Dementia Cases

The use of martingale residuals have been proposed for modelchecking and also to get a non-parametric estimate of the effectof an explanatory variable. We apply this approach to an epidemiologicalproblem which presents two characteristics: the data are lefttruncated due to delayed entry in the cohort; the data are groupedinto geographical units (parishes). This grouping suggests anatural way of smoothing the graph of residuals which is to computethe sum of the residuals for each parish. It is also naturalto present a graph with standardized residuals. We derive thevariances of the estimated residuals for left truncated datawhich allows computing the standardized residuals. This methodis applied to the study of dementia in a cohort of old people,and to the possible effect of the concentration of aluminum andsilica in drinking water on the risk of developing dementia.