Diagnostics for a Linear Model with First-Order Autoregressive Symmetrical Errors

In this article, we focus on some diagnostics for linear regression model with first-order autoregressive and symmetrical errors. The symmetrical class includes both light- and heavy-tailed univariate symmetrical distributions, which offers a more flexible framework for modeling. Maximum likelihood estimates are computed via the Fisher-score method. Score statistic and its adjustment are proposed for testing autocorrelation of the random errors. Local influence diagnostics are also derived for the model under some usual perturbation schemes. The performances of the test statistics are investigated through Monte Carlo simulations. Finally, a real data set is used to illustrate our diagnostic methods.     

Inference after separated hypotheses testing: an empirical investigation for linear models

Model selection problems arise while constructing unbiased or asymptotically unbiased estimators of measures known as discrepancies to find the best model. Most of the usual criteria are based on goodness-of-fit and parsimony. They aim to maximize a transformed version of likelihood. For linear regression models with normally distributed error, the situation is less clear when two models are equivalent: are they close to or far from the unknown true model? In this work, based on stochastic simulation and parametric simulation, we study the results of Vuong's test, Cox's test, Akaike's information criterion, Bayesian information criterion, Kullback information criterion and bias corrected Kullback information criterion and the ability of these tests to discriminate between non-nested linear models.     

Diagnostic tools in generalized Weibull linear regression models

We propose some statistical tools for diagnosing the class of generalized Weibull linear regression models [A.A. Prudente and G.M. Cordeiro, Generalized Weibull linear models, Comm. Statist. Theory Methods 39 (2010), pp. 3739–3755]. This class of models is an alternative means of analysing positive, continuous and skewed data and, due to its statistical properties, is very competitive with gamma regression models. First, we show that the Weibull model induces ma-ximum likelihood estimators asymptotically more efficient than the gamma model. Standardized residuals are defined, and their statistical properties are examined empirically. Some measures are derived based on the case-deletion model, including the generalized Cook's distance and measures for identifying influential observations on partial F-tests. The results of a simulation study conducted to assess behaviour of the global influence approach are also presented. Further, we perform a local influence analysis under the case-weights, response and explanatory variables perturbation schemes. The Weibull, gamma and other Weibull-type regression models are fitted into three data sets to illustrate the proposed diagnostic tools. Statistical analyses indicate that the Weibull model fitted into these data yields better fits than other common alternative models.     

A new look at the difference between the GEE and the GLMM when modeling longitudinal count responses

Poisson log-linear regression is a popular model for count responses. We examine two popular extensions of this model – the generalized estimating equations (GEE) and the generalized linear mixed-effects model (GLMM) – to longitudinal data analysis and complement the existing literature on characterizing the relationship between the two dueling paradigms in this setting. Unlike linear regression, the GEE and the GLMM carry significant conceptual and practical implications when applied to modeling count data. Our findings shed additional light on the differences between the two classes of models when used for count data. Our considerations are demonstrated by both real study and simulated data.     

Thin plate regression splines

Summary. I discuss the production of low rank smoothers for d  ≥ 1 dimensional data, which can be fitted by regression or penalized regression methods. The smoothers are constructed by a simple transformation and truncation of the basis that arises from the solution of the thin plate spline smoothing problem and are optimal in the sense that the truncation is designed to result in the minimum possible perturbation of the thin plate spline smoothing problem given the dimension of the basis used to construct the smoother. By making use of Lanczos iteration the basis change and truncation are computationally efficient. The smoothers allow the use of approximate thin plate spline models with large data sets, avoid the problems that are associated with 'knot placement' that usually complicate modelling with regression splines or penalized regression splines, provide a sensible way of modelling interaction terms in generalized additive models, provide low rank approximations to generalized smoothing spline models, appropriate for use with large data sets, provide a means for incorporating smooth functions of more than one variable into non-linear models and improve the computational efficiency of penalized likelihood models incorporating thin plate splines. Given that the approach produces spline-like models with a sparse basis, it also provides a natural way of incorporating unpenalized spline-like terms in linear and generalized linear models, and these can be treated just like any other model terms from the point of view of model selection, inference and diagnostics.     

A note on Stein's lemma for multivariate elliptical distributions

When two random variables are bivariate normally distributed Stein's original lemma allows to conveniently express the covariance of the first variable with a function of the second. Landsman and Neslehova (2008) extend this seminal result to the family of multivariate elliptical distributions. In this paper we use the technique of conditioning to provide a more elegant proof for their result. In doing so, we also present a new proof for the classical linear regression result that holds for the elliptical family.     

Pearson-type goodness-of-fit tests for regression

A procedure for testing the goodness of fit of linear regression models is introduced. For a given partition of the real line into cells, the proposed test is a quadratic form based on the vector of observed minus expected frequencies of the residuals obtained by maximum-likelihood estimation of the regression parameters. The quadratic form is of the same computational difficulty as the traditional Pearson-type tests with uncensored data. A statistic based on only one cell is particularly easy to apply and is used for testing the normality assumption in a real data set from astronomy. A simulation study examines the finite-sample properties of the proposed tests.     

Some Notes on Refereeing

Analysis of longitudinal data using a general linear mixed model requires the specification of a form for the covariance matrix of within-subject observations. Graphical diagnostics, such as the scatterplot matrix, can be of substantial help in making this specification. I introduce another graphical diagnostic, the Partial-Regression-on-Intervenors Scatterplot Matrix (PRISM), which complements the ordinary scatterplot matrix and which is more useful for identifying certain kinds of correlation structures. PRISMs corresponding to several commonly used correlation structures are displayed. The PRISM's usefulness in model specification is illustrated with an example of longitudinal data from a 100-kilometer road race.     

Inference for general parametric functions in box‐Cox‐type transformation models

The authors propose a simple but general method of inference for a parametric function of the Box‐Cox‐type transformation model. Their approach is built upon the classical normal theory but takes parameter estimation into account. It quickly leads to test statistics and confidence intervals for a linear combination of scaled or unsealed regression coefficients, as well as for the survivor function and marginal effects on the median or other quantité functions of an original response. The authors show through simulations that the finite‐sample performance of their method is often superior to the delta method, and that their approach is robust to mild departures from normality of error distributions. They illustrate their approach with a numerical example.     

Bayesian robust transformation and variable selection: A unified approach

The authors consider the problem of simultaneous transformation and variable selection for linear regression. They propose a fully Bayesian solution to the problem, which allows averaging over all models considered including transformations of the response and predictors. The authors use the Box‐Cox family of transformations to transform the response and each predictor. To deal with the change of scale induced by the transformations, the authors propose to focus on new quantities rather than the estimated regression coefficients. These quantities, referred to as generalized regression coefficients, have a similar interpretation to the usual regression coefficients on the original scale of the data, but do not depend on the transformations. This allows probabilistic statements about the size of the effect associated with each variable, on the original scale of the data. In addition to variable and transformation selection, there is also uncertainty involved in the identification of outliers in regression. Thus, the authors also propose a more robust model to account for such outliers based on a t‐distribution with unknown degrees of freedom. Parameter estimation is carried out using an efficient Markov chain Monte Carlo algorithm, which permits moves around the space of all possible models. Using three real data sets and a simulated study, the authors show that there is considerable uncertainty about variable selection, choice of transformation, and outlier identification, and that there is advantage in dealing with all three simultaneously. The Canadian Journal of Statistics 37: 361–380; 2009 © 2009 Statistical Society of Canada     

Generalized manova double linear hypothesis with double linear restrictions

For the generalized MANOVA model of Potthoff and Roy [7], Gleser and Olkin [3] give a likelihood ratio test criterion for testing double linear parametric functions of the regression parameters. Their theory is extended in this paper to the testing of double linear parametric functions with double linear restrictions on the parameters. The theory is presented in terms of the original variates unlike Gleser and Olkin [3] who resort to canonical transformations of the original variates.     

Multivariate Linear Regression Model with Elliptically Contoured Distributed Errors and Monotone Missing Dependent Variables

In this article, the multivariate linear regression model is studied under the assumptions that the error term of this model is described by the elliptically contoured distribution and the observations on the response variables are of a monotone missing pattern. It is primarily concerned with estimation of the model parameters, as well as with the development of the likelihood ratio test in order to examine the existence of linear constraints on the regression coefficients. An illustrative example is presented for the explanation of the results.     

Parametric Estimator of Linear Model with Interval-Censored Data

In this article, we consider the estimation of regression parameters in linear model in the presence of interval-censored data. When the response variable is interval-censored, the traditional methods can not be used to estimate the parameters directly. In this article, unbiased transformation is carried out and a new random variable which has the same expectation as the function of the response variable is established. With the regression analysis for the constructed statistic we conclude the estimator by least square method.     

Inferences on regression coefficients in a regression model under heteroscedasticity and robustness with respect to departure from normality

In this paper we consider a simple linear regression model under heteroscedasticity and nonnormality. A statistical test for testing the regression coefficient is then derived by assuming normality for the random disturbances and by applying Welch's method. Some Monte Carlo studies are generated for assessing robustness of this test. By combining Tiku's robust procedure with the new test, a robust but more powerful test is developed.     

On the asymptotic distribution of Cook's distance in logistic regression models

It sometimes occurs that one or more components of the data exert a disproportionate influence on the model estimation. We need a reliable tool for identifying such troublesome cases in order to decide either eliminate from the sample, when the data collect was badly realized, or otherwise take care on the use of the model because the results could be affected by such components. Since a measure for detecting influential cases in linear regression setting was proposed by Cook [Detection of influential observations in linear regression, Technometrics 19 (1977), pp. 15–18.], apart from the same measure for other models, several new measures have been suggested as single-case diagnostics. For most of them some cutoff values have been recommended (see [D.A. Belsley, E. Kuh, and R.E. Welsch, Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, 2nd ed., John Wiley & Sons, New York, Chichester, Brisban, (2004).], for instance), however the lack of a quantile type cutoff for Cook's statistics has induced the analyst to deal only with index plots as worthy diagnostic tools. Focussed on logistic regression, the aim of this paper is to provide the asymptotic distribution of Cook's distance in order to look for a meaningful cutoff point for detecting influential and leverage observations.     

multiple and conditional deletion diagnostics for general linear models

In this paper we develop multiple case deletion statistics for the general linear model so that a residual vector and a leverage matrix are identified which have roles analogous to residuals and leverage for ordinary least squares models. We extend the notion of the conditional deletion diagnostic to general linear models. The residuals, leverage and deletion diagnostics are illustrated with data modelled by a linear growth curve.     

Admissible variable-selection procedures when fitting misspecified regression models by least squares

For loss equal to squared error of prediction, Kempthorne(l984) has proved that all variable-selection procedures are admissible for choosing among least-squares fits of a normal linear regression model. We extend this result to the case of a normal linear regression model in which the form of the expected response vector is misspecified.     

Test-Based Interval Estimation Under the Accelerated Failure Time Model

Some new results of a distance—based (DB) model for prediction with mixed variables are presented and discussed. This model can be thought of as a linear model where predictor variables for a response Y are obtained from the observed ones via classic multidimensional scaling. A coefficient is introduced in order to choose the most predictive dimensions, providing a solution to the problem of small variances and a very large number n of observations (the dimensionality increases as n). The problem of missing data is explored and a DB solution is proposed. It is shown that this approach can be regarded as a kind of ridge regression when the usual Euclidean distance is used.     

Heteroskedastic linear regression model with compositional response and covariates

Compositional data are known as a sort of complex multidimensional data with the feature that reflect the relative information rather than absolute information. There are a variety of models for regression analysis with compositional variables. Similar to the traditional regression analysis, the heteroskedasticity still exists in these models. However, the existing heteroskedastic regression analysis methods cannot apply in these models with compositional error term. In this paper, we mainly study the heteroskedastic linear regression model with compositional response and covariates. The parameter estimator is obtained through weighted least squares method. For the hypothesis test of parameter, the test statistic is based on the original least squares estimator and corresponding heteroskedasticity-consistent covariance matrix estimator. When the proposed method is applied to both simulation and real example, we use the original least squares method as a comparison during the whole process. The results implicate the model's practicality and effectiveness in regression analysis with heteroskedasticity.     

Transformed generalized linear models

The estimation of data transformation is very useful to yield response variables satisfying closely a normal linear model. Generalized linear models enable the fitting of models to a wide range of data types. These models are based on exponential dispersion models. We propose a new class of transformed generalized linear models to extend the Box and Cox models and the generalized linear models. We use the generalized linear model framework to fit these models and discuss maximum likelihood estimation and inference. We give a simple formula to estimate the parameter that index the transformation of the response variable for a subclass of models. We also give a simple formula to estimate the r$r$th moment of the original dependent variable. We explore the possibility of using these models to time series data to extend the generalized autoregressive moving average models discussed by Benjamin et al. [Generalized autoregressive moving average models. J. Amer. Statist. Assoc. 98, 214–223]. The usefulness of these models is illustrated in a simulation study and in applications to three real data sets.