A theorem on invariance of estimability of linear parametric functions in linear models

Consider the general linear model Y = Xβ + ? , where E[??'] = σ²I and rank of X is less than or equal to the number of columns of X. It is well known that the linear parametric function λ'β is estimable if and only if λ' is in the row space of X. This paper characterizes all orthogonal matrices P such that the row space of XP is equal to the row space of X, i.e. the estimability of λ'β is invariant under P. An additional property of these matrices is the invariance of the spectrum of the information matrix X'X. An application of the results is also given.     

Use of likelihood ratio tests to detect outliers under the variance shift outlier model

In this paper, we revisit the alternative outlier model of Thompson [A note on restricted maximum likelihood estimation with an alternative outlier model, J. Roy. Stat. Soc. Ser. B 47 (1985), pp. 53–55] for detecting outliers in the linear model. Gumedze et al. [A variance shift model for detection of outliers in the linear mixed model, Comput. Statist. Data Anal. 54 (2010), pp. 2128–2144] called this model the variance shift outlier model (VSOM). The basic idea behind the VSOM is to detect observations with inflated variance and isolate them for further investigation. The VSOM is appealing because it downweights an outlier in the analysis, with the weighting determined automatically as part of the estimation procedure. We set up the VSOM as a linear mixed model and then use the likelihood ratio test (LRT) statistic as an objective measure for determining whether the weighting is required, i.e. whether the observation is an outlier. We also derived one-step updates of the variance parameter estimates based on observed, expected and average information matrices to obtain one-step LRT statistics which usually require less computation. Both the fully iterated and one-step LRTs are functions of the squared standard residuals from the null model and therefore can be computed directly without the need to fit the VSOM. We investigated the properties of the likelihood ratio tests and compare them. An extension of the model to detect a group of outliers is also given. We illustrate the proposed methodology using simulated datasets and a real dataset.     

Local Influence Analysis in AB–BA Crossover Designs

The aim of this article is to develop methodology for detecting influential observations in crossover models with random individual effects. Various case‐weighted perturbations are performed. We obtain the influence of the perturbations on each parameter estimator and on their dispersion matrices. The obtained results exhibit the possibility to obtain closed‐form expressions of the influence using the residuals in mixed linear models. Some graphical tools are also presented.     

Sampling from a Max‐Stable Process Conditional on a Homogeneous Functional with an Application for Downscaling Climate Data

Conditional simulation of max‐stable processes allows for the analysis of spatial extremes taking into account additional information provided by the conditions. Instead of observations at given sites as usually done, we consider a single condition given by a more general functional of the process as may occur in the context of climate models. As the problem turns out to be intractable analytically, we make use of Markov chain Monte Carlo methods to sample from the conditional distribution. Simulation studies indicate fast convergence of the Markov chains involved. In an application to precipitation data, the utility of the procedure as a tool to downscale climate data is demonstrated.     

Some properties of,and relationships among,several uncorrelated and homoscedastic residual vectors

In this paper we examine the properties of four types of residual vectors, arising from fitting a linear regression model to a set of data by least squares. The four types of residuals are (i) the Stepwise residuals (Hedayat and Robson, 1970), (ii) the Recursive residuals (Brown, Durbin, and Evans, 1975), (iii) the Sequentially Adjusted residuals (to be defined herein), and (iv) the BLUS residuals (Theil, 1965, 1971). We also study the relationships among the four residual vectors. It is found that, for any given sequence of observations, (i) the first three sets of residuals are identical, (ii) each of the first three sets, being identical, is a member of Thei’rs (1965, 1971) family of residuals; specifically, they are Linear Unbiased with a Scalar covariance matrix (LUS) but not Best Linear Unbiased with a Scalar covariance matrix (BLUS). We find the explicit form of the transformation matrix and show that the first three sets of residual vectors can be written as an orthogonal transformation of the BLUS residual vector. These and other properties may prove to be useful in the statistical analysis of residuals.     

Theory & Methods: Residual Diagnostic Plots for Checking for Model Mis‐specification in Time Series Regression

This paper considers residuals for time series regression. Despite much literature on visual diagnostics for uncorrelated data, there is little on the autocorrelated case. To examine various aspects of the fitted time series regression model, three residuals are considered. The fitted regression model can be checked using orthogonal residuals; the time series error model can be analysed using marginal residuals; and the white noise error component can be tested using conditional residuals. When used together, these residuals allow identification of outliers, model mis‐specification and mean shifts. Due to the sensitivity of conditional residuals to model mis‐specification, it is suggested that the orthogonal and marginal residuals be examined first.     

Jarque–Bera Test and its Competitors for Testing Normality – A Power Comparison

For testing normality we investigate the power of several tests, first of all, the well-known test of Jarque & Bera (1980) and furthermore the tests of Kuiper (1960) and Shapiro & Wilk (1965) as well as tests of Kolmogorov–Smirnov and Cramér-von Mises type. The tests on normality are based, first, on independent random variables (model I) and, second, on the residuals in the classical linear regression (model II). We investigate the exact critical values of the Jarque–Bera test and the Kolmogorov–Smirnov and Cramér-von Mises tests, in the latter case for the original and standardized observations where the unknown parameters μ and σ have to be estimated. The power comparison is carried out via Monte Carlo simulation assuming the model of contaminated normal distributions with varying parameters μ and σ and different proportions of contamination. It turns out that for the Jarque–Bera test the approximation of critical values by the chi-square distribution does not work very well. The test is superior in power to its competitors for symmetric distributions with medium up to long tails and for slightly skewed distributions with long tails. The power of the Jarque–Bera test is poor for distributions with short tails, especially if the shape is bimodal – sometimes the test is even biased. In this case a modification of the Cramér-von Mises test or the Shapiro–Wilk test may be recommended.     

Quotient spaces and statistical models

The purpose of this paper is to draw attention to the widespread occurrence of quotient spaces in statistical work. Quotient spaces are intrinsic to probability distributions, residuals and interaction in linear models, covariance functions and variograms of stochastic processes, etc. The theme is that explicit recognition of the quotient space can offer surprising conceptual simplification. The advantages of working directly with the quotient space are hard to describe in general. As the examples demonstrate, the answer lies partly in directness of approach.     

Oracle model selection for correlated data via residuals

This paper concerns model selection for autoregressive time series when the observations are contaminated with trend. We propose an adaptive least absolute shrinkage and selection operator (LASSO) type model selection method, in which the trend is estimated by B-splines, the detrended residuals are calculated, and then the residuals are used as if they were observations to optimize an adaptive LASSO type objective function. The oracle properties of such an adaptive LASSO model selection procedure are established; that is, the proposed method can identify the true model with probability approaching one as the sample size increases, and the asymptotic properties of estimators are not affected by the replacement of observations with detrended residuals. The intensive simulation studies of several constrained and unconstrained autoregressive models also confirm the theoretical results. The method is illustrated by two time series data sets, the annual U.S. tobacco production and annual tree ring width measurements.     

ANOVA-like analysis of matched series of studies with a common structure

It is shown how to condense the information contained in a series of studies, each constituted by an objects by variables matrix and a pair of weight matrices, into a structure vector and a sum of sums of squares of residuals. Based on this condensation we propose to carry out ANOVA-like inference for matched series of studies associated with the level combinations of some factors. It is shown how to validate the assumptions underlying the inference. An application to the results of local elections in Portugal is given.     

Consistent estimators of the variance-covariance matrix of the gmanova model with missing data

A common problem in multivariate general linear models is partially missing response data. The simplest method of analysis in the presence of missing data has been to delete all observations on any individual with any missing data(listwise deletion) and utilize a traditional complete data approach. However: this can result in a great loss of information: and perhaps inconsistencies in the estimation of the variance-covariance matrix. In the generalized multivariate analysis of variance(GMANOVA) model with missing data: Kleinbaum(1973) proposed an estimated generalized least squares approach. In order to apply this: however: a consistent estimate of the variance-covariance matrix is needed. Kleinbaum proposed an estimator which is unbiased and consistent: but it does not take advantage of the fact that the underlying model is GMANOVA and not MANOVA. Using the fact that the underlying model is GMANOVA we have constructed four other con¬sistent estimators. A Monte Carlo simulation experiment is conducted tto further examine how well these estimators compare to the estimator proposed by Kleinbaum.     

State-space models for multivariate longitudinal data of mixed types

We propose a class of state-space models for multivariate longitudinal data where the components of the response vector may have different distributions. The approach is based on the class of Tweedie exponential dispersion models, which accommodates a wide variety of discrete, continuous and mixed data. The latent process is assumed to be a Markov process, and the observations are conditionally independent given the latent process, over time as well as over the components of the response vector. This provides a fully parametric alternative to the quasilikelihood approach of Liang and Zeger. We estimate the regression parameters for time-varying covariates entering either via the observation model or via the latent process, based on an estimating equation derived from the Kalman smoother. We also consider analysis of residuals from both the observation model and the latent process.     

Classification of an observation into linear manifolds,their closed subsets and closed convex subsets

In this paper, we generalize the notion of classification of an observation (sample), into one of the given n populations to the case where some or all of the populations into which the new observation is to be classified may be new but related in a simple way to the given n populations. The discussion is in the frame-work of the given set of observations obeying the usual multivariate general linear hypothesis model. The set ofpopulations into which the new observation may be classified could be linear manifolds of the parameter space or their closed subsets or closed convex subsets or a combination of them or simply t subsets of the parameter space each of which has a finite number of elements. In the last case alikelihood ratio procedure can be obtained easily. Classification procedures given here are based on Mahalanobis distance. Bonferroni lower bound estimate of the probability of correctly classifying an observation is given for the case when the covariance matrix is known or is estimated from a large sample. A numerical example relating to the classification procedures suggested her is given.     

Maximum likelihood estimators for extended growth curve model with orthogonal between-individual design matrices

The extended growth curve model is discussed in this paper. There are two versions of the model studied in the literature, which differ in the way how the column spaces of the design matrices are nested. The nesting is applied either to the between-individual or to the within-individual design matrices. Although both versions are equivalent via reparametrization, the properties of estimators cannot be transferred directly because of non-linearity of estimators. Since in many applications the between-individual matrices are one-way ANOVA matrices, it is reasonable to assume orthogonality of the column spaces of between-individual design matrices along with nestedness of the column spaces of within-individual design matrices. We present the maximum likelihood estimators and their basic moments for the model with such orthogonality condition.     

Score test of homogeneity for survival data

If follow-up is made for subjects which are grouped into units, such as familial or spatial units then it may be interesting to test whether the groups are homogeneous (or independent for given explanatory variables). The effect of the groups is modelled as random and we consider a frailty proportional hazards model which allows to adjust for explanatory variables. We derive the score test of homogeneity from the marginal partial likelihood and it turns out to be the sum of a pairwise correlation term of martingale residuals and an overdispersion term. In the particular case where the sizes of the groups are equal to one, this statistic can be used for testing overdispersion. The asymptotic variance of this statistic is derived using counting process arguments. An extension to the case of several strata is given. The resulting test is computationally simple; its use is illustrated using both simulated and real data. In addition a decomposition of the score statistic is proposed as a sum of a pairwise correlation term and an overdispersion term. The pairwise correlation term can be used for constructing a statistic more robust to departure from the proportional hazard model, and the overdispesion term for constructing a test of fit of the proportional hazard model.     

Outlier detection in linear models: a comparative study in simple linear regression

Five widely used test statistics for detecting outliers and influential observations were studied using Monte Carlo method . The test statistic based on Studentized residuals, with critical values given by Tietjen, Moore and Beckman (1973), appears to be the best procedure for detecting a single outlier in simple linear regression.     

On proportional hazards assumption under the random effects models

The proportional hazards mixed-effects model (PHMM) was proposed to handle dependent survival data. Motivated by its application in genetic epidemiology, we study the interpretation of its parameter estimates under violations of the proportional hazards assumption. The estimated fixed effect turns out to be an averaged regression effect over time, while the estimated variance component could be unaffected, inflated or attenuated depending on whether the random effect is on the baseline hazard, and whether the non-proportional regression effect decreases or increases over time. Using the conditional distribution of the covariates we define the standardized covariate residuals, which can be used to check the proportional hazards assumption. The model checking technique is illustrated on a multi-center lung cancer trial.     

Bootstrapping a time series model: some empirical results

The bootstrap is a methodology for estimating standard errors. The idea is to use a Monte Carlo simulation experiment based on a nonparametric estimate of the error distribution. The main objective of this article is to demonstrate the use of the bootstrap to attach standard errors to coefficient estimates in a second-order autoregressive model fitted by least squares and maximum likelihood estimation. Additionally, a comparison of the bootstrap and the conventional methodology is made. As it turns out, the conventional asymptotic formulae (both the least squares and maximum likelihood estimates) for estimating standard errors appear to overestimate the true standard errors. But there are two problems:i. The first two observations y₁ and y₂ have been fixed, and ii. The residuals have not been inflated. After these two factors are considered in the trial and bootstrap experiment, both the conventional maximum likelihood and bootstrap estimates of the standard errors appear to be performing quite well.     

Robust sequential designs for approximately linear models

We consider the problem of the sequential choice of design points in an approximately linear model. It is assumed that the fitted linear model is only approximately correct, in that the true response function contains a nonrandom, unknown term orthogonal to the fitted response. We also assume that the parameters are estimated by M-estimation. The goal is to choose the next design point in such a way as to minimize the resulting integrated squared bias of the estimated response, to order n^-1. Explicit applications to analysis of variance and regression are given. In a simulation study the sequential designs compare favourably with some fixed-sample-size designs which are optimal for the true response to which the sequential designs must adapt.     

Change-point detection in a shape-restricted regression model

A regression model with a possible structural change and with a small number of measurements is considered. A priori information about the shape of the regression function is used to formulate the model as a linear regression model with inequality constraints and a likelihood ratio test for the presence of a change-point is constructed. The exact null distribution of the test statistic is given. Consistency of the test is proved when the noise level goes to zero. Numerical approximations to the powers against various alternatives are given and compared with the powers of the k-linear-r-ahead recursive residuals tests and CUSUM tests. Performance of four different estimators of the change-point is studied in a Monte Carlo experiment. An application of the procedures to some real data is also presented.