On multivariate confidence regions and simultaneous confidence limits for ratios

Convenient general linear model computational procedures are presented for constructing multivariate confidence regions and simultaneous confidence limits for ratios of linear combinations of the parameters. The practical consequence is that a single general linear model computer program, capable of validating the underlying model and estimating the parameters, can (after slight modification) also construct the confidence regions, and even determine their precise analytic form (ellipsoid, hyperboloid, etc.). The text is deliberately factual while the appendices extend and help clarify earlier work by Henry Scheffe. As an example, a confidence ellipse and simultaneous confidence limits are constructed for several relative potencies in a classical multiple parallel line bioassay.     

Efficient estimation of general linear mixed effects models

An extension of the linear growth curve model (Biometrics 38 (1982) 963) was proposed by Stukel and Demidenko (Biometrics 53 (1997) 720) to study the effects of population covariates on one or more characteristics of the curve, when the characteristics are expressed as linear combinations of the growth curve parameters. In the present paper, this general growth curve model receives a comprehensive theoretical treatment. A two-stage estimator, consisting of a generalized least squares estimator under constraints for the population parameters and a moment estimator for the variance parameters, is developed for application in the non-Gaussian error situation. Two likelihood based estimators, global maximum likelihood and second-stage maximum likelihood, are also developed. It is shown that all three estimators are consistent, asymptotically normally distributed, and efficient, and are equivalent when the number of individuals tends to infinity. An expression for the bias in the estimator of the population parameters is derived under second stage model misspecification. We show that if parameters that are not of primary interest are incorrectly specified, bias may occur in parameters that are of interest using the standard growth curve model. The general growth curve model does not require specification of such nuisance parameters and is robust in terms of bias. The general linear growth curve model is used to study the effects of host sex on pancreatic tumor growth in rats.     

Estimable functions in the nonlinear model

When the method of least squares is used to estimate the parameters in a general model and the generated system of normal equations is linearly dependent, the estimate of the vector of parameters which satisfies the criterion is not unique. However, there exist certain functions of the estimated vector of parameters which are invariant to the least squares solution obtained from the normal equations. We define those invariant functions to be estimable, and present a technique to determine the functions of the parameters which are estimable for the general model. The method results in solving either a linear first order partial differential equation or a system of linear first order partial differential equations corresponding, respectively, to a single or multiple dependency between columns of the Jacobian matrix of the mean of the model. The usual results concerning estimability for linear models are a special case of the general results developed.     

A note on model selection using information criteria for general linear models estimated using REML

It is common practice to compare the fit of non‐nested models using the Akaike (AIC) or Bayesian (BIC) information criteria. The basis of these criteria is the log‐likelihood evaluated at the maximum likelihood estimates of the unknown parameters. For the general linear model (and the linear mixed model, which is a special case), estimation is usually carried out using residual or restricted maximum likelihood (REML). However, for models with different fixed effects, the residual likelihoods are not comparable and hence information criteria based on the residual likelihood cannot be used. For model selection, it is often suggested that the models are refitted using maximum likelihood to enable the criteria to be used. The first aim of this paper is to highlight that both the AIC and BIC can be used for the general linear model by using the full log‐likelihood evaluated at the REML estimates. The second aim is to provide a derivation of the criteria under REML estimation. This aim is achieved by noting that the full likelihood can be decomposed into a marginal (residual) and conditional likelihood and this decomposition then incorporates aspects of both the fixed effects and variance parameters. Using this decomposition, the appropriate information criteria for model selection of models which differ in their fixed effects specification can be derived. An example is presented to illustrate the results and code is available for analyses using the ASReml‐R package.     

Test sizes and the importance of finding global maxima of the likelihood function

This paper demonstrates the importance of taking care to find a global maximum when estimating nuisance parameters required for hypothesis tests. We find that sizes can be badly inflated when this is not done for a test of the general linear regression model.     

Tests of hypotheses based on ranks in the general linear model

A unified approach is developed for testing hypotheses in the general linear model based on the ranks of the residuals. It complements the nonparametric estimation procedures recently reported in the literature. The testing and estimation procedures together provide a robust alternative to least squares. The methods are similar in spirit to least squares so that results are simple to interpret. Hypotheses concerning a subset of specified parameters can be tested, while the remaining parameters are treated as nuisance parameters. Asymptotically, the test statistic is shown to have a chi-square distribution under the null hypothesis. This result is then extended to cover a sequence of contiguous alternatives from which the Pitman efficacy is derived. The general application of the test requires the consistent estimation of a functional of the underlying distribution and one such estimate is furnished.     

JACKKNIFING THE GENERAL LINEAR MODEL

The aim of this paper is to investigate the problems of estimating a smooth function of the parameters in a general linear model and to clarify some of the points raised by Hinkley (1977) in connection with this problem. An example of the type of problem at hand is that of estimating the maximum (or minimum) mean value in a quadratic regression model. The estimator based on the least squares estimator of the parameters in the linear model is compared to the jackknife estimator and the weighted jackknife estimator proposed by Hinkley (1977). The asymptotic properties of the estimators are examined and their small sample properties are compared through simulation studies.     

Sliced inverse regression under linear constraints

We consider the semiparametric regression model introduced by Li (1991) and add to this model some linear constraints on the slope parameters. These constraints can be identifiability conditions or they may carry additional in¬formations on the slope parameters. Using a geometric argument, we develop a method to estimate the slope parameters. This link-free and distribution-free method splits in two steps: the first is a Sliced Inverse Regression (SIR); Canonical Analysis is used at the second step to transform the SIR estimates so that they satisfy the constraints. We establish yn-consistency and obtain the asymptotic distribution of the estimates.

This estimation method is applied to the general sample selection model which is very useful in Econometrics. A simulation study shows that the method performs well in the example considered.     

Categorical information and the singular linear model

C. R. Rao (1978) discusses estimation for the common linear model in the case that the variance matrix σ² Q has known singular form Q . In the more general context of inference, this model exhibits certain special features and illustrates how information concerning unknowns can separate into a categorical component and a statistical component. The categorical component establishes that certain parameters are known in value and thus are not part of the statistical inference.     

Rank-based ridge estimation in multiple linear regression

Multicollinearity and model misspecification are frequently encountered problems in practice that produce undesirable effects on classical ordinary least squares (OLS) regression estimator. The ridge regression estimator is an important tool to reduce the effects of multicollinearity, but it is still sensitive to a model misspecification of error distribution. Although rank-based statistical inference has desirable robustness properties compared to the OLS procedures, it can be unstable in the presence of multicollinearity. This paper introduces a rank regression estimator for regression parameters and develops tests for general linear hypotheses in a multiple linear regression model. The proposed estimator and the tests have desirable robustness features against the multicollinearity and model misspecification of error distribution. Asymptotic behaviours of the proposed estimator and the test statistics are investigated. Real and simulated data sets are used to demonstrate the feasibility and the performance of the estimator and the tests.     

Linear Models with Doubly Exchangeable Distributed Errors

We study the general linear model (GLM) with doubly exchangeable distributed error for m observed random variables. The doubly exchangeable general linear model (DEGLM) arises when the m-dimensional error vectors are “doubly exchangeable,” jointly normally distributed, which is a much weaker assumption than the independent and identically distributed error vectors as in the case of GLM or classical GLM (CGLM). We estimate the parameters in the model and also find their distributions. We show that the tests of intercept and slope are possible in DEGLM as a particular case using parametric bootstrap as well as multivariate Satterthwaite approximation.     

A note on the unified theory of least squares

The general form of a matrix which appears in the normal equation for estimating parameters in the Gauss-Markoff linear model has been obtained.     

A non-iterative approach to estimating parameters in a linear structural equation model

The research described herein was motivated by a study of the relationship between the performance of students in senior high schools and at universities in China. A special linear structural equation model is established, in which some parameters are known and both the responses and the covariables are measured with errors. To explore the relationship between the true responses and latent covariables and to estimate the parameters, we suggest a non-iterative estimation approach that can account for the external dependence between the true responses and latent covariables. This approach can also deal with the collinearity problem because the use of dimension-reduction techniques can remove redundant variables. Combining further with the information that some of parameters are given, we can perform estimation for the other unknown parameters. An easily implemented algorithm is provided. A simulation is carried out to provide evidence of the performance of the approach and to compare it with existing methods. The approach is applied to the education example for illustration, and it can be readily extended to more general models.     

Accounting for variability in individual hierarchical clinical trial data

Meta-analytical approaches have been extensively used to analyze medical data. In most cases, the data come from different studies or independent trials with similar characteristics. However, these methods can be applied in a broader sense. In this paper, we show how existing meta-analytic techniques can also be used as well when dealing with parameters estimated from individual hierarchical data. Specifically, we propose to apply statistical methods that account for the variances (and possibly covariances) of such measures. The estimated parameters together with their estimated variances can be incorporated into a general linear mixed model framework. We illustrate the methodology by using data from a first-in-man study and a simulated data set. The analysis was implemented with the SAS procedure MIXED and example code is offered.     

Bayesian forecasting with changing linear models

This investigation considers a general linear model which changes parameters exactly once during the observation period. Assuming all the parameters are unknown and a proper prior distribution, the Bayesian predictive distribution of the future observations is derived.

It is shown that the predictive distribution is a mixture of multivariate t distributions and that the mixing distribution is the marginal posterior mass function of the change point parameter.     

Diagnostic checks for discrete data regression models using posterior predictive simulations

Model checking with discrete data regressions can be difficult because the usual methods such as residual plots have complicated reference distributions that depend on the parameters in the model. Posterior predictive checks have been proposed as a Bayesian way to average the results of goodness-of-fit tests in the presence of uncertainty in estimation of the parameters. We try this approach using a variety of discrepancy variables for generalized linear models fitted to a historical data set on behavioural learning. We then discuss the general applicability of our findings in the context of a recent applied example on which we have worked. We find that the following discrepancy variables work well, in the sense of being easy to interpret and sensitive to important model failures: structured displays of the entire data set, general discrepancy variables based on plots of binned or smoothed residuals versus predictors and specific discrepancy variables created on the basis of the particular concerns arising in an application. Plots of binned residuals are especially easy to use because their predictive distributions under the model are sufficiently simple that model checks can often be made implicitly. The following discrepancy variables did not work well: scatterplots of latent residuals defined from an underlying continuous model and quantile–quantile plots of these residuals.     

Sufficient and admissible estimators in general multivariate linear model

The notion of linear sufficiency in general Gauss–Markov model is extended to a general multivariate linear model for any specific set of estimable functions. A general formula of the difference between the dispersion matrix of the BLUE in the original model and that in the transformed model is provided, which brings some further contributions to the theory of linear sufficiency. Moreover, a general formula of the change of BLUE due to transformation is obtained. The analysis here leads to some results, some of which are known in the literature. Besides linear sufficiency, the admissibility of a linear statistic is also extended to the multivariate case.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

A weighted dispersion function for estimation in linear models

Robust estimates for the parameters in the general linear model are proposed which are based on weighted rank statistics. The method is based on the minimization of a dispersion function defined by a weighted Gini's mean difference. The asymptotic distribution of the estimate is derived with an asymptotic linearity result. An influence function is determined to measure how the weights can reduce the influence of high-leverage points. The weights can also be used to base the ranking on a restricted set of comparisons. This is illustrated in several examples with stratified samples, treatment vs control groups and ordered alternatives.     

Generalized symmetrical partial linear model

In this work, we propose a new model called generalized symmetrical partial linear model, based on the theory of generalized linear models and symmetrical distributions. In our model the response variable follows a symmetrical distribution such a normal, Student-t, power exponential, among others. Following the context of generalized linear models we consider replacing the traditional linear predictors by the more general predictors in whose case one covariate is related with the response variable in a non-parametric fashion, that we do not specified the parametric function. As an example, we could imagine a regression model in which the intercept term is believed to vary in time or geographical location. The backfitting algorithm is used for estimating the parameters of the proposed model. We perform a simulation study for assessing the behavior of the penalized maximum likelihood estimators. We use the quantile residuals for checking the assumption of the model. Finally, we analyzed real data set related with pH rivers in Ireland.