Selection of linear mixed-effects models for clustered data

We consider model selection for linear mixed-effects models with clustered structure, where conditional Kullback–Leibler (CKL) loss is applied to measure the efficiency of the selection. We estimate the CKL loss by substituting the empirical best linear unbiased predictors (EBLUPs) into random effects with model parameters estimated by maximum likelihood. Although the BLUP approach is commonly used in predicting random effects and future observations, selecting random effects to achieve asymptotic loss efficiency concerning CKL loss is challenging and has not been well studied. In this paper, we propose addressing this difficulty using a conditional generalized information criterion (CGIC) with two tuning parameters. We further consider a challenging but practically relevant situation where the number, $m$, of clusters does not go to infinity with the sample size. Hence the random-effects variances are not consistently estimable. We show that via a novel decomposition of the CKL risk, the CGIC achieves consistency and asymptotic loss efficiency, whether $m$ is fixed or increases to infinity with the sample size. We also conduct numerical experiments to illustrate the theoretical findings.     

Unbiasedness of two-stage estimation and prediction procedures for mixed linear models

The traditional method for estimating or predicting linear combinations of the fixed effects and realized values of the random effects in mixed linear models is first to estimate the variance components and then to proceed as if the estimated values of the variance components were the true values. This two-stage procedure gives unbiased estimators or predictors of the linear combinations provided the data vector is symmetrically distributed about its expected value and provided the variance component estimators are translation-invariant and are even functions of the data vector. The standard procedures for estimating the variance components yield even, translation-invariant estimators.     

Weighted modified first order regression procedures for estimation in linear models with missingX-observations

This paper considers the estimation of coefficients in a linear regression model with missing observations in the independent variables and introduces a modification of the standard first order regression method for imputation of missing values. The modification provides stochastic values for imputation and, as an extension, makes use of the principle of weighted mixed regression. The proposed procedures are compared with two popular procedures—one which utilizes only the complete observations and the other which employs the standard first order regression imputation method for missing values. A simulation experiment to evaluate the gain in efficiency and to examine interesting issues like the impact of varying degree of multicollinearity in explanatory variables is proceeded. Some work on the case of discrete regressor variables is in progress and will be reported in a future article to follow.     

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.     

A comparison of stein-like procedures for estimating linear regression models with multicollinear data

This paper compares several Stein-like estimation methods for estimating regression parameters. The criterion function was the mean-squared error of prediction and the parameter of interest was the mean of the response variable at the sampled values of the control variables. Large sample simulation techniques were used to evaluate the mean-squared error of the predictions. The parameters of interest were varied systematically over wide ranges.     

Admissible linear estimation in singular linear models

The admissibility results of Rao (1976), proved in the context of a nonsingular covariance matrix, are exteneded to the situation where the covariance matrix is singular. Admi.s s Lb Le linear estimators in the Gauss-Markoff model are characterized and admis-sibility of the best linear unbiased estimator is investigated.     

Selection procedures for sparse data

The problem of selecting the best of k populations is studied for data which are incomplete as some of the values have been deleted randomly. This situation is met in extreme value analysis where only data exceeding a threshold are observable. For increasing sample size we study the case where the probability that a value is observed tends to zero, but the sparse condition is satisfied, so that the mean number of observable values in each population is bounded away from zero and infinity as the sample size tends to infinity. The incomplete data are described by thinned point processes which are approximated by Poisson point processes. Under weak assumptions and after suitable transformations these processes converge to a Poisson point process. Optimal selection rules for the limit model are used to construct asymptotically optimal selection rules for the original sequence of models. The results are applied to extreme value data for high thresholds data.     

Projectors and linear estimation in general linear models

The paper gives a self-contained account of minimum disper­sion linear unbiased estimation of the expectation vector in a linear model with the dispersion matrix belonging to some, rather arbitrary, set of nonnegative definite matrices. The approach to linear estimation in general linear models recommended here is a direct generalization of some ideas and results presented by Rao (1973, 19 74) for the case of a general Gauss-Markov model

A new insight into the nature of some estimation problems originaly arising in the context of a general Gauss-Markov model as well as the correspondence of results known in the literature to those obtained in the present paper for general linear models are also given. As preliminary results the theory of projectors defined by Rao (1973) is extended.     

Collinearity in generalized linear models

This paper defines collinearity for generalized linear models (GLMs), investigates its consequences and proposes diagnostic criteria. The relationship between collinearity in GLMs and standard linear models (SLMs) is explored and bounds which relate the degree of collinearity in these two models are given. Estimation based on ridge methods is discussed.     

Atypical observations in linear models

The problem of ‘atypical data point’ in the estimation of model parameters and its effect on prediction are discussed. A cross-validity procedure is then proposed to accommodate the unusual observations in the estimation and thereby to improve the prediction of future data points. Each atypical point, weighted according to cross-validitory procedure, is used in the estimation of model parameters.     

Prediction in linear mixed models

Following estimation of effects from a linear mixed model, it is often useful to form predicted values for certain factor/variate combinations. The process has been well defined for linear models, but the introduction of random effects into the model means that a decision has to be made about the inclusion or exclusion of random model terms from the predictions. This paper discusses the interpretation of predictions formed including or excluding random terms. Four datasets are used to illustrate circumstances where different prediction strategies may be appropriate: in an orthogonal design, an unbalanced nested structure, a model with cubic smoothing spline terms and for kriging after spatial analysis. The examples also show the need for different weighting schemes that recognize nesting and aliasing during prediction, and the necessity of being able to detect inestimable predictions.     

Optimal allocation for linear combinations of linear predictors in generalized linear models

The problem of the allocation of experimental units to experimental groups is studied within the context of generalized linear models. Optimal designs for the estimation of linear combinations of linear predictors are characterized, using concepts from the theory of optimal design. If there is only one linear combination of interest, then the D-optimal allocation is equivalent to the well-known Neyman allocation of subsamples in stratified sampling. However, if the number of linear combinations equals the number of design points, or experimental groups, then the equal replication of all design points is D-optimal. For cases in between, there are no easily accessible general solutions to the problem, although some particular cases are solved, including: i estimation of the n- 1 possible comparisons with a control group in an n-point, one-factor design; and ii estimation of 2 one or two of the four natural parameters of a 2 factorial design. The A-optimal allocations are determined in general.     

Reversible jump methods for generalised linear models and generalised linear mixed models

A reversible jump algorithm for Bayesian model determination among generalised linear models, under relatively diffuse prior distributions for the model parameters, is proposed. Orthogonal projections of the current linear predictor are used so that knowledge from the current model parameters is used to make effective proposals. This idea is generalised to moves of a reversible jump algorithm for model determination among generalised linear mixed models. Therefore, this algorithm exploits the full flexibility available in the reversible jump method. The algorithm is demonstrated via two examples and compared to existing methods.     

Adaptive varying-coefficient linear models

Summary. Varying-coefficient linear models arise from multivariate nonparametric regression, non-linear time series modelling and forecasting, functional data analysis, longitudinal data analysis and others. It has been a common practice to assume that the varying coefficients are functions of a given variable, which is often called an index . To enlarge the modelling capacity substantially, this paper explores a class of varying-coefficient linear models in which the index is unknown and is estimated as a linear combination of regressors and/or other variables. We search for the index such that the derived varying-coefficient model provides the least squares approximation to the underlying unknown multidimensional regression function. The search is implemented through a newly proposed hybrid backfitting algorithm. The core of the algorithm is the alternating iteration between estimating the index through a one-step scheme and estimating coefficient functions through one-dimensional local linear smoothing. The locally significant variables are selected in terms of a combined use of the t -statistic and the Akaike information criterion. We further extend the algorithm for models with two indices. Simulation shows that the methodology proposed has appreciable flexibility to model complex multivariate non-linear structure and is practically feasible with average modern computers. The methods are further illustrated through the Canadian mink–muskrat data in 1925–1994 and the pound–dollar exchange rates in 1974–1983.     

Transformed generalized linear models

The estimation of data transformation is very useful to yield response variables satisfying closely a normal linear model. Generalized linear models enable the fitting of models to a wide range of data types. These models are based on exponential dispersion models. We propose a new class of transformed generalized linear models to extend the Box and Cox models and the generalized linear models. We use the generalized linear model framework to fit these models and discuss maximum likelihood estimation and inference. We give a simple formula to estimate the parameter that index the transformation of the response variable for a subclass of models. We also give a simple formula to estimate the r$r$th moment of the original dependent variable. We explore the possibility of using these models to time series data to extend the generalized autoregressive moving average models discussed by Benjamin et al. [Generalized autoregressive moving average models. J. Amer. Statist. Assoc. 98, 214–223]. The usefulness of these models is illustrated in a simulation study and in applications to three real data sets.     

Bayesian likelihood robustness in linear models

This paper deals with the problem of robustness of Bayesian regression with respect to the data. We first give a formal definition of Bayesian robustness to data contamination, prove that robustness according to the definition cannot be obtained by using heavy-tailed error distributions in linear regression models and propose a heteroscedastic approach to achieve the desired Bayesian robustness.     

Prediction in multilevel generalized linear models

Summary.  We discuss prediction of random effects and of expected responses in multilevel generalized linear models. Prediction of random effects is useful for instance in small area estimation and disease mapping, effectiveness studies and model diagnostics. Prediction of expected responses is useful for planning, model interpretation and diagnostics. For prediction of random effects, we concentrate on empirical Bayes prediction and discuss three different kinds of standard errors; the posterior standard deviation and the marginal prediction error standard deviation (comparative standard errors) and the marginal sampling standard deviation (diagnostic standard error). Analytical expressions are available only for linear models and are provided in an appendix . For other multilevel generalized linear models we present approximations and suggest using parametric bootstrapping to obtain standard errors. We also discuss prediction of expectations of responses or probabilities for a new unit in a hypothetical cluster, or in a new (randomly sampled) cluster or in an existing cluster. The methods are implemented in gllamm and illustrated by applying them to survey data on reading proficiency of children nested in schools. Simulations are used to assess the performance of various predictions and associated standard errors for logistic random-intercept models under a range of conditions.     

Missing plot technique in linear models

This paper extends the missing plot substitution technique to the case where the missing observations-cause some previously estimable functions to become non-estimable. It is shown that with appropriate modifications, the usual methods of analysis remain valid. We also obtain necessary and sufficient conditions under which the sum of squares due to a hypothesis can be calculated without “re-estimating” the missing observations     

Residual components in generalized linear models

The adequacy of a postulated generalized linear model can often be improved by transforming predictors and/or including additional explanatory variables. To assess the fit relative to a given predictor, we define its corresponding residual component. Asymptotic bias and variance of the residual component are considered, paying particular attention to the case that the presumed model is valid.     

Detecting structural change in linear models

Consider a sequence of independent observations which change their marginal distribution at most once somewhere in the sequence and one is not certain where the change has occurred. One would be interested in detecting the change and determining the two distributions which would describe the sequence. On the other hand if no change had occurred, one would want to know the common distribution of the observations. This study develops a Bayesian test for detecting a switch from one linear model to another. The test is based on the marginal posterior mass function of the switch point and the posterior probability of a stable model. This test and an informal sequential procedure of Smith are illustrated with data generated from an unstable linear regression model, which changes the linear relationship between the dependent and independent variables