Transformation approaches of linear random-effects models

Assume that a linear random-effects model \(\mathbf{y}= \mathbf{X}\varvec{\beta }+ \varvec{\varepsilon }= \mathbf{X}(\mathbf{A}\varvec{\alpha }+ \varvec{\gamma }) + \varvec{\varepsilon }\) is transformed as \(\mathbf{T}\mathbf{y}= \mathbf{T}\mathbf{X}\varvec{\beta }+ \mathbf{T}\varvec{\varepsilon }= \mathbf{T}\mathbf{X}(\mathbf{A}\varvec{\alpha }+ \varvec{\gamma }) + \mathbf{T}\varvec{\varepsilon }\) by pre-multiplying a given matrix \(\mathbf{T}\) of arbitrary rank. These two models are not necessarily equivalent unless \(\mathbf{T}\) is of full column rank, and we have to work with this derived model in many situations. Because predictors/estimators of the parameter spaces under the two models are not necessarily the same, it is primary work to compare predictors/estimators in the two models and to establish possible links between the inference results obtained from two models. This paper presents a general algebraic approach to the problem of comparing best linear unbiased predictors (BLUPs) of parameter spaces in an original linear random-effects model and its transformations, and provides a group of fundamental and comprehensive results on mathematical and statistical properties of the BLUPs. In particular, we construct many equalities for the BLUPs under an original linear random-effects model and its transformations, and obtain necessary and sufficient conditions for the equalities to hold.     

Computations of predictors/estimators under a linear random-effects model with parameter restrictions

This paper considers the problem of simultaneously predicting/estimating unknown parameter spaces in a linear random-effects model with both parameter restrictions and missing observations. We shall establish explicit formulas for calculating the best linear unbiased predictors (BLUPs) of all unknown parameters in such a model, and derive a variety of mathematical and statistical properties of the BLUPs under general assumptions. We also discuss some matrix expressions related to the covariance matrix of the BLUP, and present various necessary and sufficient conditions for several equalities and inequalities of the covariance matrix of the BLUP to hold.     

Random-effect models with singular precision

We show that smoothing spline, intrinsic autoregression (IAR) and state-space model can be formulated as partially specified random-effect model with singular precision (SP). Various fitting methods have been suggested for the aforementioned models and this paper investigates the relationships among them, once the models have been placed under a single framework. Some methods have been previously shown to give the best linear unbiased predictors (BLUPs) under some random-effect models and here we show that they are in fact uniformly BLUPs (UBLUPs) under a class of models that are generated by the SP of random effects. We offer some new interpretations of the UBLUPs under models of SP and define BLUE and BLUP in these partially specified models without having to specify the covariance. We also show how the full likelihood inferences for random-effect models can be made for these models, so that the maximum likelihood (ML) and restricted maximum likelihood (REML) estimators can be used for the smoothing parameters in splines, etc.     

Computation aspects of the parameter estimates of linear mixed effects model in multivariate repeated measures set-up

The number of parameters mushrooms in a linear mixed effects (LME) model in the case of multivariate repeated measures data. Computation of these parameters is a real problem with the increase in the number of response variables or with the increase in the number of time points. The problem becomes more intricate and involved with the addition of additional random effects. A multivariate analysis is not possible in a small sample setting. We propose a method to estimate these many parameters in bits and pieces from baby models, by taking a subset of response variables at a time, and finally using these bits and pieces at the end to get the parameter estimates for the mother model, with all variables taken together. Applying this method one can calculate the fixed effects, the best linear unbiased predictions (BLUPs) for the random effects in the model, and also the BLUPs at each time of observation for each response variable, to monitor the effectiveness of the treatment for each subject. The proposed method is illustrated with an example of multiple response variables measured over multiple time points arising from a clinical trial in osteoporosis.     

Predicting random effects with an expanded finite population mixed model

Prediction of random effects is an important problem with expanding applications. In the simplest context, the problem corresponds to prediction of the latent value (the mean) of a realized cluster selected via two-stage sampling. Recently, Stanek and Singer [Predicting random effects from finite population clustered samples with response error. J. Amer. Statist. Assoc. 99, 119–130] developed best linear unbiased predictors (BLUP) under a finite population mixed model that outperform BLUPs from mixed models and superpopulation models. Their setup, however, does not allow for unequally sized clusters. To overcome this drawback, we consider an expanded finite population mixed model based on a larger set of random variables that span a higher dimensional space than those typically applied to such problems. We show that BLUPs for linear combinations of the realized cluster means derived under such a model have considerably smaller mean squared error (MSE) than those obtained from mixed models, superpopulation models, and finite population mixed models. We motivate our general approach by an example developed for two-stage cluster sampling and show that it faithfully captures the stochastic aspects of sampling in the problem. We also consider simulation studies to illustrate the increased accuracy of the BLUP obtained under the expanded finite population mixed model.     

Optimal prediction in finite populations under matrix loss

Optimal prediction problems in finite population are investigated. Under matrix loss, we provide necessary and sufficient conditions for the linear predictor of a general linearly predictable variable to be the best linear unbiased predictor (BLUP). The essentially unique BLUP of a linearly predictable variable is obtained in the general superpopulation model. Surprisingly, the both BLUPs under matrix and quadratic loss functions are equivalent to each other. Next, we prove that the BLUP is admissible in the class of linear predictors. Conditions for optimality of the simple projection predictor (SPP) are given. Furthermore, the robust SPP and the robust BLUP are characterized on the misspecification of the covariance matrix.     

Geometric characterization of planar regression

The geometric characterization of linear regression in terms of the ‘concentration ellipse’ by Galton [Galton, F., 1886, Family likeness in stature (with Appendix by Dickson, J.D.H.). Proceedings of the Royal Society of London, 40, 42–73.] and Pearson [Pearson, K., 1901, On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2, 559–572.] was extended to the case of unequal variances of the presumably uncorrelated errors in the experimental data [McCartin, B.J., 2003, A geometric characterization of linear regression. Statistics, 37(2), 101–117.]. In this paper, this geometric characterization is further extended to planar (and also linear) regression in three dimensions where a beautiful interpretation in terms of the concentration ellipsoid is developed.     

The Equality of the Ordinary Least Squares Estimator and the Best Linear Unbiased Estimator

It is well known that the ordinary least squares estimator of Xβ in the general linear model E _y = Xβ, cov y = σ² V, can be the best linear unbiased estimator even if V is not a multiple of the identity matrix. This article presents, in a historical perspective, the development of the several conditions for the ordinary least squares estimator to be best linear unbiased. Various characterizations of these conditions, using generalized inverses and orthogonal projectors, along with several examples, are also given. In addition, a complete set of references is provided.     

General linear estimators under the prediction error sum of squares criterion in a linear regression model

In this paper, the notion of the general linear estimator and its modified version are introduced using the singular value decomposition theorem in the linear regression model y=X β+e to improve some classical linear estimators. The optimal selections of the biasing parameters involved are theoretically given under the prediction error sum of squares criterion. A numerical example and a simulation study are finally conducted to illustrate the superiority of the proposed estimators.     

Relative Potency Estimation in Parallel-Line Assays – Method Comparison and Some Extensions

Relative potency estimations in both multiple parallel-line and slope-ratio assays involve construction of simultaneous confidence intervals for ratios of linear combinations of general linear model parameters. The key problem here is that of determining multiplicity adjusted percentage points of a multivariate t-distribution, the correlation matrix R of which depends on the unknown relative potency parameters. Several methods have been proposed in the literature on how to deal with R . In this article, we introduce a method based on an estimate of R (also called the plug-in approach) and compare it with various methods including conservative procedures based on probability inequalities. Attention is restricted to parallel-line assays though the theory is applicable for any ratios of coefficients in the general linear model. Extension of the plug-in method to linear mixed effect models is also discussed. The methods will be compared with respect to their simultaneous coverage probabilities via Monte Carlo simulations. We also evaluate the methods in terms of confidence interval width through application to data on multiple parallel-line assay.     

Multiplicative Algorithms for Maximum Penalized Likelihood Inversion with Non Negative Constraints and Generalized Error Distributions

In many linear inverse problems the unknown function f (or its discrete approximation Θ _p×1), which needs to be reconstructed, is subject to the non negative constraint(s); we call these problems the non negative linear inverse problems (NNLIPs). This article considers NNLIPs. However, the error distribution is not confined to the traditional Gaussian or Poisson distributions. We adopt the exponential family of distributions where Gaussian and Poisson are special cases. We search for the non negative maximum penalized likelihood (NNMPL) estimate of Θ. The size of Θ often prohibits direct implementation of the traditional methods for constrained optimization. Given that the measurements and point-spread-function (PSF) values are all non negative, we propose a simple multiplicative iterative algorithm. We show that if there is no penalty, then this algorithm is almost sure to converge; otherwise a relaxation or line search is necessitated to assure its convergence.     

On the Expressions of Estimability in Testing General Linear Hypotheses

In testing a general linear hypothesis of the form K ′β ? ( W ′) under a general linear model, an equivalent hypothesis involving only estimable parametric functions is provided, and then an explicit test statistic in terms of the model matrices is given. The corresponding results are expanded to the case of a general linear model with a restriction and are illustrated by an example.     

Bayes and Empirical Bayes Inference in Changepoint Problems

ABSTRACT

In the case of the random design nonparametric regression, the double smoothing technique is applied to estimate the multivariate regression function. The proposed estimator has desirable properties in both the finite sample and the asymptotic cases. In the finite sample case, it has bounded conditional (and unconditional) bias and variance. On the other hand, in the asymptotic case, it has the same mean square error as the local linear estimator in Fan (Design-Adaptive Nonparametric Regression. Journal of the American Statistical Association 1992, 87, 998–1004; Local Linear Regression Smoothers and Their Minimax Efficiencies. Annals of Statistics 1993, 21, 196–216). Simulation studies demonstrate that the proposed estimator is better than the local linear estimator, because it has a smaller sample mean integrated square error and gives smoother estimates.     

Admissible Estimation for Linear Combination of Fixed and Random Effects in General Mixed Linear Models

The general mixed linear model can be denoted by y  =  X β +  Z u  +  e , where β is a vector of fixed effects, u is a vector of random effects, and e is a vector of random errors. In this article, the problem of admissibility of Q y and Q y  +  q for estimating linear functions, ? =  L ′β +  M ′ u , of the fixed and random effects is considered, and the necessary and sufficient conditions for Q y (resp. Q y  +  q ) to be admissible in the set of homogeneous (resp. potentially inhomogeneous) linear estimators with respect to the MSE and MSEM criteria are investigated. We provide a straightforward alternative proof to the method that was utilized by Wu (1988 Wu , Q. G. ( 1988 ). Several results on admissibility of a linear estimate of stochastic regression coefficients and parameters . Acta Mathemaica Applicatae Sinica 11 ( 1 ): 95 – 106 . (in Chinese)  [Google Scholar]), Baksalary and Markiewicz (1990 Baksalary , J. K. , Markiewicz , A. ( 1990 ). Admissible linear estimators of an arbitrary vector of parametric functions in the general Gauss–Markov model . J. Stat. Plann. Infer. 26 : 161 – 171 . [Google Scholar]), and Groß and Markiewicz (1999 Groß , J. , Markiewicz , A. ( 1999 ). On admissibility of linear estimators with respect to the mean square error matrix criterion under the general mixed linear model . Statistics 33 : 57 – 71 .[Taylor & Francis Online] , [Google Scholar]). In addition, we derive the corresponding results on the admissibility problem under the generalized MSE criterion.     

Minimax prediction in the linear model with a relative squared error

Arnold and Stahlecker (Stat Pap 44:107–115, 2003) considered the prediction of future values of the dependent variable in the linear regression model with a relative squared error and deterministic disturbances. They found an explicit form for a minimax linear affine solution d* of that problem. In the paper we generalize this result proving that the decision rule d* is also minimax when the class D{\mathcal{D}} of possible predictors of the dependent variable is unrestricted. Then we show that d* remains minimax in D{\mathcal{D}} when the disturbances are random with the mean vector zero and the known positive definite covariance matrix.     

On the Loss Robustness of Least-Square Estimators

Abstract

The article revisits univariate and multivariate linear regression models. It is shown that least-square estimators (LSEs) are minimum risk estimators in general class of linear unbiased estimators under some general divergence loss. This amounts to the loss robustness of LSEs.     

Estimation of impulse response functions in two-output systems

Abstract

A single input–double output (SIDO) linear time-invariant (LTI) system is considered, whose impulse response function (IRF) is assumed to have one unknown component. The problem is to estimate this unknown component after observations of the second component. Both IRF’s components are supposed to be L₂-integrable, and the estimation is made by cross-correlating the outputs, given that the input is a standard Wiener process on R. Weak asymptotic normality of appropriately centred estimators is proved.     

SDP vs. LP Relaxations for the Moment Approach in Some Performance Evaluation Problems

Abstract

Given a Markov process, we are interested in the numerical computation of the moments of the exit time from a bounded domain. We use a moment approach which, together with appropriate semidefinite positivity moment conditions, yields a sequence of semidefinite programs (or SDP relaxations), depending on the number of moments considered, that provide a sequence of nonincreasing (resp. nondecreasing) upper (resp. lower) bounds. The results are compared to the linear Hausdorff moment conditions approach considered for the LP relaxations in Helmes et al. [Helmes, K., Röhl, S., Stockbridge, R.H. Computing moments of the exit time distribution for Markov processes by linear programming. Oper. Res. 2001, 49, 516–530]. The SDP relaxations are shown to be more general and more precise than the LP relaxations.     

Improved ridge estimators in a linear regression model

In this paper, the notion of the improved ridge estimator (IRE) is put forward in the linear regression model y=X β+e. The problem arises if augmenting the equation 0=c′α+ε instead of 0=C α+? to the model. Three special IREs are considered and studied under the mean-squared error criterion and the prediction error sum of squares criterion. The simulations demonstrate that the proposed estimators are effective and recommendable, especially when multicollinearity is severe.     

Comparing the BLUEs Under Two Linear Models

In this article, we consider two linear models, ?₁ = {y, X β, V ₁} and ?₂ = {y, X β, V ₂}, which differ only in their covariance matrices. Our main focus lies on the difference of the best linear unbiased estimators, BLUEs, of X β under these models. The corresponding problems between the models {y, X β, I _n } and {y, X β, V}, i.e., between the OLSE (ordinary least squares estimator) and BLUE, are pretty well studied. Our purpose is to review the corresponding considerations between the BLUEs of X β under ?₁ and ?₂. This article is an expository one presenting also new results.