Optimal prediction in finite populations under matrix loss

Optimal prediction problems in finite population are investigated. Under matrix loss, we provide necessary and sufficient conditions for the linear predictor of a general linearly predictable variable to be the best linear unbiased predictor (BLUP). The essentially unique BLUP of a linearly predictable variable is obtained in the general superpopulation model. Surprisingly, the both BLUPs under matrix and quadratic loss functions are equivalent to each other. Next, we prove that the BLUP is admissible in the class of linear predictors. Conditions for optimality of the simple projection predictor (SPP) are given. Furthermore, the robust SPP and the robust BLUP are characterized on the misspecification of the covariance matrix.     

Random-effect models with singular precision

We show that smoothing spline, intrinsic autoregression (IAR) and state-space model can be formulated as partially specified random-effect model with singular precision (SP). Various fitting methods have been suggested for the aforementioned models and this paper investigates the relationships among them, once the models have been placed under a single framework. Some methods have been previously shown to give the best linear unbiased predictors (BLUPs) under some random-effect models and here we show that they are in fact uniformly BLUPs (UBLUPs) under a class of models that are generated by the SP of random effects. We offer some new interpretations of the UBLUPs under models of SP and define BLUE and BLUP in these partially specified models without having to specify the covariance. We also show how the full likelihood inferences for random-effect models can be made for these models, so that the maximum likelihood (ML) and restricted maximum likelihood (REML) estimators can be used for the smoothing parameters in splines, etc.     

A prediction analysis in a constrained multivariate general linear model with future observations

Abstract

We give a mathematical analysis to some fundamental prediction problems on a constrained multivariate general linear model (CMGLM) with future observations, including the derivation of analytical formulas for calculating the best linear unbiased predictors (BLUPs) of all unknown parameter matrices, and the presentation of many novel and valuable properties of the BLUPs.     

Predicting random effects with an expanded finite population mixed model

Prediction of random effects is an important problem with expanding applications. In the simplest context, the problem corresponds to prediction of the latent value (the mean) of a realized cluster selected via two-stage sampling. Recently, Stanek and Singer [Predicting random effects from finite population clustered samples with response error. J. Amer. Statist. Assoc. 99, 119–130] developed best linear unbiased predictors (BLUP) under a finite population mixed model that outperform BLUPs from mixed models and superpopulation models. Their setup, however, does not allow for unequally sized clusters. To overcome this drawback, we consider an expanded finite population mixed model based on a larger set of random variables that span a higher dimensional space than those typically applied to such problems. We show that BLUPs for linear combinations of the realized cluster means derived under such a model have considerably smaller mean squared error (MSE) than those obtained from mixed models, superpopulation models, and finite population mixed models. We motivate our general approach by an example developed for two-stage cluster sampling and show that it faithfully captures the stochastic aspects of sampling in the problem. We also consider simulation studies to illustrate the increased accuracy of the BLUP obtained under the expanded finite population mixed model.     

On f-tests and linear hypothesis in a general linear model

A simple method of setting linear hypotheses testable by F-tests in a general linear model when the covariance matrix has a general form and is completely unknown, is provided. With some additional conditions imposed on the covariance matrix, there exist the UMP invariant tests of certain linear hypotheses. We derive them to compare the powers with those of F-tests obtained under no restrictions on the covariance matrix. The results are illustrated in a multiple regression model with some examples.     

Generalized Estimation of the BLUP in Mixed-Effects Models: A Comparison with ML and REML

The Best Linear Unbiased Predictor (BLUP) in mixed models is a function of the variance components and they are estimated using maximum likelihood (ML) or restricted ML methods. Nonconvergence of BLUP would occur due to a drawback of the standard likelihood-based approaches. In such situations, ML and REML either do not provide any BLUPs or all become equal. To overcome this drawback, we provide a generalized estimate (GE) of BLUP that does not suffer from the problem of negative or zero variance components, and compare its performance against the ML and REML estimates of BLUP. Simulated and published data are used to compare BLUP.     

Admissibilities of matrix linear estimators multivariate linear models

This article respectively provides sufficient conditions and necessary conditions of matrix linear estimators of an estimable parameter matrix linear function in multivariate linear models with and without the assumption that the underlying distribution is a normal one with completely unknown covariance matrix. In the latter model, a necessary and sufficient condition is given for matrix linear estimators to be admissible in the space of all matrix linear estimators under each of three different kinds of quadratic matrix loss functions, respectively. In the former model, a sufficient condition is first provided for matrix linear estimators to be admissible in the space of all matrix estimators having finite risks under each of the same loss functions, respectively. Furthermore in the former model, one of these sufficient conditions, correspondingly under one of the loss functions, is also proved to be necessary, if additional conditions are assumed.     

New HEAVY Models for Fat-Tailed Realized Covariances and Returns

ABSTRACT

We develop a new score-driven model for the joint dynamics of fat-tailed realized covariance matrix observations and daily returns. The score dynamics for the unobserved true covariance matrix are robust to outliers and incidental large observations in both types of data by assuming a matrix-F distribution for the realized covariance measures and a multivariate Student's t distribution for the daily returns. The filter for the unknown covariance matrix has a computationally efficient matrix formulation, which proves beneficial for estimation and simulation purposes. We formulate parameter restrictions for stationarity and positive definiteness. Our simulation study shows that the new model is able to deal with high-dimensional settings (50 or more) and captures unobserved volatility dynamics even if the model is misspecified. We provide an empirical application to daily equity returns and realized covariance matrices up to 30 dimensions. The model statistically and economically outperforms competing multivariate volatility models out-of-sample. Supplementary materials for this article are available online.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

Orthodox BLUP versus h-likelihood methods for inferences about random effects in Tweedie mixed models

Recently, the orthodox best linear unbiased predictor (BLUP) method was introduced for inference about random effects in Tweedie mixed models. With the use of h-likelihood, we illustrate that the standard likelihood procedures, developed for inference about fixed unknown parameters, can be used for inference about random effects. We show that the necessary standard error for the prediction interval of the random effect can be computed from the Hessian matrix of the h-likelihood. We also show numerically that the h-likelihood provides a prediction interval that maintains a more precise coverage probability than the BLUP method.     

On MSE of EBLUP

We consider Best Linear Unbiased Predictors (BLUPs) and Empirical Best Linear Unbiased Predictors (EBLUPs) under the general mixed linear model. The BLUP was proposed by Henderson (Ann Math Stat 21:309–310, 1950). The formula of this BLUP includes unknown elements of the variance-covariance matrix of random variables. If the elements in the formula of the BLUP proposed by Henderson (Ann Math Stat 21:309–310, 1950) are replaced by some type of estimators, we obtain the two-stage predictor called the EBLUP which is model-unbiased (Kackar and Harville in Commun Stat A 10:1249–1261, 1981). Kackar and Harville (J Am Stat Assoc 79:853–862, 1984) show an approximation of the mean square error (the MSE) of the predictor and propose an estimator of the MSE. The MSE and estimators of the MSE are also studied by Prasad and Rao (J Am Stat Assoc 85:163–171, 1990), Datta and Lahiri (Stat Sin 10:613–627, 2000) and Das et al. (Ann Stat 32(2):818–840, 2004). In the paper we consider the BLUP proposed by Royall (J Am Stat Assoc 71:657–473, 1976. Ża̧dło (On unbiasedness of some EBLU predictor. Physica-Verlag, Heidelberg, pp 2019–2026, 2004) shows that the BLUP proposed by Royall (J Am Stat Assoc 71:657–473, 1976) may be treated as a generalisation of the BLUP proposed by Henderson (Ann Math Stat 21:309–310, 1950) and proves model unbiasedness of the EBLUP based on the formula of the BLUP proposed by Royall (J Am Stat Assoc 71:657–473, 1976) under some assumptions. In this paper we derive the formula of the approximate MSE of the EBLUP and its estimators. We prove that the approximation of the MSE is accurate to terms o(D ⁻¹) and that the estimator of the MSE is approximately unbiased in the sense that its bias is o(D ⁻¹) under some assumptions, where D is the number of domains. The proof is based on the results obtained by Datta and Lahiri (Stat Sin 10:613–627, 2000). Using our results we show some EBLUP based on the special case of the general linear model. We also present the formula of its MSE and estimators of its MSE and their performance in Monte Carlo simulation study.      

On the natural restrictions in the singular Gauss–Markov model

A Gauss–Markov model is said to be singular if the covariance matrix of the observable random vector in the model is singular. In such a case, there exist some natural restrictions associated with the observable random vector and the unknown parameter vector in the model. In this paper, we derive through the matrix rank method a necessary and sufficient condition for a vector of parametric functions to be estimable, and necessary and sufficient conditions for a linear estimator to be unbiased in the singular Gauss–Markov model. In addition, we give some necessary and sufficient conditions for the ordinary least-square estimator (OLSE) and the best linear unbiased estimator (BLUE) under the model to satisfy the natural restrictions.      

Fundamental Equations of BLUE and BLUP in the Multivariate Linear Model with Applications

Ordinary least squares estimator (OLSE), best linear unbiased estimator (BLUE), and best linear unbiased predictor (BLUP) in the general linear model with new observations are generalized to the general multivariate linear model. The fundamental equations of BLUE and BLUP in the multivariate linear model are derived by two methods, including the vectorization method and projection method. By using the matrix rank method, some new results of linear BLUE-sufficiency, linear BLUP-sufficiency, and the equality of OLSE, BLUE, and BLUP are given in the multivariate linear model.     

ON UMP INVARIANT F-TEST PROCEDURES IN A GENERAL LINEAR MODEL

A simple method of setting linear hypotheses for a split mean vector testable by F-tests in a general linear model, when the covariance matrix has a general form and is completely unknown, is provided by extending the method discussed in Ukita et al. The critical functions in these F-tests are constructed as UMP invariants, when the covariance matrix has a known structure. Further critical functions in F-tests of linear hypotheses for the other split mean vector in the model are shown to be UMP invariant if the same known structure of the covariance matrix is assumed.     

Transformation approaches of linear random-effects models

Assume that a linear random-effects model \(\mathbf{y}= \mathbf{X}\varvec{\beta }+ \varvec{\varepsilon }= \mathbf{X}(\mathbf{A}\varvec{\alpha }+ \varvec{\gamma }) + \varvec{\varepsilon }\) is transformed as \(\mathbf{T}\mathbf{y}= \mathbf{T}\mathbf{X}\varvec{\beta }+ \mathbf{T}\varvec{\varepsilon }= \mathbf{T}\mathbf{X}(\mathbf{A}\varvec{\alpha }+ \varvec{\gamma }) + \mathbf{T}\varvec{\varepsilon }\) by pre-multiplying a given matrix \(\mathbf{T}\) of arbitrary rank. These two models are not necessarily equivalent unless \(\mathbf{T}\) is of full column rank, and we have to work with this derived model in many situations. Because predictors/estimators of the parameter spaces under the two models are not necessarily the same, it is primary work to compare predictors/estimators in the two models and to establish possible links between the inference results obtained from two models. This paper presents a general algebraic approach to the problem of comparing best linear unbiased predictors (BLUPs) of parameter spaces in an original linear random-effects model and its transformations, and provides a group of fundamental and comprehensive results on mathematical and statistical properties of the BLUPs. In particular, we construct many equalities for the BLUPs under an original linear random-effects model and its transformations, and obtain necessary and sufficient conditions for the equalities to hold.     

Checking identifiability of covariance parameters in linear mixed effects models

To build a linear mixed effects model, one needs to specify the random effects and often the associated parametrized covariance matrix structure. Inappropriate specification of the structures can result in the covariance parameters of the model not identifiable. Non-identifiability can result in extraordinary wide confidence intervals, and unreliable parameter inference. Sometimes software produces implication of model non-identifiability, but not always. In the simulation of fitting non-identifiable models we tried, about half of the times the software output did not look abnormal. We derive necessary and sufficient conditions of covariance parameters identifiability which does not require any prior model fitting. The results are easy to implement and are applicable to commonly used covariance matrix structures.     

OPTIMAL QUADRATIC UNBIASED ESTIMATION FOR MODELS WITH LINEAR TOEPLITZ COVARIANCE STRUCTURE

This paper deals with the problem of quadratic unbiased estimation for models with linear Toeplitz covariance structure. These serial covariance models are very useful to modelize time or spatial correlations by means of linear models. Optimality and local optimality is examined in different ways. For the nested Toeplitz models, it is shown that there does not exist a Uniformly Minimum Variance Quadratic Unbiased Estimator for at least one linear combination of covariance parameters. Moreover, empirical unbiased estimators are identified as Locally Minimum Variance Quadratic Unbiased Estimators for a particular choice on covariance parameters corresponding to the case where the covariance matrix of the observed random vector is proportional to the identity matrix. The complete Toeplitz-circulant model is also studied. For this model, the existence of a Uniformly Minimum Variance Quadratic Unbiased Estimator for each covariance parameter is proved.     

Computation aspects of the parameter estimates of linear mixed effects model in multivariate repeated measures set-up

The number of parameters mushrooms in a linear mixed effects (LME) model in the case of multivariate repeated measures data. Computation of these parameters is a real problem with the increase in the number of response variables or with the increase in the number of time points. The problem becomes more intricate and involved with the addition of additional random effects. A multivariate analysis is not possible in a small sample setting. We propose a method to estimate these many parameters in bits and pieces from baby models, by taking a subset of response variables at a time, and finally using these bits and pieces at the end to get the parameter estimates for the mother model, with all variables taken together. Applying this method one can calculate the fixed effects, the best linear unbiased predictions (BLUPs) for the random effects in the model, and also the BLUPs at each time of observation for each response variable, to monitor the effectiveness of the treatment for each subject. The proposed method is illustrated with an example of multiple response variables measured over multiple time points arising from a clinical trial in osteoporosis.     

Spatial sampling design for parameter estimation of the covariance function

We study the spatial optimal sampling design for covariance parameter estimation. The spatial process is modeled as a Gaussian random field and maximum likelihood (ML) is used to estimate the covariance parameters. We use the log determinant of the inverse Fisher information matrix as the design criterion and run simulations to investigate the relationship between the inverse Fisher information matrix and the covariance matrix of the ML estimates. A simulated annealing algorithm is developed to search for an optimal design among all possible designs on a fine grid. Since the design criterion depends on the unknown parameters, we define relative efficiency of a design and consider minimax and Bayesian criteria to find designs that are robust for a range of parameter values. Simulation results are presented for the Matérn class of covariance functions.