A Comparison of REML and Covariance Adjustment Method in the Estimation of Growth Curve Models

The classical growth curve model is considered when one continuous characteristic is measured at q time points. The covariance adjusted estimator of growth curve parameters is the OLS estimator adjusted using analysis of covariance. The covariates are obtained from functions of within individuals error contrasts. On the other hand, REML estimators emerge from maximization of the likelihood of OLS residuals. We compare the efficiency of estimators of growth curve parameters obtained by REML with that of covariance-adjusted least squares estimators with covariates selected via CAIC.     

Approaches to regression analysis with multiple measurements from individual sampling units

Application of ordinary least-squares regression to data sets which contain multiple measurements from individual sampling units produces an unbiased estimator of the parameters but a biased estimator of the covariance matrix of the parameter estimates. The present work considers a random coefficient, linear model to deal with such data sets: this model permits many senses in which multiple measurements are taken from a sampling unit, not just when it is measured at several times. Three procedures to estimate the covariance matrix of the error term of the model are considered. Given these, three procedures to estimate the parameters of the model and their covariance matrix are considered; these are ordinary least-squares, generalized least-squares, and an adjusted ordinary least-squares procedure which produces an unbiased estimator of the covariance matrix of the parameters with small samples. These various procedures are compared in simulation studies using three examples from the biological literature. The possibility of testing hypotheses about the vector of parameters is also considered. It is found that all three procedures for regression estimation produce estimators of the parameters with bias of no practical consequence, Both generalized least-squares and adjusted ordinary least-squares generally produce estimators of the covariance matrix of the parameter estimates with bias of no practical consequence, while ordinary least-squares produces a negatively biased estimator. Neither ordinary nor generalized least-squares provide satisfactory hypothesis tests of the vector of parameter estimates. It is concluded that adjusted ordinary least-squares, when applied with either of two of the procedures used to estimate the error coveriance matrix, shows promise for practical application with data sets of the nature considered here.     

Bayesian analysis of covariance under inverse Gaussian model

This paper considers the problem of analysis of covariance (ANCOVA) under the assumption of inverse Gaussian distribution for response variable from the Bayesian point of view. We develop a fully Bayesian model for ANCOVA based on the conjugate prior distributions for parameters contained in the model. The Bayes estimator of parameters, ANCOVA model and adjusted effects for both treatments and covariates along with predictive distribution of future observations are developed. We also provide the essentials for comparing adjusted treatments effects and adjusted factor effects. A simulation study and a real world application are also performed to illustrate and evaluate the proposed Bayesian model.     

Unbiased Estimator for a Covariance Matrix Under Two-Step Monotone Incomplete Sample

In this article, we consider an inference for a covariance matrix under two-step monotone incomplete sample. The maximum likelihood estimator of the mean vector is unbiased but that of the covariance matrix is biased. We derive an unbiased estimator for the covariance matrix using some fundamental properties of the Wishart matrix. The properties of the estimators are investigated and the accuracies are checked by a numerical simulation.     

Heteroskedastic linear regression model with compositional response and covariates

Compositional data are known as a sort of complex multidimensional data with the feature that reflect the relative information rather than absolute information. There are a variety of models for regression analysis with compositional variables. Similar to the traditional regression analysis, the heteroskedasticity still exists in these models. However, the existing heteroskedastic regression analysis methods cannot apply in these models with compositional error term. In this paper, we mainly study the heteroskedastic linear regression model with compositional response and covariates. The parameter estimator is obtained through weighted least squares method. For the hypothesis test of parameter, the test statistic is based on the original least squares estimator and corresponding heteroskedasticity-consistent covariance matrix estimator. When the proposed method is applied to both simulation and real example, we use the original least squares method as a comparison during the whole process. The results implicate the model's practicality and effectiveness in regression analysis with heteroskedasticity.     

Weighted quantile regression with missing covariates using empirical likelihood

This paper proposes an empirical likelihood-based weighted (ELW) quantile regression approach for estimating the conditional quantiles when some covariates are missing at random. The proposed ELW estimator is computationally simple and achieves semiparametric efficiency if the probability of missingness is correctly specified. The limiting covariance matrix of the ELW estimator can be estimated by a resampling technique, which does not involve nonparametric density estimation or numerical derivatives. Simulation results show that the ELW method works remarkably well in finite samples. A real data example is used to illustrate the proposed ELW method.     

Parameter estimation in regression for long-term survival rate from censored data

In recent years, regression models have been shown to be useful for predicting the long-term survival probabilities of patients in clinical trials. The importance of a regression model is that once the regression parameters are estimated information about the regressed quantity is immediate. A simple estimator is proposed for the regression parameters in a model for the long-term survival rate. The proposed estimator is seen to arise from an estimating function that has the missing information principle underlying its construction. When the covariate takes values in a finite set, the proposed estimating function is equivalent to an ad hoc estimating function proposed in the literature. However, in general, the two estimating functions lead to different estimators of the regression parameter. For discrete covariates, the asymptotic covariance matrix of the proposed estimator is simple to calculate using standard techniques involving the predictable covariation process of martingale transforms. An ad hoc extension to the case of a one-dimensional continuous covariate is proposed. Simplicity and generalizability are two attractive features of the proposed approach. The last mentioned feature is not enjoyed by the other estimator.     

Robust estimation of mean and covariance for longitudinal data with dropouts

In this paper, we study estimation of linear models in the framework of longitudinal data with dropouts. Under the assumptions that random errors follow an elliptical distribution and all the subjects share the same within-subject covariance matrix which does not depend on covariates, we develop a robust method for simultaneous estimation of mean and covariance. The proposed method is robust against outliers, and does not require to model the covariance and missing data process. Theoretical properties of the proposed estimator are established and simulation studies show its good performance. In the end, the proposed method is applied to a real data analysis for illustration.     

LEVERAGE ADJUSTMENTS FOR DISPERSION MODELLING IN GENERALIZED NONLINEAR MODELS

For normal linear models, it is generally accepted that residual maximum likelihood estimation is appropriate when covariance components require estimation. This paper considers generalized linear models in which both the mean and the dispersion are allowed to depend on unknown parameters and on covariates. For these models there is no closed form equivalent to residual maximum likelihood except in very special cases. Using a modified profile likelihood for the dispersion parameters, an adjusted score vector and adjusted information matrix are found under an asymptotic development that holds as the leverages in the mean model become small. Subsequently, the expectation of the fitted deviances is obtained directly to show that the adjusted score vector is unbiased at least to O(1/n) . Exact results are obtained in the single‐sample case. The results reduce to residual maximum likelihood estimation in the normal linear case.     

Regularized parameter estimation of high dimensional t distribution

We propose penalized-likelihood methods for parameter estimation of high dimensional t distribution. First, we show that a general class of commonly used shrinkage covariance matrix estimators for multivariate normal can be obtained as penalized-likelihood estimator with a penalty that is proportional to the entropy loss between the estimate and an appropriately chosen shrinkage target. Motivated by this fact, we then consider applying this penalty to multivariate t distribution. The penalized estimate can be computed efficiently using EM algorithm for given tuning parameters. It can also be viewed as an empirical Bayes estimator. Taking advantage of its Bayesian interpretation, we propose a variant of the method of moments to effectively elicit the tuning parameters. Simulations and real data analysis demonstrate the competitive performance of the new methods.     

Cox regression with missing covariate data using a modified partial likelihood method

Missing covariate values is a common problem in survival analysis. In this paper we propose a novel method for the Cox regression model that is close to maximum likelihood but avoids the use of the EM-algorithm. It exploits that the observed hazard function is multiplicative in the baseline hazard function with the idea being to profile out this function before carrying out the estimation of the parameter of interest. In this step one uses a Breslow type estimator to estimate the cumulative baseline hazard function. We focus on the situation where the observed covariates are categorical which allows us to calculate estimators without having to assume anything about the distribution of the covariates. We show that the proposed estimator is consistent and asymptotically normal, and derive a consistent estimator of the variance–covariance matrix that does not involve any choice of a perturbation parameter. Moderate sample size performance of the estimators is investigated via simulation and by application to a real data example.     

Robust estimation of a high-dimensional integrated covariance matrix

In this article, we consider a robust method of estimating a realized covariance matrix calculated as the sum of cross products of intraday high-frequency returns. According to recent articles in financial econometrics, the realized covariance matrix is essentially contaminated with market microstructure noise. Although techniques for removing noise from the matrix have been studied since the early 2000s, they have primarily investigated a low-dimensional covariance matrix with statistically significant sample sizes. We focus on noise-robust covariance estimation under converse circumstances, that is, a high-dimensional covariance matrix possibly with a small sample size. For the estimation, we utilize a statistical hypothesis test based on the characteristic that the largest eigenvalue of the covariance matrix asymptotically follows a Tracy–Widom distribution. The null hypothesis assumes that log returns are not pure noises. If a sample eigenvalue is larger than the relevant critical value, then we fail to reject the null hypothesis. The simulation results show that the estimator studied here performs better than others as measured by mean squared error. The empirical analysis shows that our proposed estimator can be adopted to forecast future covariance matrices using real data.     

More Efficient Tests Robust to Heteroskedasticity of Unknown Form

In the presence of heteroskedasticity of unknown form, the Ordinary Least Squares parameter estimator becomes inefficient, and its covariance matrix estimator inconsistent. Eicker (1963) and White (1980) were the first to propose a robust consistent covariance matrix estimator, that permits asymptotically correct inference. This estimator is widely used in practice. Cragg (1983) proposed a more efficient estimator, but concluded that tests basd on it are unreliable. Thus, this last estimator has not been used in practice. This article is concerned with finite sample properties of tests robust to heteroskedasticity of unknown form. Our results suggest that reliable and more efficient tests can be obtained with the Cragg estimators in small samples.     

The Asymptotic Covariance Matrix of the Least Squares Estimator in the Stochastic Linear Regression Model: The Case of Elliptically Symmetric Distribution

This article considers the unconditional asymptotic covariance matrix of the least squares estimator in the linear regression model with stochastic explanatory variables. The asymptotic covariance matrix of the least squares estimator of regression parameters is evaluated relative to the standard asymptotic covariance matrix when the joint distribution of the dependent and explanatory variables is in the class of elliptically symmetric distributions. An empirical example using financial data is presented. Numerical examples and simulation experiments are given to illustrate the difference of the two asymptotic covariance matrices.     

Matrix trace Wielandt inequalities with statistical applications

The vector correlation coefficient and other measures of association play a very important role in statistics and especially in multivariate analysis. In this paper a new measure of association is proposed and its upper bound is presented by using a matrix trace Wielandt inequality. Also given are relevant results involving Wishart matrices widely used in multivariate analysis, and especially a new alternative for the relative gain of the covariance adjusted estimator of a vector of parameters.     

Selection Strategy for Covariance Structure of Random Effects in Linear Mixed‐effects Models

Linear mixed‐effects models are a powerful tool for modelling longitudinal data and are widely used in practice. For a given set of covariates in a linear mixed‐effects model, selecting the covariance structure of random effects is an important problem. In this paper, we develop a joint likelihood‐based selection criterion. Our criterion is the approximately unbiased estimator of the expected Kullback–Leibler information. This criterion is also asymptotically optimal in the sense that for large samples, estimates based on the covariance matrix selected by the criterion minimize the approximate Kullback–Leibler information. Finite sample performance of the proposed method is assessed by simulation experiments. As an illustration, the criterion is applied to a data set from an AIDS clinical trial.     

On estimating a function of the mean of a multivariate normal distribution

The unique minimum variance of unbiased estimator is obtained for analysis functions of the mean of a multivariate normal distribution with either unknown covariance matrix or with covariance matrix of the form σ²v where σ² is unknown.     

Regularized covariance matrix estimation under the common principal components model

The common principal components (CPC) model provides a way to model the population covariance matrices of several groups by assuming a common eigenvector structure. When appropriate, this model can provide covariance matrix estimators of which the elements have smaller standard errors than when using either the pooled covariance matrix or the per group unbiased sample covariance matrix estimators. In this article, a regularized CPC estimator under the assumption of a common (or partially common) eigenvector structure in the populations is proposed. After estimation of the common eigenvectors using the Flury–Gautschi (or other) algorithm, the off-diagonal elements of the nearly diagonalized covariance matrices are shrunk towards zero and multiplied with the orthogonal common eigenvector matrix to obtain the regularized CPC covariance matrix estimates. The optimal shrinkage intensity per group can be estimated using cross-validation. The efficiency of these estimators compared to the pooled and unbiased estimators is investigated in a Monte Carlo simulation study, and the regularized CPC estimator is applied to a real dataset to demonstrate the utility of the method.     

Calculating the power or sample size for the logistic and proportional hazards models

An algorithm is presented for calculating the power for the logistic and proportional hazards models in which some of the covariates are discrete and the remainders are multivariate normal. The mean and covariance matrix of the multivariate normal covariates may depend on the discrete covariates.

The algorithm, which finds the power of the Wald test, uses the result that the information matrix can be calculated using univariate numerical integration even when there are several continuous covariates. The algorithm is checked using simulation and in certain situations gives more accurate results than current methods which are based on simple formulae. The algorithm is used to explore properties of these models, in particular, the power gain from a prognostic covariate in the analysis of a clinical trial or observational study. The methods can be extended to determine power for other generalized linear models.