Reml estimation for repeated measures analysis

Models for repeated measures or growth curves consist of a mean response plus error and the errors are usually correlated. Both maximum likelihood and residual maximum likelihood (REML) estimators of a regression model with dependent errors are derived for cases in which the variance matrix of the error model admits a convenient Cholesky factorisation. This factorisation may be linked to methods for producing recursive estimates of the regression parameters and recursive residuals to provide a convenient computational method. The method is used to develop a general approach to repeated measures analysis.     

Editorial collaborators

A simulation study of the binomial-logit model with correlated random effects is carried out based on the generalized linear mixed model (GLMM) methodology. Simulated data with various numbers of regression parameters and different values of the variance component are considered. The performance of approximate maximum likelihood (ML) and residual maximum likelihood (REML) estimators is evaluated. For a range of true parameter values, we report the average biases of estimators, the standard error of the average bias and the standard error of estimates over the simulations. In general, in terms of bias, the two methods do not show significant differences in estimating regression parameters. The REML estimation method is slightly better in reducing the bias of variance component estimates.     

Estimation of a linear transformation and an associated distributional problem

A multivariate “errors in variables” regression model is proposed which generalizes a model previously considered by Gleser and Watson (1973). Maximum likelihood estimators [MLE's] for the parameters of this model are obtained, and the consistency properties of these estimators are investigated. Distribution of the MLE of the “error” variance is obtained in a simple case while the mean and the variance of the estimator are obtained in this case without appealing to the exact distribution.     

Parameter estimation in semi-linear models using a maximal invariant likelihood function

In this paper, we consider the problem of estimation of semi-linear regression models. Using invariance arguments, Bhowmik and King [2007. Maximal invariant likelihood based testing of semi-linear models. Statist. Papers 48, 357–383] derived the probability density function of the maximal invariant statistic for the non-linear component of these models. Using this density function as a likelihood function allows us to estimate these models in a two-step process. First the non-linear component parameters are estimated by maximising the maximal invariant likelihood function. Then the non-linear component, with the parameter values replaced by estimates, is treated as a regressor and ordinary least squares is used to estimate the remaining parameters. We report the results of a simulation study conducted to compare the accuracy of this approach with full maximum likelihood and maximum profile-marginal likelihood estimation. We find maximising the maximal invariant likelihood function typically results in less biased and lower variance estimates than those from full maximum likelihood.     

Bayesian shrinkage estimates for regression coefficients in m populations

For a linear regression model over m populations with separate regression coefficients but a common error variance, a Bayesian model is employed to obtain regression coefficient estimates which are shrunk toward an overall value. The formulation uses Normal priors on the coefficients and diffuse priors on the grand mean vectors, the error variance, and the between-to-error variance ratios. The posterior density of the parameters which were given diffuse priors is obtained. From this the posterior means and variances of regression coefficients and the predictive mean and variance of a future observation are obtained directly by numerical integration in the balanced case, and with the aid of series expansions in the approximately balanced case. An example is presented and worked out for the case of one predictor variable. The method is an extension of Box & Tiao's Bayesian estimation of means in the balanced one-way random effects model.     

Linear regression through the origin with constant coefficient of variation for the inverse gaussian distribution

A simple linear regression model with no intercept term for the situation where the response variable obeys an inverse Gaussian distribution and the coefficient of variation is an unknown constant is discussed. Maximum likelihood estimators and the confidence limits of the regression parameter are obtained. Finally uniformly minimum variance unbiased estimators of parameters are given.     

New Shrinkage Parameters for the Liu-type Logistic Estimators

The binary logistic regression is a widely used statistical method when the dependent variable has two categories. In most of the situations of logistic regression, independent variables are collinear which is called the multicollinearity problem. It is known that multicollinearity affects the variance of maximum likelihood estimator (MLE) negatively. Therefore, this article introduces new shrinkage parameters for the Liu-type estimators in the Liu (2003) in the logistic regression model defined by Huang (2012) in order to decrease the variance and overcome the problem of multicollinearity. A Monte Carlo study is designed to show the goodness of the proposed estimators over MLE in the sense of mean squared error (MSE) and mean absolute error (MAE). Moreover, a real data case is given to demonstrate the advantages of the new shrinkage parameters.     

Accounting for Uncertainty in Heteroscedasticity in Nonlinear Regression

Toxicologists and pharmacologists often describe toxicity of a chemical using parameters of a nonlinear regression model. Thus estimation of parameters of a nonlinear regression model is an important problem. The estimates of the parameters and their uncertainty estimates depend upon the underlying error variance structure in the model. Typically, a priori the researcher would not know if the error variances are homoscedastic (i.e., constant across dose) or if they are heteroscedastic (i.e., the variance is a function of dose). Motivated by this concern, in this paper we introduce an estimation procedure based on preliminary test which selects an appropriate estimation procedure accounting for the underlying error variance structure. Since outliers and influential observations are common in toxicological data, the proposed methodology uses M-estimators. The asymptotic properties of the preliminary test estimator are investigated; in particular its asymptotic covariance matrix is derived. The performance of the proposed estimator is compared with several standard estimators using simulation studies. The proposed methodology is also illustrated using a data set obtained from the National Toxicology Program.     

Random effects regression mixtures for analyzing infant habituation

Random effects regression mixture models are a way to classify longitudinal data (or trajectories) having possibly varying lengths. The mixture structure of the traditional random effects regression mixture model arises through the distribution of the random regression coefficients, which is assumed to be a mixture of multivariate normals. An extension of this standard model is presented that accounts for various levels of heterogeneity among the trajectories, depending on their assumed error structure. A standard likelihood ratio test is presented for testing this error structure assumption. Full details of an expectation-conditional maximization algorithm for maximum likelihood estimation are also presented. This model is used to analyze data from an infant habituation experiment, where it is desirable to assess whether infants comprise different populations in terms of their habituation time.     

Empirical Likelihood Based Rank Regression Inference

Rank regression procedures have been proposed and studied for numerous research applications that do not satisfy the underlying assumptions of the more common linear regression models. This article develops confidence regions for the slope parameter of rank regression using an empirical likelihood (EL) ratio method. It has the advantage of not requiring variance estimation which is required for the normal approximation method. The EL method is also range respecting and results in asymmetric confidence intervals. Simulation studies are used to compare and evaluate normal approximation versus EL inference methods for various conditions such as different sample size or error distribution. The simulation study demonstrates our proposed EL method almost outperforms the traditional method in terms of coverage probability, lower-tail side error, and upper-tail side error. An application of stability analysis also shows the EL method results in shorter confidence intervals for real life data.     

Target estimation for the logistic regression model

The method of target estimation developed by Cabrera and Fernholz [(1999). Target estimation for bias and mean square error reduction. The Annals of Statistics, 27(3), 1080–1104.] to reduce bias and variance is applied to logistic regression models of several parameters. The expectation functions of the maximum likelihood estimators for the coefficients in the logistic regression models of one and two parameters are analyzed and simulations are given to show a reduction in both bias and variability after targeting the maximum likelihood estimators. In addition to bias and variance reduction, it is found that targeting can also correct the skewness of the original statistic. An example based on real data is given to show the advantage of using target estimators for obtaining better confidence intervals of the corresponding parameters. The notion of the target median is also presented with some applications to the logistic models.     

Bivariate Weibull regression model based on censored samples

The most natural parametric distribution to consider is the Weibull model because it allows for both the proportional hazard model and accelerated failure time model. In this paper, we propose a new bivariate Weibull regression model based on censored samples with common covariates. There are some interesting biometrical applications which motivate to study bivariate Weibull regression model in this particular situation. We obtain maximum likelihood estimators for the parameters in the model and test the significance of the regression parameters in the model. We present a simulation study based on 1000 samples and also obtain the power of the test statistics.     

Smoothing spline based tests for non-linearity in a partially linear model

This paper deals with testing for non-linearity in a regression model with one possibly non-linear component being estimated non-parametrically using smoothing splines. We propose two new variance–covariance based tests for detecting non-linearity applying a likelihood ratio hypothesis testing approach. The first test is for the inclusion of a possibly non-linear component and the second one is for linearity of a possibly non-linear component. The tests are based on a stochastic model in state space form given by Wahba (J. Roy. Statist. Soc. Ser. B 40 (1978) 364), Wecker and Ansley (J. Amer. Statist. Assoc. 78 (1983) 81) and de Jong and Mazzi (Modeling and smoothing unequally spaced sequence data, University of York and University of British Columbia, Unpublished paper) for which smoothing splines provide an optimal estimate. Pitrun (A smoothing spline approach to non-linear interface for time series, Department of Econometrics and Business Statistics, Monash University, Unpublished Ph.D. thesis) derived the variance–covariance structure of this model, which allows the use of a marginal likelihood approach. This leads naturally to marginal-likelihood based likelihood ratio tests for non-linearity. Small sample properties of the new tests have been investigated via Monte Carlo studies.     

The effects of model mis-specifications in linear measurement error models

In the literature, there are many results on the consequences of mis-specified models for linear models with error in the response only, see, e.g., Seber(1977). There are also discussions of estimation for the model writh errors both in the response and in the predictor variables (called measurement error models; see, e.g., Fuller(1987)). In this paper, we consider the problem of model mis-specification for measurement error models. Only a few special cases have been tackled in the past (Edland, 1996; Carroll and Ruppert, 1996 and Lakshminarayanan Amp; Gunst, 1984); we deal with the situation here in some generality. Results have been obtained as follows: (a) When a model is under-fitted, the estimate of the variance of the measurement error will be asymptotically biased, as will the regression coefficients, and the asymptotic biases in the estimates of the regression coefficients will always exist for under-fitted models. Even orthogonality of the variables in the model will not make the biases vanish. (b)For over-fitting, the estimates of the variances of measurement errors and of the regression coefficients are asymptotically unbiased. However, the variance of the estimated regression coefficients will increase. Over-fitting will cause larger changes in the variances of the estimated parameters in measurement error models than in no measurement error models.     

Focused Information Criterion and Model Averaging in Quantile Regression

In this article, we study model selection and model averaging in quantile regression. Under general conditions, we develop a focused information criterion and a frequentist model average estimator for the parameters in quantile regression model, and examine their theoretical properties. The new procedures provide a robust alternative to the least squares method or likelihood method, and a major advantage of the proposed procedures is that when the variance of random error is infinite, the proposed procedure works beautifully while the least squares method breaks down. A simulation study and a real data example are presented to show that the proposed method performs well with a finite sample and is easy to use in practice.     

Fitting a non-linear model with errors in both variables and its application *

Estimation of the parameters of a non-linear model is considered when both measured variables have random errors. The maximum likelihood estimates with the asymptotic variance and covariance matrix are presented. Real data are used to illustrate the procedure discussed.     

Machine Learning Vasicek Model Calibration with Gaussian Processes

In this article, we calibrate the Vasicek interest rate model under the risk neutral measure by learning the model parameters using Gaussian processes for machine learning regression. The calibration is done by maximizing the likelihood of zero coupon bond log prices, using mean and covariance functions computed analytically, as well as likelihood derivatives with respect to the parameters. The maximization method used is the conjugate gradients. The only prices needed for calibration are zero coupon bond prices and the parameters are directly obtained in the arbitrage free risk neutral measure.     

Orthogonality of the mean and error distribution in generalized linear models

We show that the mean-model parameter is always orthogonal to the error distribution in generalized linear models. Thus, the maximum likelihood estimator of the mean-model parameter will be asymptotically efficient regardless of whether the error distribution is known completely, known up to a finite vector of parameters, or left completely unspecified, in which case the likelihood is taken to be an appropriate semiparametric likelihood. Moreover, the maximum likelihood estimator of the mean-model parameter will be asymptotically independent of the maximum likelihood estimator of the error distribution. This generalizes some well-known results for the special cases of normal, gamma, and multinomial regression models, and, perhaps more interestingly, suggests that asymptotically efficient estimation and inferences can always be obtained if the error distribution is non parametrically estimated along with the mean. In contrast, estimation and inferences using misspecified error distributions or variance functions are generally not efficient.     

A method of selecting the levels of regressor variables in lognormal regression model

Kupper and Meydrech and Myers and Lahoda introduced the mean squared error (MSE) approach to study response surface designs, Duncan and DeGroot derived a criterion for optimality of linear experimental designs based on minimum mean squared error. However, minimization of the MSE of an estimator maxr renuire some knowledge about the unknown parameters. Without such knowledge construction of designs optimal in the sense of MSE may not be possible. In this article a simple method of selecting the levels of regressor variables suitable for estimating some functions of the parameters of a lognormal regression model is developed using a criterion for optimality based on the variance of an estimator. For some special parametric functions, the criterion used here is equivalent to the criterion of minimizing the mean squared error. It is found that the maximum likelihood estimators of a class of parametric functions can be improved substantially (in the sense of MSE) by proper choice of the values of regressor variables. Moreover, our approach is applicable to analysis of variance as well as regression designs.     

Efficient estimation of general linear mixed effects models

An extension of the linear growth curve model (Biometrics 38 (1982) 963) was proposed by Stukel and Demidenko (Biometrics 53 (1997) 720) to study the effects of population covariates on one or more characteristics of the curve, when the characteristics are expressed as linear combinations of the growth curve parameters. In the present paper, this general growth curve model receives a comprehensive theoretical treatment. A two-stage estimator, consisting of a generalized least squares estimator under constraints for the population parameters and a moment estimator for the variance parameters, is developed for application in the non-Gaussian error situation. Two likelihood based estimators, global maximum likelihood and second-stage maximum likelihood, are also developed. It is shown that all three estimators are consistent, asymptotically normally distributed, and efficient, and are equivalent when the number of individuals tends to infinity. An expression for the bias in the estimator of the population parameters is derived under second stage model misspecification. We show that if parameters that are not of primary interest are incorrectly specified, bias may occur in parameters that are of interest using the standard growth curve model. The general growth curve model does not require specification of such nuisance parameters and is robust in terms of bias. The general linear growth curve model is used to study the effects of host sex on pancreatic tumor growth in rats.