A monte carlo comparison of the smoothing,scoring and em algorithms for dispersion matrix estimation with incomplete growth curve data

Incomplete growth curve data often result from missing or mistimed observations in a repeated measures design. Virtually all methods of analysis rely on the dispersion matrix estimates. A Monte Carlo simulation was used to compare three methods of estimation of dispersion matrices for incomplete growth curve data. The three methods were: 1) maximum likelihood estimation with a smoothing algorithm, which finds the closest positive semidefinite estimate of the pairwise estimated dispersion matrix; 2) a mixed effects model using the EM (estimation maximization) algorithm; and 3) a mixed effects model with the scoring algorithm. The simulation included 5 dispersion structures, 20 or 40 subjects with 4 or 8 observations per subject and 10 or 30% missing data. In all the simulations, the smoothing algorithm was the poorest estimator of the dispersion matrix. In most cases, there were no significant differences between the scoring and EM algorithms. The EM algorithm tended to be better than the scoring algorithm when the variances of the random effects were close to zero, especially for the simulations with 4 observations per subject and two random effects.     

Equivalence of maximum likelihood estimation and iterative two-stage estimation for seemingly unrelated regression models

The seemingly unrelated regression model is viewed in the context of repeated measures analysis. Regression parameters and the variance-covariance matrix of the seemingly unrelated regression model can be estimated by using two-stage Aitken estimation. The first stage is to obtain a consistent estimator of the variance-covariance matrix. The second stage uses this matrix to obtain the generalized least squares estimators of the regression parameters. The maximum likelihood (ML) estimators of the regression parameters can be obtained by performing the two-stage estimation iteratively. The iterative two-stage estimation procedure is shown to be equivalent to the EM algorithm (Dempster, Laird, and Rubin, 1977) proposed by Jennrich and Schluchter (1986) and Laird, Lange, and Stram (1987) for repeated measures data. The equivalence of the iterative two-stage estimator and the ML estimator has been previously demonstrated empirically in a Monte Carlo study by Kmenta and Gilbert (1968). It does not appear to be widely known that the two estimators are equivalent theoretically. This paper demonstrates this equivalence.     

Simulations and computations of nonparametric density estimates for the deconvolution problem

The nonparametric density function estimation using sample observations which are contaminated with random noise is studied. The particular form of contamination under consideration is Y = X + Z, where Y is an observable random variableZ is a random noise variable with known distribution, and X is an absolutely continuous random variable which cannot be observed directly. The finite sample size performance of a strongly consistent estimator for the density function of the random variable X is illustrated for different distributions. The estimator uses Fourier and kernel function estimation techniques and allows the user to choose constants which relate to bandwidth windows and limits on integration and which greatly affect the appearance and properties of the estimates. Numerical techniques for computation of the estimated densities and for optimal selection of the constant are given.     

Conditional mode estimation for functional stationary ergodic data with responses missing at random

In this paper, we investigate the asymptotic properties of a non-parametric conditional mode estimation given a functional explanatory variable, when functional stationary ergodic data and missing at random responses are observed. First of all, we establish asymptotic properties for a conditional density estimator from which we derive almost sure convergence (with rate) and asymptotic normality of a conditional mode estimator. This new estimate take into account missing data, and a simulation study is performed to illustrate how this fact allows to get higher predictive performances than those obtained with standard estimates.     

Influence diagnostics on the coefficient of variation of elliptically contoured distributions

In this article, we study the behavior of the coefficient of variation (CV) of a random variable that follows a symmetric distribution in the real line. Specifically, we estimate this coefficient using the maximum-likelihood (ML) method. In addition, we provide asymptotic inference for this parameter, which allows us to contrast hypothesis and construct confidence intervals. Furthermore, we produce influence diagnostics to evaluate the sensitivity of the ML estimate of this coefficient when atypical data are present. Moreover, we illustrate the obtained results by using financial real data. Finally, we carry out a simulation study to detect the potential influence of atypical observations on the ML estimator of the CV of a symmetric distribution. The illustration and simulation demonstrate the robustness of the ML estimation of this coefficient.     

Unbiased Estimator for a Covariance Matrix Under Two-Step Monotone Incomplete Sample

In this article, we consider an inference for a covariance matrix under two-step monotone incomplete sample. The maximum likelihood estimator of the mean vector is unbiased but that of the covariance matrix is biased. We derive an unbiased estimator for the covariance matrix using some fundamental properties of the Wishart matrix. The properties of the estimators are investigated and the accuracies are checked by a numerical simulation.     

Bias corrected MLEs under progressive type-II censoring scheme

ABSTRACT

Censoring frequently occurs in survival analysis but naturally observed lifetimes are not of a large size. Thus, inferences based on the popular maximum likelihood (ML) estimation which often give biased estimates should be corrected in the sense of bias. Here, we investigate the biases of ML estimates under the progressive type-II censoring scheme (pIIcs). We use a method proposed in Efron and Johnstone [Fisher's information in terms of the hazard rate. Technical Report No. 264, January 1987, Stanford University, Stanford, California; 1987] to derive general expressions for bias corrected ML estimates under the pIIcs. This requires derivation of the Fisher information matrix under the pIIcs. As an application, exact expressions are given for bias corrected ML estimates of the Weibull distribution under the pIIcs. The performance of the bias corrected ML estimates and ML estimates are compared by simulations and a real data application.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

Model selection of generalized partially linear models with missing covariates

In this paper, a generalized partially linear model (GPLM) with missing covariates is studied and a Monte Carlo EM (MCEM) algorithm with penalized-spline (P-spline) technique is developed to estimate the regression coefficients and nonparametric function, respectively. As classical model selection procedures such as Akaike's information criterion become invalid for our considered models with incomplete data, some new model selection criterions for GPLMs with missing covariates are proposed under two different missingness mechanism, say, missing at random (MAR) and missing not at random (MNAR). The most attractive point of our method is that it is rather general and can be extended to various situations with missing observations based on EM algorithm, especially when no missing data involved, our new model selection criterions are reduced to classical AIC. Therefore, we can not only compare models with missing observations under MAR/MNAR settings, but also can compare missing data models with complete-data models simultaneously. Theoretical properties of the proposed estimator, including consistency of the model selection criterions are investigated. A simulation study and a real example are used to illustrate the proposed methodology.     

A High‐dimensional Focused Information Criterion

The focused information criterion for model selection is constructed to select the model that best estimates a particular quantity of interest, the focus, in terms of mean squared error. We extend this focused selection process to the high‐dimensional regression setting with potentially a larger number of parameters than the size of the sample. We distinguish two cases: (i) the case where the considered submodel is of low dimension and (ii) the case where it is of high dimension. In the former case, we obtain an alternative expression of the low‐dimensional focused information criterion that can directly be applied. In the latter case, we use a desparsified estimator that allows us to derive the mean squared error of the focus estimator. We illustrate the performance of the high‐dimensional focused information criterion with a numerical study and a real dataset.     

The em algorithm for the quasi-likelihood regression model

The objective of this paper is to present a method which can accommodate certain types of missing data by using the quasi-likelihood function for the complete data. This method can be useful when we can make first and second moment assumptions only; in addition, it can be helpful when the EM algorithm applied to the actual likelihood becomes overly complicated. First we derive a loss function for the observed data using an exponential family density which has the same mean and variance structure of the complete data. This loss function is the counterpart of the quasi-deviance for the observed data. Then the loss function is minimized using the EM algorithm. The use of the EM algorithm guarantees a decrease in the loss function at every iteration. When the observed data can be expressed as a deterministic linear transformation of the complete data, or when data are missing completely at random, the proposed method yields consistent estimators. Examples are given for overdispersed polytomous data, linear random effects models, and linear regression with missing covariates. Simulation results for the linear regression model with missing covariates show that the proposed estimates are more efficient than estimates based on completely observed units, even when outcomes are bimodal or skewed.     

A robust multivariate Birnbaum–Saunders distribution: EM estimation

We propose here a robust multivariate extension of the bivariate Birnbaum–Saunders (BS) distribution derived by Kundu et al. [Bivariate Birnbaum–Saunders distribution and associated inference. J Multivariate Anal. 2010;101:113–125], based on scale mixtures of normal (SMN) distributions that are used for modelling symmetric data. This resulting multivariate BS-type distribution is an absolutely continuous distribution whose marginal and conditional distributions are of BS-type distribution of Balakrishnan et al. [Estimation in the Birnbaum–Saunders distribution based on scalemixture of normals and the EM algorithm. Stat Oper Res Trans. 2009;33:171–192]. Due to the complexity of the likelihood function, parameter estimation by direct maximization is very difficult to achieve. For this reason, we exploit the nice hierarchical representation of the proposed distribution to propose a fast and accurate EM algorithm for computing the maximum likelihood (ML) estimates of the model parameters. We then evaluate the finite-sample performance of the developed EM algorithm and the asymptotic properties of the ML estimates through empirical experiments. Finally, we illustrate the obtained results with a real data and display the robustness feature of the estimation procedure developed here.     

Efficient estimation for incomplete multivariate data

We review the Fisher scoring and EM algorithms for incomplete multivariate data from an estimating function point of view, and examine the corresponding quasi-score functions under second-moment assumptions. A bias-corrected REML-type estimator for the covariance matrix is derived, and the Fisher, Godambe and empirical sandwich information matrices are compared. We make a numerical investigation of the two algorithms, and compare with a hybrid algorithm, where Fisher scoring is used for the mean vector and the EM algorithm for the covariance matrix.     

Heteroscedastic and heavy-tailed regression with mixtures of skew Laplace normal distributions

Joint modelling skewness and heterogeneity is challenging in data analysis, particularly in regression analysis which allows a random probability distribution to change flexibly with covariates. This paper, based on a skew Laplace normal (SLN) mixture of location, scale, and skewness, introduces a new regression model which provides a flexible modelling of location, scale and skewness parameters simultaneously. The maximum likelihood (ML) estimators of all parameters of the proposed model via the expectation-maximization (EM) algorithm as well as their asymptotic properties are derived. Numerical analyses via a simulation study and a real data example are used to illustrate the performance of the proposed model.     

A general maximum likelihood analysis of overdispersion in generalized linear models

This paper presents an EM algorithm for maximum likelihood estimation in generalized linear models with overdispersion. The algorithm is initially derived as a form of Gaussian quadrature assuming a normal mixing distribution, but with only slight variation it can be used for a completely unknown mixing distribution, giving a straightforward method for the fully non-parametric ML estimation of this distribution. This is of value because the ML estimates of the GLM parameters may be sensitive to the specification of a parametric form for the mixing distribution. A listing of a GLIM4 algorithm for fitting the overdispersed binomial logit model is given in an appendix.A simple method is given for obtaining correct standard errors for parameter estimates when using the EM algorithm.Several examples are discussed.     

Estimation of nonlinear CUB models via numerical optimization and EM algorithm

The analysis of human perceptions is often carried out by resorting to surveys and questionnaires, where respondents are asked to express ratings about the objects being evaluated. A class of mixture models, called CUB (Combination of Uniform and shifted Binomial), has been recently proposed in this context. This article focuses on a model of this class, the Nonlinear CUB, and investigates some computational issues concerning parameter estimation, which is performed by Maximum Likelihood. More specifically, we consider two main approaches to optimize the log-likelihood: the classical numerical methods of optimization and the EM algorithm. The classical numerical methods comprise the widely used algorithms Nelder–Mead, Newton–Raphson, Broyden–Fletcher–Goldfarb–Shanno (BFGS), Berndt–Hall–Hall–Hausman (BHHH), Simulated Annealing, Conjugate Gradients and usually have the advantage of a fast convergence. On the other hand, the EM algorithm deserves consideration for some optimality properties in the case of mixture models, but it is slower. This article has a twofold aim: first we show how to obtain explicit formulas for the implementation of the EM algorithm in nonlinear CUB models and we formally derive the asymptotic variance–covariance matrix of the Maximum Likelihood estimator; second, we discuss and compare the performance of the two above mentioned approaches to the log-likelihood maximization.     

Forecasting time series with missing data using Holt's model

This paper deals with the prediction of time series with missing data using an alternative formulation for Holt's model with additive errors. This formulation simplifies both the calculus of maximum likelihood estimators of all the unknowns in the model and the calculus of point forecasts. In the presence of missing data, the EM algorithm is used to obtain maximum likelihood estimates and point forecasts. Based on this application we propose a leave-one-out algorithm for the data transformation selection problem which allows us to analyse Holt's model with multiplicative errors. Some numerical results show the performance of these procedures for obtaining robust forecasts.     

Optimal step-stress test under type I progressive group-censoring with random removals

Some traditional life tests result in no or very few failures by the end of test. In such cases, one approach is to do life testing at higher-than-usual stress conditions in order to obtain failures quickly. This paper discusses a k-level step-stress accelerated life test under type I progressive group-censoring with random removals. An exponential failure time distribution with mean life that is a log-linear function of stress and a cumulative exposure model are considered. We derive the maximum likelihood estimators of the model parameters and establish the asymptotic properties of the estimators. We investigate four selection criteria which enable us to obtain the optimum test plans. One is to minimize the asymptotic variance of the maximum likelihood estimator of the logarithm of the mean lifetime at use-condition, and the other three criteria are to maximize the determinant, trace and the smallest eigenvalue of Fisher's information matrix. Some numerical studies are discussed to illustrate the proposed criteria.     

Convergence properties of the EM algorithm in constrained parameter spaces

The established general results on convergence properties of the EM algorithm require the sequence of EM parameter estimates to fall in the interior of the parameter space over which the likelihood is being maximized. This paper presents convergence properties of the EM sequence of likelihood values and parameter estimates in constrained parameter spaces for which the sequence of EM parameter estimates may converge to the boundary of the constrained parameter space contained in the interior of the unconstrained parameter space. Examples of the behavior of the EM algorithm applied to such parameter spaces are presented.     

A new method for the estimation of variance matrix with prescribed zeros in nonlinear mixed effects models

We propose a new method for the Maximum Likelihood Estimator (MLE) of nonlinear mixed effects models when the variance matrix of Gaussian random effects has a prescribed pattern of zeros (PPZ). The method consists of coupling the recently developed Iterative Conditional Fitting (ICF) algorithm with the Expectation Maximization (EM) algorithm. It provides positive definite estimates for any sample size, and does not rely on any structural assumption concerning the PPZ. It can be easily adapted to many versions of EM.