Likelihood inference for small variance components

The authors explore likelihood‐based methods for making inferences about the components of variance in a general normal mixed linear model. In particular, they use local asymptotic approximations to construct confidence intervals for the components of variance when the components are close to the boundary of the parameter space. In the process, they explore the question of how to profile the restricted likelihood (REML). Also, they show that general REML estimates are less likely to fall on the boundary of the parameter space than maximum‐likelihood estimates and that the likelihood‐ratio test based on the local asymptotic approximation has higher power than the likelihood‐ratio test based on the usual chi‐squared approximation. They examine the finite‐sample properties of the proposed intervals by means of a simulation study.     

On the Convergence of the Monte Carlo Maximum Likelihood Method for Latent Variable Models

While much used in practice, latent variable models raise challenging estimation problems due to the intractability of their likelihood. Monte Carlo maximum likelihood (MCML), as proposed by Geyer & Thompson (1992 ), is a simulation-based approach to maximum likelihood approximation applicable to general latent variable models. MCML can be described as an importance sampling method in which the likelihood ratio is approximated by Monte Carlo averages of importance ratios simulated from the complete data model corresponding to an arbitrary value of the unknown parameter. This paper studies the asymptotic (in the number of observations) performance of the MCML method in the case of latent variable models with independent observations. This is in contrast with previous works on the same topic which only considered conditional convergence to the maximum likelihood estimator, for a fixed set of observations. A first important result is that when is fixed, the MCML method can only be consistent if the number of simulations grows exponentially fast with the number of observations. If on the other hand, is obtained from a consistent sequence of estimates of the unknown parameter, then the requirements on the number of simulations are shown to be much weaker.     

A RESTRICTED MAXIMUM LIKELIHOOD PROCEDURE FOR ESTIMATING THE VARIANCE FUNCTION OF AN IMMUNOASSAY

Restricted maximum likelihood (REML) is a procedure for estimating a variance function in a heteroscedastic linear model. Although REML has been extended to non-linear models, the case in which the data are dominated by replicated observations with unknown values of the independent variable of interest, such as the concentration of a substance in a blood sample, has not been considered. We derive a REML procedure for an immunoassay and show that the resulting estimator is superior to those currently being used. Some interesting properties of the REML estimator are derived, and its relationship to other estimators is discussed.     

LIKELIHOOD MOMENT ESTIMATION FOR THE GENERALIZED PARETO DISTRIBUTION

Traditional methods for estimating parameters in the generalized Pareto distribution have theoretical and computational defects. The moment estimator and the probability‐weighted moment estimator have low asymptotic efficiencies. They may not exist or may give nonsensical estimates. The maximum likelihood estimator, which sometimes does not exist, is asymptotically efficient, but its computation is complex and has convergence problems. The likelihood moment estimator is proposed, which is computationally easy and has high asymptotic efficiency.     

A note on model selection using information criteria for general linear models estimated using REML

It is common practice to compare the fit of non‐nested models using the Akaike (AIC) or Bayesian (BIC) information criteria. The basis of these criteria is the log‐likelihood evaluated at the maximum likelihood estimates of the unknown parameters. For the general linear model (and the linear mixed model, which is a special case), estimation is usually carried out using residual or restricted maximum likelihood (REML). However, for models with different fixed effects, the residual likelihoods are not comparable and hence information criteria based on the residual likelihood cannot be used. For model selection, it is often suggested that the models are refitted using maximum likelihood to enable the criteria to be used. The first aim of this paper is to highlight that both the AIC and BIC can be used for the general linear model by using the full log‐likelihood evaluated at the REML estimates. The second aim is to provide a derivation of the criteria under REML estimation. This aim is achieved by noting that the full likelihood can be decomposed into a marginal (residual) and conditional likelihood and this decomposition then incorporates aspects of both the fixed effects and variance parameters. Using this decomposition, the appropriate information criteria for model selection of models which differ in their fixed effects specification can be derived. An example is presented to illustrate the results and code is available for analyses using the ASReml‐R package.     

Asymptotic variance approximations for invariant estimators in uncertain asset-pricing models

This article derives explicit expressions for the asymptotic variances of the maximum likelihood and continuously-updated GMM estimators in models that may not satisfy the fundamental asset-pricing restrictions in population. The proposed misspecification-robust variance estimators allow the researcher to conduct valid inference on the model parameters even when the model is rejected by the data. While the results for the maximum likelihood estimator are only applicable to linear asset-pricing models, the asymptotic distribution of the continuously-updated GMM estimator is derived for general, possibly nonlinear, models. The large corrections in the asymptotic variances, that arise from explicitly incorporating model misspecification in the analysis, are illustrated using simulations and an empirical application.     

Econometric Reviews

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

ASYMPTOTIC LIKELIHOOD APPROXIMATIONS USING A PARTIAL LAPLACE APPROXIMATION

Elimination of a nuisance variable is often non‐trivial and may involve the evaluation of an intractable integral. One approach to evaluate these integrals is to use the Laplace approximation. This paper concentrates on a new approximation, called the partial Laplace approximation, that is useful when the integrand can be partitioned into two multiplicative disjoint functions. The technique is applied to the linear mixed model and shows that the approximate likelihood obtained can be partitioned to provide a conditional likelihood for the location parameters and a marginal likelihood for the scale parameters equivalent to restricted maximum likelihood (REML). Similarly, the partial Laplace approximation is applied to the t‐distribution to obtain an approximate REML for the scale parameter. A simulation study reveals that, in comparison to maximum likelihood, the scale parameter estimates of the t‐distribution obtained from the approximate REML show reduced bias.     

Generalized mixed estimator for nonlinear models: a maximum likelihood approach

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

Analysis of covariance under inverse Gaussian model

This paper considers the problem of analysis of covariance (ANCOVA) under the assumption of inverse Gaussian distribution for response variable. We develop the essential methodology for estimating the model parameters via maximum likelihood method. The general form of the maximum likelihood estimator is obtained in color closed form. Adjusted treatment effects and adjusted covariate effects are given, too. We also provide the asymptotic distribution of the proposed estimators. A simulation study and a real world application are also performed to illustrate and evaluate the proposed methodology.     

Shape Constrained Non‐parametric Estimators of the Baseline Distribution in Cox Proportional Hazards Model

Abstract. We investigate non‐parametric estimation of a monotone baseline hazard and a decreasing baseline density within the Cox model. Two estimators of a non‐decreasing baseline hazard function are proposed. We derive the non‐parametric maximum likelihood estimator and consider a Grenander type estimator, defined as the left‐hand slope of the greatest convex minorant of the Breslow estimator. We demonstrate that the two estimators are strongly consistent and asymptotically equivalent and derive their common limit distribution at a fixed point. Both estimators of a non‐increasing baseline hazard and their asymptotic properties are obtained in a similar manner. Furthermore, we introduce a Grenander type estimator for a non‐increasing baseline density, defined as the left‐hand slope of the least concave majorant of an estimator of the baseline cumulative distribution function, derived from the Breslow estimator. We show that this estimator is strongly consistent and derive its asymptotic distribution at a fixed point.     

A Partial Likelihood Estimator of Vaccine Efficacy

A partial likelihood method is proposed for estimating vaccine efficacy for a general epidemic model. In contrast to the maximum likelihood estimator (MLE) which requires complete observation of the epidemic, the suggested method only requires information on the sequence in which individuals are infected and not the exact infection times. A simulation study shows that the method performs almost as well as the MLE. The method is applied to data on the infectious disease mumps.     

Penalized Maximum Likelihood Estimator for Normal Mixtures

The estimation of the parameters of a mixture of Gaussian densities is considered, within the framework of maximum likelihood. Due to unboundedness of the likelihood function, the maximum likelihood estimator fails to exist. We adopt a solution to likelihood function degeneracy which consists in penalizing the likelihood function. The resulting penalized likelihood function is then bounded over the parameter space and the existence of the penalized maximum likelihood estimator is granted. As original contribution we provide asymptotic properties, and in particular a consistency proof, for the penalized maximum likelihood estimator. Numerical examples are provided in the finite data case, showing the performances of the penalized estimator compared to the standard one.     

Confidence intervals for variance components in unbalanced one-way random effects model using non-normal distributions

In scenarios where the variance of a response variable can be attributed to two sources of variation, a confidence interval for a ratio of variance components gives information about the relative importance of the two sources. For example, if measurements taken from different laboratories are nine times more variable than the measurements taken from within the laboratories, then 90% of the variance in the responses is due to the variability amongst the laboratories and 10% of the variance in the responses is due to the variability within the laboratories. Assuming normally distributed sources of variation, confidence intervals for variance components are readily available. In this paper, however, simulation studies are conducted to evaluate the performance of confidence intervals under non-normal distribution assumptions. Confidence intervals based on the pivotal quantity method, fiducial inference, and the large-sample properties of the restricted maximum likelihood (REML) estimator are considered. Simulation results and an empirical example suggest that the REML-based confidence interval is favored over the other two procedures in unbalanced one-way random effects model.     

Pairwise likelihood estimation for multivariate mixed Poisson models generated by Gamma intensities

Estimating the parameters of multivariate mixed Poisson models is an important problem in image processing applications, especially for active imaging or astronomy. The classical maximum likelihood approach cannot be used for these models since the corresponding masses cannot be expressed in a simple closed form. This paper studies a maximum pairwise likelihood approach to estimate the parameters of multivariate mixed Poisson models when the mixing distribution is a multivariate Gamma distribution. The consistency and asymptotic normality of this estimator are derived. Simulations conducted on synthetic data illustrate these results and show that the proposed estimator outperforms classical estimators based on the method of moments. An application to change detection in low-flux images is also investigated.     

ESTIMATION OF THE CONCENTRATION PARAMETERS OF THE FISHER MATRIX DISTRIBUTION ON 50(3) AND THE BINGHAM DISTRIBUTION ON Sq, q≥ 2

Two procedures are considered for estimating the concentration parameters of the Fisher matrix distribution for rotations or orientations in three dimensions. The first is maximum likelihood. The use of a convenient 1-dimensional integral representation of the normalising constant, which greatly simplifies the computation, is suggested. The second approach exploits the equivalence of the Fisher distribution for rotations in three dimensions, and the Bingham distribution for axes in four dimensions. We describe a pseudo likelihood procedure which works for the Bingham distribution in any dimension. This alternative approach does not require numerical integration. Results on the asymptotic efficiency of the pseudo likelihood estimator relative to the maximum likelihood estimator are given, and the two estimators are compared in the analysis of a well-known vectorcardiography dataset.     

Is a Brownian Motion Skew?

We study the asymptotic behaviour of the maximum likelihood estimator corresponding to the observation of a trajectory of a skew Brownian motion, through a uniform time discretization. We characterize the speed of convergence and the limiting distribution when the step size goes to zero, which in this case are non‐classical, under the null hypothesis of the skew Brownian motion being an usual Brownian motion. This allows to design a test on the skewness parameter. We show that numerical simulations can be easily performed to estimate the skewness parameter and provide an application in Biology.     

Empirical likelihood for generalized linear models with fixed and adaptive designs

Empirical likelihood inference for generalized linear models with fixed and adaptive designs is considered. It is shown that the empirical log-likelihood ratio at the true parameters converges to the standard chi-square distribution. Furthermore, we obtain the maximum empirical likelihood estimate of the unknown parameter and the resulting estimator is shown to be asymptotically normal. Some simulations are conducted to illustrate the proposed method.     

Likelihood Analysis of the I(2) Model

The I (2) model is defined as a submodel of the general vector autoregressive model, by two reduced rank conditions. The model describes stochastic processes with stationary second difference. A parametrization is suggested which makes likelihood inference feasible. Consistency of the maximum likelihood estimator is proved, and the asymptotic distribution of the maximum likelihood estimator is given. It is shown that the asymptotic distribution is either Gaussian, mixed Gaussian or, in some cases, even more complicated.     

Parameter estimation in a verhulst stochastic model

Summary Maximum likelihood estimation is a well-known statistical tool. When applied to the study of dynamical continuous-time phenomena, it requires some specific assumptions. In this paper we discuss some properties of the maximum likelihood estimator for a stochastic Verhulst model and we show some simulation results on the behaviour of the corresponding discrete estimator.