Robustness and maximum likelihood estimation

The usual concept of robustness is called "criterion" or "non-adaptive" robustness to distinguish it from "inference" or "adaptive" robustness. The former term is appled to describe relative insensitivity to changes in the parent distribution, while the latter specifically implies dependence on and hence adaptation to changes in the parent distribution. It is argued that knowledge of, and sensitivity to the parent distribution is an important aspect of inference, and thus the latter concept of robustness is more relevant than the former. This focuses attention on adaptive procedures that use most of the sample information, that is, are efficient. Maximum likelihood has been criticized as depending critically on knowledge of the exact parent distribution, and hence of lacking criterion or non-adaptive robustness. This might have been justified when computational parameter to allow for uncertainly of shape. then the method of maximim likelihood is hsown to possess the more important requirement of being adaptive and efficent, capable of assessing the more relevant creiterion of inference or adaptive robustness.     

Symmetric maximum kernel likelihood estimation

We introduce an estimator for the population mean based on maximizing likelihoods formed from a symmetric kernel density estimate. Due to these origins, we have dubbed the estimator the symmetric maximum kernel likelihood estimate (smkle). A speedy computational method to compute the smkle based on binning is implemented in a simulation study which shows that the smkle at an optimal bandwidth is decidedly superior in terms of efficiency to the sample mean and other measures of location for heavy-tailed symmetric distributions. An empirical rule and a computational method to estimate this optimal bandwidth are developed and used to construct bootstrap confidence intervals for the population mean. We show that the intervals have approximately nominal coverage and have significantly smaller average width than the corresponding intervals for other measures of location.     

Local maximum likelihood estimation and inference

Local maximum likelihood estimation is a nonparametric counterpart of the widely used parametric maximum likelihood technique. It extends the scope of the parametric maximum likelihood method to a much wider class of parametric spaces. Associated with this nonparametric estimation scheme is the issue of bandwidth selection and bias and variance assessment. This paper provides a unified approach to selecting a bandwidth and constructing confidence intervals in local maximum likelihood estimation. The approach is then applied to least squares nonparametric regression and to nonparametric logistic regression. Our experiences in these two settings show that the general idea outlined here is powerful and encouraging.     

Efficient ascertainment schemes for maximum likelihood estimation

While well chosen sampling schemes may substantially increase efficiency of observational studies, some sampling schemes may instead decrease efficiency. Rules of thumb how to choose sampling schemes are only available for some special cases. In this paper we provide tools to compare efficiencies, and cost adjusted efficiencies, of different sampling schemes, in order to facilitate this choice. The method can be used for both categorical and continuous outcome variables. Some examples are presented, focusing on data from ascertainment sampling schemes. A Monte Carlo method is used to overcome computational issues wherever needed. The results are illustrated in graphs.     

Modified maximum likelihood estimation for rayleigh distribution

For defining a Modified Maximum Likelihood Estimate of the scale parameter of Rayleigh distribution, a hyperbolic approximation is used instead of linear approximation for a function which appears in the Maximum Likelihood equation. This estimate is shown to perform better, in the sense of accuracy and simplicity of calculation, than the one based on linear approximation for the same function. Also the estimate of the scale parameter obtained is shown to be asymptotically unbiased. Numerical computation for random samples of different sizes from Rayleigh distribution, using type I1 censoring is done and is shown to be better than that obtained by Lee et al. (1980)     

Nonparametric maximum likelihood estimation for shifted curves

The analysis of a sample of curves can be done by self-modelling regression methods. Within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set-up. We derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model. The nonparametric maximum likelihood estimator that we find turns out to be a Nadaraya–Watson-like estimator, but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function, since the score equation automatically selects the shape and the smoothing parameter for the estimation. We apply the method to a sample of electrophoretic spectra to illustrate how it works.     

N-stage splitting for maximum penalized likelihood estimation

An n-stage splitting algorithm for the solution of maximum penalized likelihood estimation (MPLE) problems is compared to the one-step-late (OSL) algorithm. General conditions under which the asymptotic rate of convergence of this splitting algorithm. exceeds that of the OSL algorithm are given. A one-dimensional positive data example, illustrates the comparison of the rates of convergence of these two algorithms.     

Pseudo maximum likelihood estimation for the dirichlet-multinomial distribution

Pseudo maximum likelihood estimation (PML) for the Dirich-let-multinomial distribution is proposed and examined in this pa-per. The procedure is compared to that based on moments (MM) for its asymptotic relative efficiency (ARE) relative to the maximum likelihood estimate (ML). It is found that PML, requiring much less computational effort than ML and possessing considerably higher ARE than MM, constitutes a good compromise between ML and MM. PML is also found to have very high ARE when an estimate for the scale parameter in the Dirichlet-multinomial distribution is all that is needed.     

Sequential maximum likelihood estimation for reflected Ornstein-Uhlenbeck processes

The paper studies the properties of a sequential maximum likelihood estimator of the drift parameter in a one dimensional reflected Ornstein-Uhlenbeck process. We observe the process until the observed Fisher information reaches a specified precision level. We derive the explicit formulas for the sequential estimator and its mean squared error. The estimator is shown to be unbiased and uniformly normally distributed. A simulation study is conducted to assess the performance of the estimator compared with the ordinary maximum likelihood estimator.     

Multilevel maximum likelihood estimation with application to covariance matrices

The asymptotic variance of the maximum likelihood estimate is proved to decrease when the maximization is restricted to a subspace that contains the true parameter value. Maximum likelihood estimation allows a systematic fitting of covariance models to the sample, which is important in data assimilation. The hierarchical maximum likelihood approach is applied to the spectral diagonal covariance model with different parameterizations of eigenvalue decay, and to the sparse inverse covariance model with specified parameter values on different sets of nonzero entries. It is shown computationally that using smaller sets of parameters can decrease the sampling noise in high dimension substantially.     

Nonparametric maximum likelihood estimation for artificially truncated absence data

In manpower planning it is cornmoniy tue case tnat employees withuraw from active service for a period of time before returning to take up post at a later date. Such periods of absence are frequently of major concern to employers who are anxious to ensure that employees return as soon as possible. The distribution of duration of such periods of absence are therefore of considerable interest as is the probability that such employees will ever return to active service. In this paper we derive a nonparametric estimator for such a lifetime distribution based on renewal data which are subject to various forms of incompleteness, namely right censoring, left and right truncation, and forward recurrence. Artificial truncation is used to ensure that the data are time homogeneous. A nonparametric maximum likelihood estimator for the lifetime.     

On maximum likelihood estimation of the binomial parameter n

This paper gives a necessary and sufficient condition for the existence of a finite conditional maximum-likelihood estimate for the binomial parameter n. Upper bounds for the conditional and beta-binomial maximum-likelihood estimators are derived. An example is given to show that the conditional likelihood function and the beta-binomial likelihood function may not be unimodal.     

Partitioned algorithms for maximum likelihood and other non-linear estimation

There are a variety of methods in the literature which seek to make iterative estimation algorithms more manageable by breaking the iterations into a greater number of simpler or faster steps. Those algorithms which deal at each step with a proper subset of the parameters are called in this paper partitioned algorithms. Partitioned algorithms in effect replace the original estimation problem with a series of problems of lower dimension. The purpose of the paper is to characterize some of the circumstances under which this process of dimension reduction leads to significant benefits.Four types of partitioned algorithms are distinguished: reduced objective function methods, nested (partial Gauss-Seidel) iterations, zigzag (full Gauss-Seidel) iterations, and leapfrog (non-simultaneous) iterations. Emphasis is given to Newton-type methods using analytic derivatives, but a nested EM algorithm is also given. Nested Newton methods are shown to be equivalent to applying to same Newton method to the reduced objective function, and are applied to separable regression and generalized linear models. Nesting is shown generally to improve the convergence of Newton-type methods, both by improving the quadratic approximation to the log-likelihood and by improving the accuracy with which the observed information matrix can be approximated. Nesting is recommended whenever a subset of parameters is relatively easily estimated. The zigzag method is shown to produce a stable but generally slow iteration; it is fast and recommended when the parameter subsets have approximately uncorrelated estimates. The leapfrog iteration has less guaranteed properties in general, but is similar to nesting and zigzagging when the parameter subsets are orthogonal.     

Bias-reduced maximum likelihood estimation of the zero-inflated Poisson distribution

Using the techniques developed by Subrahmaniam and Ching’anda (1978), we study the robustness to nonnormality of the linear discriminant functions. It is seen that the LDF procedure is quite robust against the likelihood ratio rule. The latter yields in all cases much smaller overall error rates; however, the disparity between the error rates of the LDF and LR procedures is not large enough to warrant the recommendation to use the more complicated LR procedure.     

Penalized maximum likelihood estimation for Gaussian hidden Markov models

ABSTRACT

The likelihood function of a Gaussian hidden Markov model is unbounded, which is why the maximum likelihood estimator (MLE) is not consistent. A penalized MLE is introduced along with a rigorous consistency proof.     

Weighted discrepancies and maximum likelihood estimation for discrete distributions

The paper shows that many estimation methods, including ML, moments, even-points, empirical c.f. and minimum chi-square, can be regarded as scoring procedures using weighted sums of the discrepancies between observed and expected frequencies The nature of the weights is investigated for many classes of distributions; the study of approximations to the weights clarifies the relationships between estimation methods, and also leads to useful formulae for initial values for ML iteration.     

A characteristic function-based approach to approximate maximum likelihood estimation

The choice of the summary statistics in approximate maximum likelihood is often a crucial issue. We develop a criterion for choosing the most effective summary statistic and then focus on the empirical characteristic function. In the iid setting, the approximating posterior distribution converges to the approximate distribution of the parameters conditional upon the empirical characteristic function. Simulation experiments suggest that the method is often preferable to numerical maximum likelihood. In a time-series framework, no optimality result can be proved, but the simulations indicate that the method is effective in small samples.     

Quasi maximum likelihood estimation of dynamic panel data models

This article establishes the almost sure convergence and asymptotic normality of levels and differenced quasi maximum likelihood (QML) estimators of dynamic panel data models. The QML estimators are robust with respect to initial conditions, conditional and time-series heteroskedasticity, and misspecification of the log-likelihood. The article also provides an ECME algorithm for calculating levels QML estimates. Finally, it compares the finite-sample performance of levels and differenced QML estimators, the differenced generalized method of moments (GMM) estimator, and the system GMM estimator. The QML estimators usually have smaller— typically substantially smaller—bias and root mean squared errors than the panel data GMM estimators.     

Aspects of maximum likelihood estimation of asymmetric growth curves

As an extension to the work of Oliver, the variance-covariance matrices for the powered logistic and Bass equations were obtained and used to show that the addition of a fourth parameter resulted in the estimates for the remaining three parameters corresponding to those of the simple logistic becoming more unreliable. The degree of confidence that could be placed on the estimated parameters of the powered logistic decreased with the degree of asymmetry. However, the Bass equation proved much more resilient to this problem. A Monte Carlo study shows that, in small samples and for data sets that resemble new technologies in the early stages of di usion, the maximum likelihood estimates do not have the desirable properties of estimates based on large samples.     

Approximate maximum likelihood estimation in the presence of extra-poisson variation

A two-stage hierarchical model for analysis of discrete data with extra-Poisson variation is examined. The model consists of a Poisson distribution with a mixing lognormal distribution for the mean. A method of approximate maximum likelihood estimation of the parameters is proposed. The method uses the EM algorithm and approximations to facilitate its implementation are derived. Approximate standard errors of the estimates are provided and a numerical example is used to illustrate the method.