Numerical maximum likelihood estimation for the g-and-k and generalized g-and-h distributions

Continuing increases in computing power and availability mean that many maximum likelihood estimation (MLE) problems previously thought intractable or too computationally difficult can now be tackled numerically. However, ML parameter estimation for distributions whose only analytical expression is as quantile functions has received little attention. Numerical MLE procedures for parameters of new families of distributions, the g-and-k and the generalized g-and-h distributions, are presented and investigated here. Simulation studies are included, and the appropriateness of using asymptotic methods examined. Because of the generality of these distributions, the investigations are not only into numerical MLE for these distributions, but are also an initial investigation into the performance and problems for numerical MLE applied to quantile-defined distributions in general. Datasets are also fitted using the procedures here. Results indicate that sample sizes significantly larger than 100 should be used to obtain reliable estimates through maximum likelihood.     

Penalized maximum likelihood estimation for Gaussian hidden Markov models

ABSTRACT

The likelihood function of a Gaussian hidden Markov model is unbounded, which is why the maximum likelihood estimator (MLE) is not consistent. A penalized MLE is introduced along with a rigorous consistency proof.     

A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions

A faster alternative to the EM algorithm in finite mixture distributions is described, which alternates EM iterations with Gauss-Newton iterations using the observed information matrix. At the expense of modest additional analytical effort in obtaining the observed information, the hybrid algorithm reduces the computing time required and provides asymptotic standard errors at convergence. The algorithm is illustrated on the two-component normal mixture.     

Sequential maximum likelihood estimation for reflected Ornstein-Uhlenbeck processes

The paper studies the properties of a sequential maximum likelihood estimator of the drift parameter in a one dimensional reflected Ornstein-Uhlenbeck process. We observe the process until the observed Fisher information reaches a specified precision level. We derive the explicit formulas for the sequential estimator and its mean squared error. The estimator is shown to be unbiased and uniformly normally distributed. A simulation study is conducted to assess the performance of the estimator compared with the ordinary maximum likelihood estimator.     

Bootstrap confidence intervals of CNpk for inverse Rayleigh and log-logistic distributions

In this article bootstrap confidence intervals of process capability index as suggested by Chen and Pearn [An application of non-normal process capability indices. Qual Reliab Eng Int. 1997;13:355–360] are studied through simulation when the underlying distributions are inverse Rayleigh and log-logistic distributions. The well-known maximum likelihood estimator is used to estimate the parameter. The bootstrap confidence intervals considered in this paper consists of various confidence intervals. A Monte Carlo simulation has been used to investigate the estimated coverage probabilities and average widths of the bootstrap confidence intervals. Application examples on two distributions for process capability indices are provided for practical use.     

Pseudo maximum likelihood estimation for the dirichlet-multinomial distribution

Pseudo maximum likelihood estimation (PML) for the Dirich-let-multinomial distribution is proposed and examined in this pa-per. The procedure is compared to that based on moments (MM) for its asymptotic relative efficiency (ARE) relative to the maximum likelihood estimate (ML). It is found that PML, requiring much less computational effort than ML and possessing considerably higher ARE than MM, constitutes a good compromise between ML and MM. PML is also found to have very high ARE when an estimate for the scale parameter in the Dirichlet-multinomial distribution is all that is needed.     

Modified maximum likelihood and modified moment estimators for the three-parameter weibull distribution

This article is concerned with modifications of both maximum likelihood and moment estimators for parameters of the three-parameter Wei bull distribution. Modifications presented here are basically the same as those previously proposed by the authors (1980, 1981, 1982) in connection with the lognormal and the gamma distributions. Computer programs were prepared for the practical application of these estimators and an illustrative example is included. Results of a simulation study provide insight into the sampling behavior of the new estimators and include comparisons with the traditional moment and maximum likelihood estimators. For some combinations of parameter values, some of the modified estimators considered here enjoy advantages over both moment and maximum likelihood estimators with respect to bias, variance, and/or ease of calculation.     

Bootstrap confidence intervals in mixtures of discrete distributions

The problem of building bootstrap confidence intervals for small probabilities with count data is addressed. The law of the independent observations is assumed to be a mixture of a given family of power series distributions. The mixing distribution is estimated by nonparametric maximum likelihood and the corresponding mixture is used for resampling. We build percentile-t and Efron percentile bootstrap confidence intervals for the probabilities and we prove their consistency in probability. The new theoretical results are supported by simulation experiments for Poisson and geometric mixtures. We compare percentile-t and Efron percentile bootstrap intervals with eight other bootstrap or asymptotic theory based intervals. It appears that Efron percentile bootstrap intervals outperform the competitors in terms of coverage probability and length.     

Fitting probability distributions to economic growth: a maximum likelihood approach

The growth rate of the gross domestic product (GDP) usually carries heteroscedasticity, asymmetry and fat-tails. In this study three important and significantly heteroscedastic GDP series are examined. A Normal, normal-mixture, normal-asymmetric Laplace distribution and a Student's t-Asymmetric Laplace (TAL) distribution mixture are considered for distributional fit comparison of GDP growth series after removing heteroscedasticity. The parameters of the distributions have been estimated using maximum likelihood method. Based on the results of different accuracy measures, goodness-of-fit tests and plots, we find out that in the case of asymmetric, heteroscedastic and highly leptokurtic data the TAL-distribution fits better than the alternatives. In the case of asymmetric, heteroscedastic but less leptokurtic data the NM fit is superior. Furthermore, a simulation study has been carried out to obtain standard errors for the estimated parameters. The results of this study might be used in e.g. density forecasting of GDP growth series or to compare different economies.     

Partially projected gradient algorithms for computing nonparametric maximum likelihood estimates of mixing distributions

There exist primarily three different types of algorithms for computing nonparametric maximum likelihood estimates (NPMLEs) of mixing distributions in the literature, which are the EM-type algorithms, the vertex direction algorithms such as VDM and VEM, and the algorithms based on general constrained optimization techniques such as the projected gradient method. It is known that the projected gradient algorithm may run into stagnation during iterations. When a stagnation occurs, VDM steps need to be added. We argue that the abrupt switch to VDM steps can significantly reduce the efficiency of the projected gradient algorithm, and is usually unnecessary. In this paper, we define a group of partially projected directions, which can be regarded as hybrids of ordinary projected gradient directions and VDM directions. Based on these directions, four new algorithms are proposed for computing NPMLEs of mixing distributions. The properties of the algorithms are discussed and their convergence is proved. Extensive numerical simulations show that the new algorithms outperform the existing methods, especially when a NPMLE has a large number of support points or when high accuracy is required.     

Towards a unified definition of maximum likelihood

A unified definition of maximum likelihood (ml) is given. It is based on a pairwise comparison of probability measures near the observed data point. This definition does not suffer from the usual inadequacies of earlier definitions, i.e., it does not depend on the choice of a density version in the dominated case. The definition covers the undominated case as well, i.e., it provides a consistent approach to nonparametric ml problems, which heretofore have been solved on a more less ad hoc basis. It is shown that the new ml definition is a true extension of the classical ml approach, as it is practiced in the dominated case. Hence the classical methodology can simply be subsumed. Parametric and nonparametric examples are discussed.     

Skewness and mixtures of normal distributions

The effect of skewness on hypothesis tests for the existence of a mixture of univariate and bivariate normal distributions is examined through a Monte Carlo study. A likelihood ratio test based on results of the simultaneous estimation of skewness parameters, derived from power transformations, with mixture parameters is proposed. This procedure detects the difference between inherent distributional skewness and the apparent skewness which is a manifestation of the mixture of several distributions. The properties of this test are explored through a simulation study.     

Symmetric maximum kernel likelihood estimation

We introduce an estimator for the population mean based on maximizing likelihoods formed from a symmetric kernel density estimate. Due to these origins, we have dubbed the estimator the symmetric maximum kernel likelihood estimate (smkle). A speedy computational method to compute the smkle based on binning is implemented in a simulation study which shows that the smkle at an optimal bandwidth is decidedly superior in terms of efficiency to the sample mean and other measures of location for heavy-tailed symmetric distributions. An empirical rule and a computational method to estimate this optimal bandwidth are developed and used to construct bootstrap confidence intervals for the population mean. We show that the intervals have approximately nominal coverage and have significantly smaller average width than the corresponding intervals for other measures of location.     

On the uniqueness of the maximum likelihood estimator in truncated regression models

In this short note it is demonstrated that although the log-likelihood function for the truncated normal regression model may not be globally concave, it will possess a unique maximum if one exists. This is because the hessian matrix is negative semi-definite when evaluated at any possible solution to the likelihood equations. Since this rules out any saddle points or local minima, more than two local maxima occuring is impossible.     

The combined dynamically weighted modified maximum likelihood estimators of the location and scale parameters

In this study, two new types of estimators of the location and scale parameters are proposed having high efficiency and robustness; the dynamically weighted modified maximum likelihood (DWMML) and the combined dynamically weighted modified maximum likelihood (CDWMML) estimators. Three pairs of the DWMML and two pairs of the CDWMML estimators of the location and scale parameters are produced, namely, the DWMML1, the DWMML2 and the DWMML3, and the CDWMML1 and the CDWMML2 estimators, respectively. Based on the simulation results, the DWMML1 estimators of the location and scale parameters are almost fully efficient (under normality) and robust at the same time. The DWMML3 estimators are asymptotically fully efficient and more robust than the M-estimators. The DWMML2 estimators are a compromise between efficiency and robustness. The CDWMML1 and CDWMML2 estimators are jointly very efficient and robust. Particularly, the CDWMML1 and CDWMML2 estimators of the scale parameter are superior compared to the other estimators of the scale parameter.     

Nonparametric maximum likelihood estimation for shifted curves

The analysis of a sample of curves can be done by self-modelling regression methods. Within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set-up. We derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model. The nonparametric maximum likelihood estimator that we find turns out to be a Nadaraya–Watson-like estimator, but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function, since the score equation automatically selects the shape and the smoothing parameter for the estimation. We apply the method to a sample of electrophoretic spectra to illustrate how it works.     

The evaluation of exact maximum likelihood estimates for varma models

This paper describes an algorithm for the evaluation of the exact likelihood function in order to obtain estimates of the coefficients of vector autoregressive moving average (VARMA) models. The use of the algorithm is illustrated by a Monte Carlo experiment and an application to the analysis of a set of bivariate animal population data. Fanally it is shown how to extend the algorithm, in a simple manner, to obtain exact maximum likelihood estimates of the coefficients of vector autoregressive moving average models with included exogenous variables.     

Generalized mixed estimator for nonlinear models: a maximum likelihood approach

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

Bootstrap for U-statistics: a new approach

Bootstrap for nonlinear statistics like U-statistics of dependent data has been studied by several authors. This is typically done by producing a bootstrap version of the sample and plugging it into the statistic. We suggest an alternative approach of getting a bootstrap version of U-statistics. We will show the consistency of the new method and compare its finite sample properties in a simulation study and by applying both methods to financial data.     

Product limit estimates: a generalized maximum likelihood study

The generalized maximum likelihood estimate (GMLE) assumptions are studied for four product-limit estimates (PLE): Censoring PLE (Kaplan-Meier estimate), truncation PLE, censoring-truncation PLE, and the degenerated PLE - the empirical distribution function. This paper shows that all the PLE's are also the GMLE's even if they are derived from partial likelihoods by natural parameterization techniques. However, a counter example is given to show that Kiefer Wolfowitz's assumption (1956) for consistency of GMLE can hardly be satisfied for un-dominated case.