Towards a unified definition of maximum likelihood

A unified definition of maximum likelihood (ml) is given. It is based on a pairwise comparison of probability measures near the observed data point. This definition does not suffer from the usual inadequacies of earlier definitions, i.e., it does not depend on the choice of a density version in the dominated case. The definition covers the undominated case as well, i.e., it provides a consistent approach to nonparametric ml problems, which heretofore have been solved on a more less ad hoc basis. It is shown that the new ml definition is a true extension of the classical ml approach, as it is practiced in the dominated case. Hence the classical methodology can simply be subsumed. Parametric and nonparametric examples are discussed.     

On a criticism of the profile likelihood function

The profile likelihood function is often criticized for giving strange or unintuitive results. In the cases discussed here these are due to the use of density functions that have singularities. These singularities are naturally inherited by the profile likelihood function. It is therefore apparently important to be reminded that likelihood functions are proportional to probability functions, and so cannot have singularities. When this issue is addressed, then the profile likelihood poses no problems of this sort. This is of particular importance since the profile likelihood is a commonly used method for dealing with separate estimation of parameters.     

The score function and a comparison of various adjustments of the profile likelihood

Several adjustments of the profile likelihood have the common effect of reducing the bias of the associated score function. Hence expansions for the adjusted score functions differ by a term, D_ξ, that has small asymptotic order (n ^?½). The effect of D_ξ on other quantities of interest is studied. In particular, we find the bias and variance of the adjusted maximum-likelihood estimate to be relatively unaffected, while differences in the Bartlett correction depend on D_ξ in a simple way.     

Improved likelihood inference for the shape parameter in Weibull regression

We obtain adjustments to the profile likelihood function in Weibull regression models with and without censoring. Specifically, we consider two different modified profile likelihoods: (i) the one proposed by Cox and Reid [Cox, D.R. and Reid, N., 1987, Parameter orthogonality and approximate conditional inference. Journal of the Royal Statistical Society B, 49, 1–39.], and (ii) an approximation to the one proposed by Barndorff–Nielsen [Barndorff–Nielsen, O.E., 1983, On a formula for the distribution of the maximum likelihood estimator. Biometrika, 70, 343–365.], the approximation having been obtained using the results by Fraser and Reid [Fraser, D.A.S. and Reid, N., 1995, Ancillaries and third-order significance. Utilitas Mathematica, 47, 33–53.] and by Fraser et al. [Fraser, D.A.S., Reid, N. and Wu, J., 1999, A simple formula for tail probabilities for frequentist and Bayesian inference. Biometrika, 86, 655–661.]. We focus on point estimation and likelihood ratio tests on the shape parameter in the class of Weibull regression models. We derive some distributional properties of the different maximum likelihood estimators and likelihood ratio tests. The numerical evidence presented in the paper favors the approximation to Barndorff–Nielsen's adjustment.     

A maximum likelihood prediction function for the linear model with consistency results

An alternative technique to current methods for constructing a prediction function for the normal linear regression model is proposed based on the concept of maximum likelihood. The form of this prediction function is evaluated and normalized to produce a multivariate Student's t-density. Consistency properties are established under regularity conditions, and an empirical comparison, based on the Kullback-Leibler information divergence, is made with some other prediction functions.     

Maximum likelihood and bayes prediction of current system lifetime

The Power Law Process is often used to analyse failure data of repairable systems undergoing development testing where the system failure intensity decreases as a result of repeated application of corrective actions. At the end of the development program, the system failure intensity is assumed to remain constant and the current system lifetime is assumed to be exponentially distributed. In this paper, prediction limits on the current system lifetime have been derived both in the maximum likelihood and Bayesian context. Exact values and a closed form approximation of percentage points of the pivotal quantity used in the classical approach are given in the case of failure truncated testing. For both failure and time truncated testing, the Bayesian approach is developed both when no prior knowledge is available and when information on the reliability growth rate can be given. A numerical example is also given.     

Parameter estimation in semi-linear models using a maximal invariant likelihood function

In this paper, we consider the problem of estimation of semi-linear regression models. Using invariance arguments, Bhowmik and King [2007. Maximal invariant likelihood based testing of semi-linear models. Statist. Papers 48, 357–383] derived the probability density function of the maximal invariant statistic for the non-linear component of these models. Using this density function as a likelihood function allows us to estimate these models in a two-step process. First the non-linear component parameters are estimated by maximising the maximal invariant likelihood function. Then the non-linear component, with the parameter values replaced by estimates, is treated as a regressor and ordinary least squares is used to estimate the remaining parameters. We report the results of a simulation study conducted to compare the accuracy of this approach with full maximum likelihood and maximum profile-marginal likelihood estimation. We find maximising the maximal invariant likelihood function typically results in less biased and lower variance estimates than those from full maximum likelihood.     

Continuous outcomes stochastically mismeasured as discrete random variables: a maximum likelihood approach

In an economic model of retirement behavior, a continuous dependent variable was required; the variable could only be estimated discretely with error, however. Parameter estimates using this dependent variable and ordinary least squares regression are inefficient. In th is paper, we develop a maximum likelihood procedure which adjusts for this heteroscedasticity.     

Asymptotic properties of a conditional maximum-likelihood estimator

Inference for a scalar interest parameter in the presence of nuisance parameters is considered in terms of the conditional maximum-likelihood estimator developed by Cox and Reid (1987). Parameter orthogonality is assumed throughout. The estimator is analyzed by means of stochastic asymptotic expansions in three cases: a scalar nuisance parameter, m nuisance parameters from m independent samples, and a vector nuisance parameter. In each case, the expansion for the conditional maximum-likelihood estimator is compared with that for the usual maximum-likelihood estimator. The means and variances are also compared. In each of the cases, the bias of the conditional maximum-likelihood estimator is unaffected by the nuisance parameter to first order. This is not so for the maximum-likelihood estimator. The assumption of parameter orthogonality is crucial in attaining this result. Regardless of parametrization, the difference in the two estimators is first-order and is deterministic to this order.     

The evaluation of exact maximum likelihood estimates for varma models

This paper describes an algorithm for the evaluation of the exact likelihood function in order to obtain estimates of the coefficients of vector autoregressive moving average (VARMA) models. The use of the algorithm is illustrated by a Monte Carlo experiment and an application to the analysis of a set of bivariate animal population data. Fanally it is shown how to extend the algorithm, in a simple manner, to obtain exact maximum likelihood estimates of the coefficients of vector autoregressive moving average models with included exogenous variables.     

A note on comparison and improvement of estimators based on likelihood

The problem of estimation of a parameter of interest in the presence of a nuisance parameter, which is either location or scale, is considered. Three estimators are taken into account: usual maximum likelihood (ML) estimator, maximum integrated likelihood estimator and the bias-corrected ML estimator. General results on comparison of these estimators w.r.t. the second-order risk based on the mean-squared error are obtained. Possible improvements of basic estimators via the notion of admissibility and methodology given in Ghosh and Sinha [A necessary and sufficient condition for second order admissibility with applications to Berkson's bioassay problem. Ann Stat. 1981;9(6):1334–1338] are considered. In the recent paper by Tanaka et al. [On improved estimation of a gamma shape parameter. Statistics. 2014; doi:10.1080/02331888.2014.915842], this problem was considered for estimating the shape parameter of gamma distribution. Here, we perform more accurate comparison of estimators for this case as well as for some other cases.     

Unimodality of the pareto distribution likelihood function for multicensored samples and implications for estimation

This paper discusses maximum likelihood parameter estimation in the Pareto distribution for multicensored samples. In particu-

lar, the modality of the associated conditional log-likelihood function is investigated in order to resolve questions concerninc

the existence and uniqurneas of the lnarimum likelihood estimates.For the cases with one parameter known, the maximum likelihood

estimates of the remaining unknown parameters are shown to exist and to be unique. When both parameters are unknown, the maximum likelihood estimates may or may not exist and be unique. That is, their existence and uniqueness would seem to depend solely upon the information inherent in the sample data. In viav of the possible nonexistence and/or non-uniqueness of the maximum likelihood estimates when both parameters are unknown, alternatives to standard iterative numerical methods are explored.     

On the uniqueness of the maximum likelihood estimator in truncated regression models

In this short note it is demonstrated that although the log-likelihood function for the truncated normal regression model may not be globally concave, it will possess a unique maximum if one exists. This is because the hessian matrix is negative semi-definite when evaluated at any possible solution to the likelihood equations. Since this rules out any saddle points or local minima, more than two local maxima occuring is impossible.     

The combined dynamically weighted modified maximum likelihood estimators of the location and scale parameters

In this study, two new types of estimators of the location and scale parameters are proposed having high efficiency and robustness; the dynamically weighted modified maximum likelihood (DWMML) and the combined dynamically weighted modified maximum likelihood (CDWMML) estimators. Three pairs of the DWMML and two pairs of the CDWMML estimators of the location and scale parameters are produced, namely, the DWMML1, the DWMML2 and the DWMML3, and the CDWMML1 and the CDWMML2 estimators, respectively. Based on the simulation results, the DWMML1 estimators of the location and scale parameters are almost fully efficient (under normality) and robust at the same time. The DWMML3 estimators are asymptotically fully efficient and more robust than the M-estimators. The DWMML2 estimators are a compromise between efficiency and robustness. The CDWMML1 and CDWMML2 estimators are jointly very efficient and robust. Particularly, the CDWMML1 and CDWMML2 estimators of the scale parameter are superior compared to the other estimators of the scale parameter.     

Semiparametric inference with a functional-form empirical likelihood

A functional-form empirical likelihood method is proposed as an alternative method to the empirical likelihood method. The proposed method has the same asymptotic properties as the empirical likelihood method but has more flexibility in choosing the weight construction. Because it enjoys the likelihood-based interpretation, the profile likelihood ratio test can easily be constructed with a chi-square limiting distribution. Some computational details are also discussed, and results from finite-sample simulation studies are presented.     

Pseudo maximum likelihood estimation for the dirichlet-multinomial distribution

Pseudo maximum likelihood estimation (PML) for the Dirich-let-multinomial distribution is proposed and examined in this pa-per. The procedure is compared to that based on moments (MM) for its asymptotic relative efficiency (ARE) relative to the maximum likelihood estimate (ML). It is found that PML, requiring much less computational effort than ML and possessing considerably higher ARE than MM, constitutes a good compromise between ML and MM. PML is also found to have very high ARE when an estimate for the scale parameter in the Dirichlet-multinomial distribution is all that is needed.