An approximation for analyzing a broad class of implicitly and explicitly defined estimators

An approximation is presented that can be used to gain insight into the characteristics – such as outlier sensitivity, bias, and variability – of a wide class of estimators, including maximum likelihood and least squares. The approximation relies on a convenient form for an arbitrary order Taylor expansion in a multivariate setting. The implicit function theorem can be used to construct the expansion when the estimator is not defined in closed form. We present several finite-sample and asymptotic properties of such Taylor expansions, which are useful in characterizing the difference between the estimator and the expansion.     

Finite sample properties of maximum likelihood and quasi-maximum likelihood estimators of egarch models

In this paper I examine finite sample properties of the maximum likelihood and quasi-maximum likelihood estimators of EGARCH(1,1) processes using Monte Carlo methods. I use response surface methodology to summarize the results of a wide array of experiments which suggest that the maximum likelihood estimator has reasonable finite sample properties. The Gaussian quasi-maximum likelihood estimator has poor finite sample properties when the data generating process has conditional excess kurtosis. Some of these poor properties appear to be asymptotic in nature.     

Finite sample properties of maximum likelihood and quasi-maximum likelihood estimators of egarch models

In this paper I examine finite sample properties of the maximum likelihood and quasi-maximum likelihood estimators of EGARCH(1,1) processes using Monte Carlo methods. I use response surface methodology to summarize the results of a wide array of experiments which suggest that the maximum likelihood estimator has reasonable finite sample properties. The Gaussian quasi-maximum likelihood estimator has poor finite sample properties when the data generating process has conditional excess kurtosis. Some of these poor properties appear to be asymptotic in nature.     

AN APPROXIMATE MAXIMUM LIKELIHOOD ESTIMATOR FOR CHAIN BINOMIAL MODELS

Using the Poisson approximation to the Binomial distribution, we construct an approximate maximum likelihood estimator (MLE) for a class of chain binomial models. Our estimator proves to have properties which may make it preferable to the exact WLE.     

Efficiency of projected score methods in rectangular array asymptotics

Summary. The paper considers a rectangular array asymptotic embedding for multistratum data sets, in which both the number of strata and the number of within-stratum replications increase, and at the same rate. It is shown that under this embedding the maximum likelihood estimator is consistent but not efficient owing to a non-zero mean in its asymptotic normal distribution. By using a projection operator on the score function, an adjusted maximum likelihood estimator can be obtained that is asymptotically unbiased and has a variance that attains the Cramér–Rao lower bound. The adjusted maximum likelihood estimator can be viewed as an approximation to the conditional maximum likelihood estimator.     

Bias Corrected Maximum Likelihood Estimator Under the Generalized Linear Model for a Binary Variable

Under the generalized linear models for a binary variable, an approximate bias of the maximum likelihood estimator of the coefficient, that is a special case of linear parameter in Cordeiro and McCullagh (1991), is derived without a calculation of the third-order derivative of the log likelihood function. Using the obtained approximate bias of the maximum likelihood estimator, a bias-corrected maximum likelihood estimator is defined. Through a simulation study, we show that the bias-corrected maximum likelihood estimator and its variance estimator have a better performance than the maximum likelihood estimator and its variance estimator.     

Limited information likelihood analysis of survey data

Analysts of survey data are often interested in modelling the population process, or superpopulation, that gave rise to a 'target' set of survey variables. An important tool for this is maximum likelihood estimation. A survey is said to provide limited information for such inference if data used in the design of the survey are unavailable to the analyst. In this circumstance, sample inclusion probabilities, which are typically available, provide information which needs to be incorporated into the analysis. We consider the case where these inclusion probabilities can be modelled in terms of a linear combination of the design and target variables, and only sample values of these are available. Strict maximum likelihood estimation of the underlying superpopulation means of these variables appears to be analytically impossible in this case, but an analysis based on approximations to the inclusion probabilities leads to a simple estimator which is a close approximation to the maximum likelihood estimator. In a simulation study, this estimator outperformed several other estimators that are based on approaches suggested in the sampling literature.     

Computation of probability and non-centrality parameter of a non-central f-distribution

The small sample properties of the score function approximation to the maximum likelihood estimator for the three-parameter lognormal distribution using an alternative parameterization are considered. The new set of parameters is a continuous function of the usual parameters. However, unlike with the usual parameterization, the score function technique for this parameterization is extremely insensitive to starting values. Further, it is shown that whenever the sample third moment is less than zero, a local maximum to the likelihood function exists at a boundary point. For the usual parameterization, this point is unattainable. However, the alternative parameter space can be expanded to include these boundary points. This procedure results in good estimates of the expected value, variance, extreme percentiles and other parameters of the distribution even in samples where, with the typical parameterization, the estimation procedure fails to converge.     

Estimation of the scale parameter of the half-logistic distribution under progressively type II censored sample

Based on a progressively type II censored sample, the maximum likelihood and Bayes estimators of the scale parameter of the half-logistic distribution are derived. However, since the maximum likelihood estimator (MLE) and Bayes estimator do not exist in an explicit form for the scale parameter, we consider a simple method of deriving an explicit estimator by approximating the likelihood function and derive the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney and Kadane in J Am Stat Assoc 81:82–86, 1986) and importance sampling methods are used for obtaining the Bayes estimator. In order to compare the performance of the MLE, approximate MLE and Bayes estimates of the scale parameter, we use Monte Carlo simulation.     

A tobit model with garch errors

In the context of time series regression, we extend the standard Tobit model to allow for the possibility of conditional heteroskedastic error processes of the GARCH type. We discuss the likelihood function of the Tobit model in the presence of conditionally heteroskedastic errors. Expressing the exact likelihood function turns out to be infeasible, and we propose an approximation by treating the model as being conditionally Gaussian. The performance of the estimator is investigated by means of Monte Carlo simulations. We find that, when the error terms follow a GARCH process, the proposed estimator considerably outperforms the standard Tobit quasi maximum likelihood estimator. The efficiency loss due to the approximation of the likelihood is finally evaluated.     

An Alternative to Efron's Redistribution-of-Mass Construction of the Kaplan—Meier Estimator

Kaplan and Meier (1958) derived the nonparametric maximum likelihood estimator of the survival function for the case in which some survival times are right-censored. Efron (1967) proposed a redistribution-of-mass construction of the Kaplan—Meier estimator that emphasized and illustrated the contribution of the censored observations. This article presents an alternative construction that, unlike Efron's method, redistributes the mass initially associated with each censored observation directly to the uncensored observations. The proposed construction avoids distributing a given mass more than once and provides additional insight into the nature of the Kaplan—Meier estimator.     

Statistical inference for discretely observed Markov jump processes

Summary.  Likelihood inference for discretely observed Markov jump processes with finite state space is investigated. The existence and uniqueness of the maximum likelihood estimator of the intensity matrix are investigated. This topic is closely related to the imbedding problem for Markov chains. It is demonstrated that the maximum likelihood estimator can be found either by the EM algorithm or by a Markov chain Monte Carlo procedure. When the maximum likelihood estimator does not exist, an estimator can be obtained by using a penalized likelihood function or by the Markov chain Monte Carlo procedure with a suitable prior. The methodology and its implementation are illustrated by examples and simulation studies.     

Estimation of the scale parameter of the half-logistic distribution with multiply type II censored sample

In this paper, we consider the maximum likelihood and Bayes estimation of the scale parameter of the half-logistic distribution based on a multiply type II censored sample. However, the maximum likelihood estimator(MLE) and Bayes estimator do not exist in an explicit form for the scale parameter. We consider a simple method of deriving an explicit estimator by approximating the likelihood function and discuss the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney & Kadane, 1986) is used to obtain the Bayes estimator. In order to compare the MLE, approximate MLE and Bayes estimates of the scale parameter, Monte Carlo simulation is used.     

Likelihood Based Inference in Non-linear Regression Models Using the p* and R* Approach

We develop second order asymptotic results for likelihood-based inference in Gaussian non-linear regression models. We provide an approximation to the conditional density of the maximum likelihood estimator given an approximate ancillary statistic (the affine ancillary). From this approximation, we derive a statistic to test an hypothesis on one component of the parameter. This test statistic is an adjustment of the signed log-likelihood ratio statistic. The distributional approximations (for the maximum likelihood estimator and for the test statistic) are of second order in large deviation regions.     

Penalized likelihood approach for the four-parameter kappa distribution

The four-parameter kappa distribution (K4D) is a generalized form of some commonly used distributions such as generalized logistic, generalized Pareto, generalized Gumbel, and generalized extreme value (GEV) distributions. Owing to its flexibility, the K4D is widely applied in modeling in several fields such as hydrology and climatic change. For the estimation of the four parameters, the maximum likelihood approach and the method of L-moments are usually employed. The L-moment estimator (LME) method works well for some parameter spaces, with up to a moderate sample size, but it is sometimes not feasible in terms of computing the appropriate estimates. Meanwhile, using the maximum likelihood estimator (MLE) with small sample sizes shows substantially poor performance in terms of a large variance of the estimator. We therefore propose a maximum penalized likelihood estimation (MPLE) of K4D by adjusting the existing penalty functions that restrict the parameter space. Eighteen combinations of penalties for two shape parameters are considered and compared. The MPLE retains modeling flexibility and large sample optimality while also improving on small sample properties. The properties of the proposed estimator are verified through a Monte Carlo simulation, and an application case is demonstrated taking Thailand’s annual maximum temperature data.     

Estimation in zero-inflated binomial regression with missing covariates

We investigate inverse-probability-weighted (IPW) maximum likelihood estimation in zero-inflated binomial regression with missing-at-random covariates. Large sample properties (consistency, asymptotic normality) of the IPW estimator are established. Finite sample properties are assessed via simulations. The methodology is illustrated on a real data set.     

Asymptotic estimators of the sample size in a record model

The connection between Stirling numbers of the first kind and records is well-known. Applying this relationship, we derive bounds for the maximum likelihood estimator of the sample size based on the number of observed records. The proof proceeds by a remarkable expression of the mode of the unsigned Stirling numbers of the the first kind due to Hammersley. Moreover, this representation of the mode leads to an accurate approximation of the maximum likelihood estimator.     

Preliminary test and Bayes Estimation of a Location Parameter Under Blinex Loss

In this article, the preliminary test estimator is considered under the BLINEX loss function. The problem under consideration is the estimation of the location parameter from a normal distribution. The risk under the null hypothesis for the preliminary test estimator, the exact risk function for restricted maximum likelihood and approximated risk function for the unrestricted maximum likelihood estimator, are derived under BLINEX loss and the different risk structures are compared to one another both analytically and computationally. As a motivation on the use of BLINEX rather than LINEX, the risk for the preliminary test estimator under BLINEX loss is compared to the risk of the preliminary test estimator under LINEX loss and it is shown that the LINEX expected loss is higher than BLINEX expected loss. Furthermore, two feasible Bayes estimators are derived under BLINEX loss, and a feasible Bayes preliminary test estimator is defined and compared to the classical preliminary test estimator.     

Robust estimation of linear state space models

The model parameters of linear state space models are typically estimated with maximum likelihood estimation, where the likelihood is computed analytically with the Kalman filter. Outliers can deteriorate the estimation. Therefore we propose an alternative estimation method. The Kalman filter is replaced by a robust version and the maximum likelihood estimator is robustified as well. The performance of the robust estimator is investigated in a simulation study. Robust estimation of time varying parameter regression models is considered as a special case. Finally, the methodology is applied to real data.     

Penalized Maximum Likelihood Estimator for Normal Mixtures

The estimation of the parameters of a mixture of Gaussian densities is considered, within the framework of maximum likelihood. Due to unboundedness of the likelihood function, the maximum likelihood estimator fails to exist. We adopt a solution to likelihood function degeneracy which consists in penalizing the likelihood function. The resulting penalized likelihood function is then bounded over the parameter space and the existence of the penalized maximum likelihood estimator is granted. As original contribution we provide asymptotic properties, and in particular a consistency proof, for the penalized maximum likelihood estimator. Numerical examples are provided in the finite data case, showing the performances of the penalized estimator compared to the standard one.