Asymptotical improvement of maximum likelihood estimators on Kullback–Leibler loss

We discuss the general form of a first-order correction to the maximum likelihood estimator which is expressed in terms of the gradient of a function, which could for example be the logarithm of a prior density function. In terms of Kullback–Leibler divergence, the correction gives an asymptotic improvement over maximum likelihood under rather general conditions. The theory is illustrated for Bayes estimators with conjugate priors. The optimal choice of hyper-parameter to improve the maximum likelihood estimator is discussed. The results based on Kullback–Leibler risk are extended to a wide class of risk functions.     

Efficiency of projected score methods in rectangular array asymptotics

Summary. The paper considers a rectangular array asymptotic embedding for multistratum data sets, in which both the number of strata and the number of within-stratum replications increase, and at the same rate. It is shown that under this embedding the maximum likelihood estimator is consistent but not efficient owing to a non-zero mean in its asymptotic normal distribution. By using a projection operator on the score function, an adjusted maximum likelihood estimator can be obtained that is asymptotically unbiased and has a variance that attains the Cramér–Rao lower bound. The adjusted maximum likelihood estimator can be viewed as an approximation to the conditional maximum likelihood estimator.     

On the consistency of MLE in finite mixture models of exponential families

Finite mixtures of densities from an exponential family are frequently used in the statistical analysis of data. Modelling by finite mixtures of densities from different exponential families provide more flexibility in the fittings, and get better results. However, in mixture problems, the log-likelihood function very often does not have an upper bound and therefore a global maximum does not always exist. Redner and Walker (1984. Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26, 195–239) provide conditions to assure the existence, consistency and asymptotic normality of the maximum likelihood estimator.     

Adaptive wavelet estimation of a density from mixtures under multiplicative censoring

In this paper, a mixture model under multiplicative censoring is considered. We investigate the estimation of a component of the mixture (a density) from the observations. A new adaptive estimator based on wavelets and a hard thresholding rule is constructed for this problem. Under mild assumptions on the model, we study its asymptotic properties by determining an upper bound of the mean integrated squared error over a wide range of Besov balls. We prove that the obtained upper bound is sharp.     

Estimation of the scale parameter of the half-logistic distribution with multiply type II censored sample

In this paper, we consider the maximum likelihood and Bayes estimation of the scale parameter of the half-logistic distribution based on a multiply type II censored sample. However, the maximum likelihood estimator(MLE) and Bayes estimator do not exist in an explicit form for the scale parameter. We consider a simple method of deriving an explicit estimator by approximating the likelihood function and discuss the asymptotic variances of MLE and approximate MLE. Also, an approximation based on the Laplace approximation (Tierney & Kadane, 1986) is used to obtain the Bayes estimator. In order to compare the MLE, approximate MLE and Bayes estimates of the scale parameter, Monte Carlo simulation is used.     

Variance estimation in the central limit theorem for Markov chains

This article concerns the variance estimation in the central limit theorem for finite recurrent Markov chains. The associated variance is calculated in terms of the transition matrix of the Markov chain. We prove the equivalence of different matrix forms representing this variance. The maximum likelihood estimator for this variance is constructed and it is proved that it is strongly consistent and asymptotically normal. The main part of our analysis consists in presenting closed matrix forms for this new variance. Additionally, we prove the asymptotic equivalence between the empirical and the maximum likelihood estimation (MLE) for the stationary distribution.     

Change point problems in the model of logistic regression

The paper considers generalized maximum likelihood asymptotic power one tests which aim to detect a change point in logistic regression when the alternative specifies that a change occurred in parameters of the model. A guaranteed non-asymptotic upper bound for the significance level of each of the tests is presented. For cases in which the test supports the conclusion that there was a change point, we propose a maximum likelihood estimator of that point and present results regarding the asymptotic properties of the estimator. An important field of application of this approach is occupational medicine, where for a lot chemical compounds and other agents, so-called threshold limit values (or TLVs) are specified.We demonstrate applications of the test and the maximum likelihood estimation of the change point using an actual problem that was encountered with real data.     

Nonparametric estimation of the conditional distribution function in a semiparametric censorship model

In this paper we propose a new nonparametric estimator of the conditional distribution function under a semiparametric censorship model. We establish an asymptotic representation of the estimator as a sum of iid random variables, balanced by some kernel weights. This representation is used for obtaining large sample results such as the rate of uniform convergence of the estimator, or its limit distributional law. We prove that the new estimator outperforms the conditional Kaplan–Meier estimator for censored data, in the sense that it exhibits lower asymptotic variance. Illustration through real data analysis is provided.     

Asymptotics for an Adaptive Trimmed Likelihood Location Estimator

An asymptotic normality result is given for an adaptive trimmed likelihood estimator of location, which parallels the asymptotic normality result for the adaptive trimmed mean. The new result comes out of studying the adaptive trimmed likelihood estimator modelled parametrically by a normal family but then examining the behavior when the underlying distribution is in fact some F different from normal. The asymptotic variance of the adaptive estimator is equal to the asymptotic variance of the trimmed likelihood estimator at the optimal trimming proportion for the distribution F, subject to that trimming proportion being positive and F being suitably smooth.     

Inference based on adaptive grid selection of probability transforms

Probability transform-based inference, for example, characteristic function-based inference, is a good alternative to likelihood methods when the probability density function is unavailable or intractable. However, a set of grids needs to be determined to provide an effective estimator based on probability transforms. This paper is concerned with parametric inference based on adaptive selection of grids. By employing a closeness measure to evaluate the asymptotic variance of the transform-based estimator, we propose a statistical inference procedure, accompanied with adaptive grid selection. The selection algorithm aims for a small set of grids, and yet the resulting estimator can be highly efficient. Generally, the asymptotic variance is very close to that of the maximum likelihood estimator.     

Estimation of treatment effects based on possibly misspecified Cox regression

In randomized clinical trials, a treatment effect on a time-to-event endpoint is often estimated by the Cox proportional hazards model. The maximum partial likelihood estimator does not make sense if the proportional hazard assumption is violated. Xu and O'Quigley (Biostatistics 1:423-439, 2000) proposed an estimating equation, which provides an interpretable estimator for the treatment effect under model misspecification. Namely it provides a consistent estimator for the log-hazard ratio among the treatment groups if the model is correctly specified, and it is interpreted as an average log-hazard ratio over time even if misspecified. However, the method requires the assumption that censoring is independent of treatment group, which is more restricted than that for the maximum partial likelihood estimator and is often violated in practice. In this paper, we propose an alternative estimating equation. Our method provides an estimator of the same property as that of Xu and O'Quigley under the usual assumption for the maximum partial likelihood estimation. We show that our estimator is consistent and asymptotically normal, and derive a consistent estimator of the asymptotic variance. If the proportional hazards assumption holds, the efficiency of the estimator can be improved by applying the covariate adjustment method based on the semiparametric theory proposed by Lu and Tsiatis (Biometrika 95:679-694, 2008).     

Guaranteed Local Maximum Likelihood Detection of a Change Point in Nonparametric Logistic Regression

We consider nonparametric logistic regression and propose a generalized likelihood test for detecting a threshold effect that indicates a relationship between some risk factor and a defined outcome above the threshold but none below it. One important field of application is occupational medicine and in particular, epidemiological studies. In epidemiological studies, segmented fully parametric logistic regression models are often threshold models, where it is assumed that the exposure has no influence on a response up to a possible unknown threshold, and has an effect beyond that threshold. Finding efficient methods for detection and estimation of a threshold is a very important task in these studies. This article proposes such methods in a context of nonparametric logistic regression. We use a local version of unknown likelihood functions and show that under rather common assumptions the asymptotic power of our test is one. We present a guaranteed non asymptotic upper bound for the significance level of the proposed test. If applying the test yields the acceptance of the conclusion that there was a change point (and hence a threshold limit value), we suggest using the local maximum likelihood estimator of the change point and consider the asymptotic properties of this estimator.     

Point estimation after early stopping in a repeated measures trial

When the individual measurements are statistically independent, the maximum likelihood estimator calculated at the end of a sequential procedure overestimates the underlying effect. There are many clinical trials in which we are interested in comparing changes in responses between two treatment groups sequentially. Lee and DeMets (1991, JASA 86, 757–762) proposed a group sequential method for comparing rates of change when a response variable is measured for eaeh patient at successive follow-up visits. They assumed that the response follows the linear mixed effects model and derived the asymptotic joint distribution of the sequentially computed statistics. In this article, we consider the maximum likelihood estimator (MLE), the median unbiased estimator (MUE) and the midpoint of a 100(1-α)% confidence interval as point estimators for the rate of change in the linear mixed effects model, and investigate their properties by Monte Carlo simulation.     

Minimum Hellinger distance estimation for randomized play the winner design

Response-adaptive designs in clinical trials incorporate information from prior patient responses in order to assign better performing treatments to the future patients of a clinical study. An example of a response adaptive design that has received much attention in recent years is the randomized play the winner design (RPWD). Beran [1977. Minimum Hellinger distance estimates for parametric models. Ann. Statist. 5, 445–463] investigated the problem of minimum Hellinger distance procedure (MHDP) for continuous data and showed that minimum Hellinger distance estimator (MHDE) of a finite dimensional parameter is as efficient as the MLE (maximum likelihood estimator) under a true model assumption. This paper develops minimum Hellinger distance methodology for data generated using RPWD. A new algorithm using the Monte Carlo approximation to the estimating equation is proposed. Consistency and asymptotic normality of the estimators are established and the robustness and small sample performance of the estimators are illustrated using simulations. The methodology when applied to the clinical trial data conducted by Eli-Lilly and Company, brings out the treatment effect in one of the strata using the frequentist techniques compared to the Bayesian argument of Tamura et al [1994. A case study of an adaptive clinical trialin the treatment of out-patients with depressive disorder. J. Amer. Statist. Assoc. 89, 768–776].     

On bias reduction estimators of skew-normal and skew-t distributions

A particular concerns of researchers in statistical inference is bias in parameters estimation. Maximum likelihood estimators are often biased and for small sample size, the first order bias of them can be large and so it may influence the efficiency of the estimator. There are different methods for reduction of this bias. In this paper, we proposed a modified maximum likelihood estimator for the shape parameter of two popular skew distributions, namely skew-normal and skew-t, by offering a new method. We show that this estimator has lower asymptotic bias than the maximum likelihood estimator and is more efficient than those based on the existing methods.     

A generalized drop-the-loser rule for multi-treatment clinical trials

Urn models are popular for response adaptive designs in clinical studies. Among different urn models, Ivanova's drop-the-loser rule is capable of producing superior adaptive treatment allocation schemes. Ivanova [2003. A play-the-winner-type urn model with reduced variability. Metrika 58, 1–13] obtained the asymptotic normality only for two treatments. Recently, Zhang et al. [2007. Generalized drop-the-loser urn for clinical trials with delayed responses. Statist. Sinica, in press] extended the drop-the-loser rule to tackle more general circumstances. However, their discussion is also limited to only two treatments. In this paper, the drop-the-loser rule is generalized to multi-treatment clinical trials, and delayed responses are allowed. Moreover, the rule can be used to target any desired pre-specified allocation proportion. Asymptotic properties, including strong consistency and asymptotic normality, are also established for general multi-treatment cases.     

Estimation for Location-Scale Models with Censored Data

This article considers a class of estimators for the location and scale parameters in the location-scale model based on ‘synthetic data’ when the observations are randomly censored on the right. The asymptotic normality of the estimators is established using counting process and martingale techniques when the censoring distribution is known and unknown, respectively. In the case when the censoring distribution is known, we show that the asymptotic variances of this class of estimators depend on the data transformation and have a lower bound which is not achievable by this class of estimators. However, in the case that the censoring distribution is unknown and estimated by the Kaplan–Meier estimator, this class of estimators has the same asymptotic variance and attains the lower bound for variance for the case of known censoring distribution. This is different from censored regression analysis, where asymptotic variances depend on the data transformation. Our method has three valuable advantages over the method of maximum likelihood estimation. First, our estimators are available in a closed form and do not require an iterative algorithm. Second, simulation studies show that our estimators being moment-based are comparable to maximum likelihood estimators and outperform them when sample size is small and censoring rate is high. Third, our estimators are more robust to model misspecification than maximum likelihood estimators. Therefore, our method can serve as a competitive alternative to the method of maximum likelihood in estimation for location-scale models with censored data. A numerical example is presented to illustrate the proposed method.     

Estimation and inference of combining quantile and least-square regressions with missing data

In this paper, we consider how to incorporate quantile information to improve estimator efficiency for regression model with missing covariates. We combine the quantile information with least-squares normal equations and construct an unbiased estimating equations (EEs). The lack of smoothness of the objective EEs is overcome by replacing them with smooth approximations. The maximum smoothed empirical likelihood (MSEL) estimators are established based on inverse probability weighted (IPW) smoothed EEs and their asymptotic properties are studied under some regular conditions. Moreover, we develop two novel testing procedures for the underlying model. The finite-sample performance of the proposed methodology is examined by simulation studies. A real example is used to illustrate our methods.     

Penalized Maximum Likelihood Estimator for Normal Mixtures

The estimation of the parameters of a mixture of Gaussian densities is considered, within the framework of maximum likelihood. Due to unboundedness of the likelihood function, the maximum likelihood estimator fails to exist. We adopt a solution to likelihood function degeneracy which consists in penalizing the likelihood function. The resulting penalized likelihood function is then bounded over the parameter space and the existence of the penalized maximum likelihood estimator is granted. As original contribution we provide asymptotic properties, and in particular a consistency proof, for the penalized maximum likelihood estimator. Numerical examples are provided in the finite data case, showing the performances of the penalized estimator compared to the standard one.     

Bias-reduced extreme quantile estimators of Weibull tail-distributions

In this paper, we consider the problem of estimating an extreme quantile of a Weibull tail-distribution. The new extreme quantile estimator has a reduced bias compared to the more classical ones proposed in the literature. It is based on an exponential regression model that was introduced in Diebolt et al. [2007. Bias-reduced estimators of the Weibull-tail coefficient. Test, to appear]. The asymptotic normality of the extreme quantile estimator is established. We also introduce an adaptive selection procedure to determine the number of upper order statistics to be used. A simulation study as well as an application to a real data set is provided in order to prove the efficiency of the above-mentioned methods.