Consistency of the NPML Estimator in the Right-Censored Transformation Model

Abstract.  This paper studies the representation and large-sample consistency for non-parametric maximum likelihood estimators (NPMLEs) of an unknown baseline continuous cumulative-hazard-type function and parameter of group survival difference, based on right-censored two-sample survival data with marginal survival function assumed to follow a transformation model, a slight generalization of the class of frailty survival regression models. The paper's main theoretical results are existence and unique a.s. limit, characterized variationally, for large data samples of the NPMLE of baseline nuisance function in an appropriately defined neighbourhood of the true function when the group difference parameter is fixed, leading to consistency of the NPMLE when the difference parameter is fixed at a consistent estimator of its true value. The joint NPMLE is also shown to be consistent. An algorithm for computing it numerically, based directly on likelihood equations in place of the expectation-maximization (EM) algorithm, is illustrated with real data.     

Constrained nonparametric maximum likelihood estimation of stochastically ordered survivor functions

This paper considers estimators of survivor functions subject to a stochastic ordering constraint based on right censored data. We present the constrained nonparametric maximum likelihood estimator (C‐NPMLE) of the survivor functions in one‐and two‐sample settings where the survivor distributions could be discrete or continuous and discuss the non‐uniqueness of the estimators. We also present a computationally efficient algorithm to obtain the C‐NPMLE. To address the possibility of non‐uniqueness of the C‐NPMLE of $S_1(t)$ when $S_1(t)\le S_2(t)$ , we consider the maximum C‐NPMLE (MC‐NPMLE) of $S_1(t)$ . In the one‐sample case with arbitrary upper bound survivor function $S_2(t)$ , we present a novel and efficient algorithm for finding the MC‐NPMLE of $S_1(t)$ . Dykstra ( 1982 ) also considered constrained nonparametric maximum likelihood estimation for such problems, however, as we show, Dykstra's method has an error and does not always give the C‐NPMLE. We corrected this error and simulation shows improvement in efficiency compared to Dykstra's estimator. Confidence intervals based on bootstrap methods are proposed and consistency of the estimators is proved. Data from a study on larynx cancer are analysed to illustrate the method. The Canadian Journal of Statistics 40: 22–39; 2012 © 2012 Statistical Society of Canada     

Estimating the survival function based on the semi-Markov model for dependent censoring

In this paper, we study a nonparametric maximum likelihood estimator (NPMLE) of the survival function based on a semi-Markov model under dependent censoring. We show that the NPMLE is asymptotically normal and achieves asymptotic nonparametric efficiency. We also provide a uniformly consistent estimator of the corresponding asymptotic covariance function based on an information operator. The finite-sample performance of the proposed NPMLE is examined with simulation studies, which show that the NPMLE has smaller mean squared error than the existing estimators and its corresponding pointwise confidence intervals have reasonable coverages. A real example is also presented.     

Nonparametric analysis of interval censored and doubly truncated data

Doubly truncated data appear in a number of applications, including astronomy and survival analysis. For double-truncated data, the lifetime T is observable only when U≤T≤V, where U and V are the left-truncated and right-truncated time, respectively. In some situations, the lifetime T also suffers interval censoring. Using the EM algorithm of Turnbull [The empirical distribution function with arbitrarily grouped censored and truncated data, J. R. Stat. Soc. Ser. B 38 (1976), pp. 290–295] and iterative convex minorant algorithm [P. Groeneboom and J.A. Wellner, Information Bounds and Nonparametric Maximum Likelihood Estimation, Birkhäuser, Basel, 1992], we study the performance of the nonparametric maximum-likelihood estimates (NPMLEs) of the distribution function of T. Simulation results indicate that the NPMLE performs adequately for the finite sample.     

Nonparametric estimators of survival function under the mixed case interval-censored model with left truncation

It is well known that the nonparametric maximum likelihood estimator (NPMLE) can severely underestimate the survival probabilities at early times for left-truncated and interval-censored (LT-IC) data. For arbitrarily truncated and censored data, Pan and Chappel (JAMA Stat Probab Lett 38:49–57, 1998a, Biometrics 54:1053–1060, 1998b) proposed a nonparametric estimator of the survival function, called the iterative Nelson estimator (INE). Their simulation study showed that the INE performed well in overcoming the under-estimation of the survival function from the NPMLE for LT-IC data. In this article, we revisit the problem of inconsistency of the NPMLE. We point out that the inconsistency is caused by the likelihood function of the left-censored observations, where the left-truncated variables are used as the left endpoints of censoring intervals. This can lead to severe underestimation of the survival function if the NPMLE is obtained using Turnbull’s (JAMA 38:290–295, 1976) EM algorithm. To overcome this problem, we propose a modified maximum likelihood estimator (MMLE) based on a modified likelihood function, where the left endpoints of censoring intervals for left-censored observations are the maximum of left-truncated variables and the estimated left endpoint of the support of the left-censored times. Simulation studies show that the MMLE performs well for finite sample and outperforms both the INE and NPMLE.

     

The nonparametric maximum likelihood estimator for middle-censored data

In this note, we consider data subjected to middle censoring where the variable of interest becomes unobservable when it falls within an interval of censorship. We demonstrate that the nonparametric maximum likelihood estimator (NPMLE) of distribution function can be obtained by using Turnbull's (1976) EM algorithm or self-consistent estimating equation (Jammalamadaka and Mangalam, 2003) with an initial estimator which puts mass only on the innermost intervals. The consistency of the NPMLE can be established based on the asymptotic properties of self-consistent estimators (SCE) with mixed interval-censored data ( [Yu et al., 2000] and [Yu et al., 2001]).     

The random partition masking model for interval-censored and masked competing risks data

We study the nonparametric maximum likelihood estimate (NPMLE) of the cdf or sub-distribution functions of the failure time for the failure causes in a series system. The study is motivated by a cancer research data (from the Memorial Sloan-Kettering Cancer Center) with interval-censored time and masked failure cause. The NPMLE based on this data set suggests that the existing masking models are not appropriate. We propose a new model called the random partition masking model, which does not rely on the commonly used symmetry assumption (namely, given the failure cause, the probability of observing the masked failure causes is independent of the failure time; see Flehinger et al. Inference about defects in the presence of masking, Technometrics 38 (1996), pp. 247–255). The RPM model is easier to implement in simulation studies than the existing models. We discuss the algorithms for computing the NPMLE and study its asymptotic properties. Our simulation and data analysis indicate that the NPMLE is feasible for a moderate sample size.     

Nonparametric estimation of the cumulative intensities in an interval censored competing risks model

The nonparametric maximum likelihood estimation (NPMLE) of the distribution function from the interval censored (IC) data has been extensively studied in the extant literature. The NPMLE was also developed for the subdistribution functions in an IC competing risks model and in an illness-death model under various interval-censoring scenarios. But the important problem of estimation of the cumulative intensities (CIs) in the interval-censored models has not been considered previously. We develop the NPMLE of the CI in a simple alive/dead model and of the CIs in a competing risks model. Assuming that data are generated by a discrete and finite mixed case interval censoring mechanism we provide a discussion and the simulation study of the asymptotic properties of the NPMLEs of the CIs. In particular we show that they are asymptotically unbiased; in contrast the ad hoc estimators presented in extant literature are substantially biased. We illustrate our methods with the data from a prospective cohort study on the longevity of dental veneers.     

Stochastic EM algorithm for generalized exponential cure rate model and an empirical study

In this paper, we consider two well-known parametric long-term survival models, namely, the Bernoulli cure rate model and the promotion time (or Poisson) cure rate model. Assuming the long-term survival probability to depend on a set of risk factors, the main contribution is in the development of the stochastic expectation maximization (SEM) algorithm to determine the maximum likelihood estimates of the model parameters. We carry out a detailed simulation study to demonstrate the performance of the proposed SEM algorithm. For this purpose, we assume the lifetimes due to each competing cause to follow a two-parameter generalized exponential distribution. We also compare the results obtained from the SEM algorithm with those obtained from the well-known expectation maximization (EM) algorithm. Furthermore, we investigate a simplified estimation procedure for both SEM and EM algorithms that allow the objective function to be maximized to split into simpler functions with lower dimensions with respect to model parameters. Moreover, we present examples where the EM algorithm fails to converge but the SEM algorithm still works. For illustrative purposes, we analyze a breast cancer survival data. Finally, we use a graphical method to assess the goodness-of-fit of the model with generalized exponential lifetimes.     

The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components

The paper is focussing on some recent developments in nonparametric mixture distributions. It discusses nonparametric maximum likelihood estimation of the mixing distribution and will emphasize gradient type results, especially in terms of global results and global convergence of algorithms such as vertex direction or vertex exchange method. However, the NPMLE (or the algorithms constructing it) provides also an estimate of the number of components of the mixing distribution which might be not desirable for theoretical reasons or might be not allowed from the physical interpretation of the mixture model. When the number of components is fixed in advance, the before mentioned algorithms can not be used and globally convergent algorithms do not exist up to now. Instead, the EM algorithm is often used to find maximum likelihood estimates. However, in this case multiple maxima are often occuring. An example from a meta-analyis of vitamin A and childhood mortality is used to illustrate the considerable, inferential importance of identifying the correct global likelihood. To improve the behavior of the EM algorithm we suggest a combination of gradient function steps and EM steps to achieve global convergence leading to the EM algorithm with gradient function update (EMGFU). This algorithms retains the number of components to be exactly k and typically converges to the global maximum. The behavior of the algorithm is highlighted at hand of several examples.     

Computing the log concave NPMLE for interval censored data

In analyzing interval censored data, a non-parametric estimator is often desired due to difficulties in assessing model fits. Because of this, the non-parametric maximum likelihood estimator (NPMLE) is often the default estimator. However, the estimates for values of interest of the survival function, such as the quantiles, have very large standard errors due to the jagged form of the estimator. By forcing the estimator to be constrained to the class of log concave functions, the estimator is ensured to have a smooth survival estimate which has much better operating characteristics than the unconstrained NPMLE, without needing to specify a parametric family or smoothing parameter. In this paper, we first prove that the likelihood can be maximized under a finite set of parameters under mild conditions, although the log likelihood function is not strictly concave. We then present an efficient algorithm for computing a local maximum of the likelihood function. Using our fast new algorithm, we present evidence from simulated current status data suggesting that the rate of convergence of the log-concave estimator is faster (between \(n^{2/5}\) and \(n^{1/2}\)) than the unconstrained NPMLE (between \(n^{1/3}\) and \(n^{1/2}\)).     

The Two-interval Line-segment Problem

In this paper we define and study the non-parametric maximum likelihood estimator (NPMLE) in the one-dimensional line-segment problem, where we observe line-segments on the real line through an interval with a gap which is smaller than the two remaining intervals. We define the self-consistency equations for the NPMLE and provide a quick algorithm for solving them. We prove supremum norm weak convergence to a Gaussian process and efficiency of the NPMLE. The problem has a geological application in the study of the lifespan of species     

Bayesian Parametric Accelerated Failure Time Spatial Model and its Application to Prostate Cancer

Prostate cancer is the most common cancer diagnosed in American men and the second leading cause of death from malignancies. There are large geographical variation and racial disparities existing in the survival rate of prostate cancer. Much work on the spatial survival model is based on the proportional hazards model, but few focused on the accelerated failure time model. In this paper, we investigate the prostate cancer data of Louisiana from the SEER program and the violation of the proportional hazards assumption suggests the spatial survival model based on the accelerated failure time model is more appropriate for this data set. To account for the possible extra-variation, we consider spatially-referenced independent or dependent spatial structures. The deviance information criterion (DIC) is used to select a best fitting model within the Bayesian frame work. The results from our study indicate that age, race, stage and geographical distribution are significant in evaluating prostate cancer survival.     

Discrete regularized discriminant analysis

A method of regularized discriminant analysis for discrete data, denoted DRDA, is proposed. This method is related to the regularized discriminant analysis conceived by Friedman (1989) in a Gaussian framework for continuous data. Here, we are concerned with discrete data and consider the classification problem using the multionomial distribution. DRDA has been conceived in the small-sample, high-dimensional setting. This method has a median position between multinomial discrimination, the first-order independence model and kernel discrimination. DRDA is characterized by two parameters, the values of which are calculated by minimizing a sample-based estimate of future misclassification risk by cross-validation. The first parameter is acomplexity parameter which provides class-conditional probabilities as a convex combination of those derived from the full multinomial model and the first-order independence model. The second parameter is asmoothing parameter associated with the discrete kernel of Aitchison and Aitken (1976). The optimal complexity parameter is calculated first, then, holding this parameter fixed, the optimal smoothing parameter is determined. A modified approach, in which the smoothing parameter is chosen first, is discussed. The efficiency of the method is examined with other classical methods through application to data.     

Mixed Discrete and Continuous Cox Regression Model

The Cox (1972) regression model is extended to include discrete and mixed continuous/discrete failure time data by retaining the multiplicative hazard rate form of the absolutely continuous model. Application of martingale arguments to the regression parameter estimating function show the Breslow (1974) estimator to be consistent and asymptotically Gaussian under this model. A computationally convenient estimator of the variance of the score function can be developed, again using martingale arguments. This estimator reduces to the usual hypergeometric form in the special case of testing equality of several survival curves, and it leads more generally to a convenient consistent variance estimator for the regression parameter. A small simulation study is carried out to study the regression parameter estimator and its variance estimator under the discrete Cox model special case and an application to a bladder cancer recurrence dataset is provided.     

Performance of nonparametric maximum likelihood estimator in a

When the probability of selecting an individual in a population is propor­tional to its lifelength, it is called length biased sampling. A nonparametric maximum likelihood estimator (NPMLE) of survival in a length biased sam­ple is given in Vardi (1982). In this study, we examine the performance of Vardi's NPMLE in estimating the true survival curve when observations are from a length biased sample. We also compute estimators based on a linear combination (LCE) of empirical distribution function (EDF) estimators and weighted estimators. In our simulations, we consider observations from a mix­ture of two different distributions, one from F and the other from G which is a length biased distribution of F. Through a series of simulations with vari­ous proportions of length biasing in a sample, we show that the NPMLE and the LCE closely approximate the true survival curve. Throughout the sur­vival curve, the EDF estimators overestimate the survival. We also consider a case where the observations are from three different weighted distributions, Again, both the NPMLE and the LCE closely approximate the true distribu­tion, indicating that the length biasedness is properly adjusted for. Finally, an efficiency study shows that Vardi's estimators are more efficient than the EDF estimators in the lower percentiles of the survival curves.     

Estimating the causal effects in randomized trials for survival data with a cure fraction and non compliance

We consider causal inference in randomized studies for survival data with a cure fraction and all-or-none treatment non compliance. To describe the causal effects, we consider the complier average causal effect (CACE) and the complier effect on survival probability beyond time t (CESP), where CACE and CESP are defined as the difference of cure rate and non cured subjects’ survival probability between treatment and control groups within the complier class. These estimands depend on the distributions of survival times in treatment and control groups. Given covariates and latent compliance type, we model these distributions with transformation promotion time cure model whose parameters are estimated by maximum likelihood. Both the infinite dimensional parameter in the model and the mixture structure of the problem create some computational difficulties which are overcome by an expectation-maximization (EM) algorithm. We show the estimators are consistent and asymptotically normal. Some simulation studies are conducted to assess the finite-sample performance of the proposed approach. We also illustrate our method by analyzing a real data from the Healthy Insurance Plan of Greater New York.     

Higher order inference for stress–strength reliability with independent Burr-type X distributions

In this paper, a small-sample asymptotic method is proposed for higher order inference in the stress–strength reliability model, R=P(Y<X), where X and Y are distributed independently as Burr-type X distributions. In a departure from the current literature, we allow the scale parameters of the two distributions to differ, and the likelihood-based third-order inference procedure is applied to obtain inference for R. The difficulty of the implementation of the method is in obtaining the the constrained maximum likelihood estimates (MLE). A penalized likelihood method is proposed to handle the numerical complications of maximizing the constrained likelihood model. The proposed procedures are illustrated using a sample of carbon fibre strength data. Our results from simulation studies comparing the coverage probabilities of the proposed small-sample asymptotic method with some existing large-sample asymptotic methods show that the proposed method is very accurate even when the sample sizes are small.     

Estimating the concordance probability in a survival analysis with a discrete number of risk groups

A clinical risk classification system is an important component of a treatment decision algorithm. A measure used to assess the strength of a risk classification system is discrimination, and when the outcome is survival time, the most commonly applied global measure of discrimination is the concordance probability. The concordance probability represents the pairwise probability of lower patient risk given longer survival time. The c-index and the concordance probability estimate have been used to estimate the concordance probability when patient-specific risk scores are continuous. In the current paper, the concordance probability estimate and an inverse probability censoring weighted c-index are modified to account for discrete risk scores. Simulations are generated to assess the finite sample properties of the concordance probability estimate and the weighted c-index. An application of these measures of discriminatory power to a metastatic prostate cancer risk classification system is examined.     

A Nonparametric Estimator of Survival Functions for Arbitrarily Truncated and Censored Data

It is well-known that the nonparametric maximum likelihood estimator (NPMLE) may severely under-estimate the survival function with left truncated data. Based on the Nelson estimator (for right censored data) and self-consistency we suggest a nonparametric estimator of the survival function, the iterative Nelson estimator (INE), for arbitrarily truncated and censored data, where only few nonparametric estimators are available. By simulation we show that the INE does well in overcoming the under-estimation of the survival function from the NPMLE for left-truncated and interval-censored data. An interesting application of the INE is as a diagnostic tool for other estimators, such as the monotone MLE or parametric MLEs. The methodology is illustrated by application to two real world problems: the Channing House and the Massachusetts Health Care Panel Study data sets.