Computationally simple estimation and improved efficiency for special cases of double truncation

Doubly truncated survival data arise when event times are observed only if they occur within subject specific intervals of times. Existing iterative estimation procedures for doubly truncated data are computationally intensive (Turnbull 38:290–295, 1976; Efron and Petrosian 94:824–825, 1999; Shen 62:835–853, 2010a). These procedures assume that the event time is independent of the truncation times, in the sample space that conforms to their requisite ordering. This type of independence is referred to as quasi-independence. In this paper we identify and consider two special cases of quasi-independence: complete quasi-independence and complete truncation dependence. For the case of complete quasi-independence, we derive the nonparametric maximum likelihood estimator in closed-form. For the case of complete truncation dependence, we derive a closed-form nonparametric estimator that requires some external information, and a semi-parametric maximum likelihood estimator that achieves improved efficiency relative to the standard nonparametric maximum likelihood estimator, in the absence of external information. We demonstrate the consistency and potentially improved efficiency of the estimators in simulation studies, and illustrate their use in application to studies of AIDS incubation and Parkinson’s disease age of onset.     

Computing the log concave NPMLE for interval censored data

In analyzing interval censored data, a non-parametric estimator is often desired due to difficulties in assessing model fits. Because of this, the non-parametric maximum likelihood estimator (NPMLE) is often the default estimator. However, the estimates for values of interest of the survival function, such as the quantiles, have very large standard errors due to the jagged form of the estimator. By forcing the estimator to be constrained to the class of log concave functions, the estimator is ensured to have a smooth survival estimate which has much better operating characteristics than the unconstrained NPMLE, without needing to specify a parametric family or smoothing parameter. In this paper, we first prove that the likelihood can be maximized under a finite set of parameters under mild conditions, although the log likelihood function is not strictly concave. We then present an efficient algorithm for computing a local maximum of the likelihood function. Using our fast new algorithm, we present evidence from simulated current status data suggesting that the rate of convergence of the log-concave estimator is faster (between \(n^{2/5}\) and \(n^{1/2}\)) than the unconstrained NPMLE (between \(n^{1/3}\) and \(n^{1/2}\)).     

A pool-adjacent-violators type algorithm for non-parametric estimation of current status data with dependent censoring

A likelihood based approach to obtaining non-parametric estimates of the failure time distribution is developed for the copula based model of Wang et al. (Lifetime Data Anal 18:434–445, 2012) for current status data under dependent observation. Maximization of the likelihood involves a generalized pool-adjacent violators algorithm. The estimator coincides with the standard non-parametric maximum likelihood estimate under an independence model. Confidence intervals for the estimator are constructed based on a smoothed bootstrap. It is also shown that the non-parametric failure distribution is only identifiable if the copula linking the observation and failure time distributions is fully-specified. The method is illustrated on a previously analyzed tumorigenicity dataset.     

Parametric Estimation of Menarcheal Age Distribution Based on Recall Data

Menarche, the onset of menstruation, is an important maturational event of female childhood. Most of the studies of age at menarche make use of dichotomous (status quo) data. More information can be harnessed from recall data, but such data are often censored in a informative way. We show that the usual maximum likelihood estimator based on interval censored data, which ignores the informative nature of censoring, can be biased and inconsistent. We propose a parametric estimator of the menarcheal age distribution on the basis of a realistic model of the recall phenomenon. We identify the additional information contained in the recall data and demonstrate theoretically as well as through simulations the advantage of the maximum likelihood estimator based on recall data over that based on status quo data.     

Misclassification of current status data

We describe a simple method for nonparametric estimation of a distribution function based on current status data where observations of current status information are subject to misclassification. Nonparametric maximum likelihood techniques lead to use of a straightforward set of adjustments to the familiar pool-adjacent-violators estimator used when misclassification is assumed absent. The methods consider alternative misclassification models and are extended to regression models for the underlying survival time. The ideas are motivated by and applied to an example on human papilloma virus (HPV) infection status of a sample of women examined in San Francisco.     

A new kind of stochastic restricted biased estimator for logistic regression model

In the logistic regression model, the variance of the maximum likelihood estimator is inflated and unstable when the multicollinearity exists in the data. There are several methods available in literature to overcome this problem. We propose a new stochastic restricted biased estimator. We study the statistical properties of the proposed estimator and compare its performance with some existing estimators in the sense of scalar mean squared criterion. An example and a simulation study are provided to illustrate the performance of the proposed estimator.KEYWORDS: Logistic regression, maximum likelihood estimator, mean squared error matrix, ridge regression, simulation study, stochastic restricted estimatorMathematics Subject Classifications: Primary 62J05, Secondary 62J07     

Nonparametric estimators of survival function under the mixed case interval-censored model with left truncation

It is well known that the nonparametric maximum likelihood estimator (NPMLE) can severely underestimate the survival probabilities at early times for left-truncated and interval-censored (LT-IC) data. For arbitrarily truncated and censored data, Pan and Chappel (JAMA Stat Probab Lett 38:49–57, 1998a, Biometrics 54:1053–1060, 1998b) proposed a nonparametric estimator of the survival function, called the iterative Nelson estimator (INE). Their simulation study showed that the INE performed well in overcoming the under-estimation of the survival function from the NPMLE for LT-IC data. In this article, we revisit the problem of inconsistency of the NPMLE. We point out that the inconsistency is caused by the likelihood function of the left-censored observations, where the left-truncated variables are used as the left endpoints of censoring intervals. This can lead to severe underestimation of the survival function if the NPMLE is obtained using Turnbull’s (JAMA 38:290–295, 1976) EM algorithm. To overcome this problem, we propose a modified maximum likelihood estimator (MMLE) based on a modified likelihood function, where the left endpoints of censoring intervals for left-censored observations are the maximum of left-truncated variables and the estimated left endpoint of the support of the left-censored times. Simulation studies show that the MMLE performs well for finite sample and outperforms both the INE and NPMLE.

     

Pseudo MLE for semiparametric transformation model with doubly truncated data

In this article, we consider the efficient estimation of the semiparametric transformation model with doubly truncated data. We propose a two-step approach for obtaining the pseudo maximum likelihood estimators (PMLE) of regression parameters. In the first step, the truncation time distribution is estimated by the nonparametric maximum likelihood estimator (Shen, 2010a) when the distribution function $K$ of the truncation time is unspecified or by the conditional maximum likelihood estimator (Bilker and Wang, 1996) when $K$ is parameterized. In the second step, using the pseudo complete-data likelihood function with the estimated distribution of truncation time, we propose expectation–maximization algorithms for obtaining the PMLE. We establish the consistency of the PMLE. The simulation study indicates that the PMLE performs well in finite samples. The proposed method is illustrated using an AIDS data set.     

A new linearized ridge Poisson estimator in the presence of multicollinearity

Poisson regression is a very commonly used technique for modeling the count data in applied sciences, in which the model parameters are usually estimated by the maximum likelihood method. However, the presence of multicollinearity inflates the variance of maximum likelihood (ML) estimator and the estimated parameters give unstable results. In this article, a new linearized ridge Poisson estimator is introduced to deal with the problem of multicollinearity. Based on the asymptotic properties of ML estimator, the bias, covariance and mean squared error of the proposed estimator are obtained and the optimal choice of shrinkage parameter is derived. The performance of the existing estimators and proposed estimator is evaluated through Monte Carlo simulations and two real data applications. The results clearly reveal that the proposed estimator outperforms the existing estimators in the mean squared error sense.KEYWORDS: Poisson regression, multicollinearity, ridge Poisson estimator, linearized ridge regression estimator, mean squared errorMathematics Subject Classifications: 62J07, 62F10     

CAMCR: Computer-Assisted Mixture model analysis for Capture–Recapture count data

Population size estimation with discrete or nonparametric mixture models is considered, and reliable ways of construction of the nonparametric mixture model estimator are reviewed and set into perspective. Construction of the maximum likelihood estimator of the mixing distribution is done for any number of components up to the global nonparametric maximum likelihood bound using the EM algorithm. In addition, the estimators of Chao and Zelterman are considered with some generalisations of Zelterman’s estimator. All computations are done with CAMCR, a special software developed for population size estimation with mixture models. Several examples and data sets are discussed and the estimators illustrated. Problems using the mixture model-based estimators are highlighted.     

Identity for the NPMLE in Censored Data Models

We derive an identity for nonparametric maximum likelihood estimators (NPMLE) and regularized MLEs in censored data models which expresses the standardized maximum likelihood estimator in terms of the standardized empirical process. This identity provides an effective starting point in proving both consistency and efficiency of NPMLE and regularized MLE. The identity and corresponding method for proving efficiency is illustrated for the NPMLE in the univariate right-censored data model, the regularized MLE in the current status data model and for an implicit NPMLE based on a mixture of right-censored and current status data. Furthermore, a general algorithm for estimation of the limiting variance of the NPMLE is provided. This revised version was published online in July 2006 with corrections to the Cover Date.     

Bayes and maximum likelihood for $$L^1$$-Wasserstein deconvolution of Laplace mixtures

We consider the problem of recovering a distribution function on the real line from observations additively contaminated with errors following the standard Laplace distribution. Assuming that the latent distribution is completely unknown leads to a nonparametric deconvolution problem. We begin by studying the rates of convergence relative to the \(L^2\)-norm and the Hellinger metric for the direct problem of estimating the sampling density, which is a mixture of Laplace densities with a possibly unbounded set of locations: the rate of convergence for the Bayes’ density estimator corresponding to a Dirichlet process prior over the space of all mixing distributions on the real line matches, up to a logarithmic factor, with the \(n^{-3/8}\log ^{1/8}n\) rate for the maximum likelihood estimator. Then, appealing to an inversion inequality translating the \(L^2\)-norm and the Hellinger distance between general kernel mixtures, with a kernel density having polynomially decaying Fourier transform, into any \(L^p\)-Wasserstein distance, \(p\ge 1\), between the corresponding mixing distributions, provided their Laplace transforms are finite in some neighborhood of zero, we derive the rates of convergence in the \(L^1\)-Wasserstein metric for the Bayes’ and maximum likelihood estimators of the mixing distribution. Merging in the \(L^1\)-Wasserstein distance between Bayes and maximum likelihood follows as a by-product, along with an assessment on the stochastic order of the discrepancy between the two estimation procedures.     

On nonparametric maximum likelihood estimation with interval censoring and left truncation

Summary.  A graph theoretical approach is employed to describe the support set of the nonparametric maximum likelihood estimator for the cumulative distribution function given interval-censored and left-truncated data. A necessary and sufficient condition for the existence of a nonparametric maximum likelihood estimator is then derived. Two previously analysed data sets are revisited.     

A Bezier curve method in interval-censored data

In this paper we propose a Bezier curve method to estimate the survival function and the median survival time in interval-censored data. We compare the proposed estimator with other existing methods such as the parametric method, the single point imputation method, and the nonparametric maximum likelihood estimator through extensive numerical studies, and it is shown that the proposed estimator performs better than others in the sense of mean squared error and mean integrated squared error. An illustrative example based on a real data set is given.     

Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator

The penalized maximum likelihood estimator (PMLE) has been widely used for variable selection in high-dimensional data. Various penalty functions have been employed for this purpose, e.g., Lasso, weighted Lasso, or smoothly clipped absolute deviations. However, the PMLE can be very sensitive to outliers in the data, especially to outliers in the covariates (leverage points). In order to overcome this disadvantage, the usage of the penalized maximum trimmed likelihood estimator (PMTLE) is proposed to estimate the unknown parameters in a robust way. The computation of the PMTLE takes advantage of the same technology as used for PMLE but here the estimation is based on subsamples only. The breakdown point properties of the PMTLE are discussed using the notion of $d$ -fullness. The performance of the proposed estimator is evaluated in a simulation study for the classical multiple linear and Poisson linear regression models.     

Nonparametric Bayes estimation of gap-time distribution with recurrent event data

Nonparametric Bayes (NPB) estimation of the gap-time survivor function governing the time to occurrence of a recurrent event in the presence of censoring is considered. In our Bayesian approach, the gap-time distribution, denoted by F, has a Dirichlet process prior with parameter α. We derive NPB and nonparametric empirical Bayes (NPEB) estimators of the survivor function F?=1?F and construct point-wise credible intervals. The resulting Bayes estimator of F? extends that based on single-event right-censored data, and the PL-type estimator is a limiting case of this Bayes estimator. Through simulation studies, we demonstrate that the PL-type estimator has smaller biases but higher root-mean-squared errors (RMSEs) than those of the NPB and the NPEB estimators. Even in the case of a mis-specified prior measure parameter α, the NPB and the NPEB estimators have smaller RMSEs than the PL-type estimator, indicating robustness of the NPB and NPEB estimators. In addition, the NPB and NPEB estimators are smoother (in some sense) than the PL-type estimator.     

Nonparametric Bayesian estimation of survival function under random left truncation

Several observational studies give rise to randomly left truncated data. In a nonparametric model for such data X denotes a variable of interest, T denotes the truncation variable and the distributions of both X and T are left unspecified. For this model, the product-limit estimator, which is also the maximum likelihood estimator of the survival curve, has been widely discussed. In this article, a nonparametric Bayes estimator of the survival function based on randomly left truncated data and Dirichlet process prior is presented. Some new results on the mixtures of Dirichlet processes in the context of truncated data are obtained. These results are then used to derive the Bayes estimator of the survival function under squared error loss. The weak convergence of the Bayes estimator is studied. An example using transfusion related AIDS data quoted in Kalbfleisch and Lawless (1989) is considered.     

Cox regression for current status data with mismeasured covariates

Covariate measurement error problems have been extensively studied in the context of right‐censored data but less so for current status data. Motivated by the zebrafish basal cell carcinoma (BCC) study, where the occurrence time of BCC was only known to lie before or after a sacrifice time and where the covariate (Sonic hedgehog expression) was measured with error, the authors describe a semiparametric maximum likelihood method for analyzing current status data with mismeasured covariates under the proportional hazards model. They show that the estimator of the regression coefficient is asymptotically normal and efficient and that the profile likelihood ratio test is asymptotically Chi‐squared. They also provide an easily implemented algorithm for computing the estimators. They evaluate their method through simulation studies, and illustrate it with a real data example. The Canadian Journal of Statistics 39: 73–88; 2011 © 2011 Statistical Society of Canada     

A Generalization of Turnbull’s Estimator for Interval-Censored and Doubly Truncated Data

Nonparametric estimates of the conditional distribution of a response variable given a covariate are important for data exploration purposes. In this article, we propose a nonparametric estimator of the conditional distribution function in the case where the response variable is subject to interval censoring and double truncation. Using the approach of Dehghan and Duchesne (2011), the proposed method consists in adding weights that depend on the covariate value in the self-consistency equation of Turnbull (1976), which results in a nonparametric estimator. We demonstrate by simulation that the estimator, bootstrap variance estimation and bandwidth selection all perform well in finite samples.     

Efficient Estimation of the Partly Linear Additive Hazards Model with Current Status Data

This paper focuses on efficient estimation, optimal rates of convergence and effective algorithms in the partly linear additive hazards regression model with current status data. We use polynomial splines to estimate both cumulative baseline hazard function with monotonicity constraint and nonparametric regression functions with no such constraint. We propose a simultaneous sieve maximum likelihood estimation for regression parameters and nuisance parameters and show that the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound. In addition, we show that rates of convergence for the estimators of nonparametric functions are optimal. We implement the proposed estimation through a backfitting algorithm on generalized linear models. We conduct simulation studies to examine the finite‐sample performance of the proposed estimation method and present an analysis of renal function recovery data for illustration.