Nonparametric estimation of time-to-event distribution based on recall data in observational studies

In a cross-sectional observational study, time-to-event distribution can be estimated from data on current status or from recalled data on the time of occurrence. In either case, one can treat the data as having been interval censored, and use the nonparametric maximum likelihood estimator proposed by Turnbull (J R Stat Soc Ser B 38:290–295, 1976). However, the chance of recall may depend on the time span between the occurrence of the event and the time of interview. In such a case, the underlying censoring would be informative, rendering the Turnbull estimator inappropriate. In this article, we provide a nonparametric maximum likelihood estimator of the distribution of interest, by using a model adapted to the special nature of the data at hand. We also provide a computationally simple approximation of this estimator, and establish the consistency of both the original and the approximate versions, under mild conditions. Monte Carlo simulations indicate that the proposed estimators have smaller bias than the Turnbull estimator based on incomplete recall data, smaller variance than the Turnbull estimator based on current status data, and smaller mean squared error than both of them. The method is applied to menarcheal data from a recent Anthropometric study of adolescent and young adult females in Kolkata, India.     

Pivotal inference for the inverse Rayleigh distribution based on general progressively Type-II censored samples

In this paper, we consider the problem of estimating the scale parameter of the inverse Rayleigh distribution based on general progressively Type-II censored samples and progressively Type-II censored samples. The pivotal quantity method is used to derive the estimator of the scale parameter. Besides, considering that the maximum likelihood estimator is tough to obtain for this distribution, we derive an explicit estimator of the scale parameter by approximating the likelihood equation with Taylor expansion. The interval estimation is also studied based on pivotal inference. Then we conduct Monte Carlo simulations and compare the performance of different estimators. We demonstrate that the pivotal inference is simpler and more effective. The further application of the pivotal quantity method is also discussed theoretically. Finally, two real data sets are analyzed using our methods.     

Exact inference for the two-parameter exponential distribution under Type-II hybrid censoring

Epstein (1954) introduced the Type-I hybrid censoring scheme as a mixture of Type-I and Type-II censoring schemes. Childs et al. (2003) introduced the Type-II hybrid censoring scheme as an alternative to Type-I hybrid censoring scheme, and provided the exact distribution of the maximum likelihood estimator of the mean of a one-parameter exponential distribution based on Type-II hybrid censored samples. The associated confidence interval also has been provided. The main aim of this paper is to consider a two-parameter exponential distribution, and to derive the exact distribution of the maximum likelihood estimators of the unknown parameters based on Type-II hybrid censored samples. The marginal distributions and the exact confidence intervals are also provided. The results can be used to derive the exact distribution of the maximum likelihood estimator of the percentile point, and to construct the associated confidence interval. Different methods are compared using extensive simulations and one data analysis has been performed for illustrative purposes.     

Aalen's linear model for doubly censored data

Double censoring often occurs in registry studies when left censoring is present in addition to right censoring. In this work, we examine estimation of Aalen's nonparametric regression coefficients based on doubly censored data. We propose two estimation techniques. The first type of estimators, including ordinary least squared (OLS) estimator and weighted least squared (WLS) estimators, are obtained using martingale arguments. The second type of estimator, the maximum likelihood estimator (MLE), is obtained via expectation-maximization (EM) algorithms that treat the survival times of left censored observations as missing. Asymptotic properties, including the uniform consistency and weak convergence, are established for the MLE. Simulation results demonstrate that the MLE is more efficient than the OLS and WLS estimators.     

Copula‐based semiparametric analysis for time series data with detection limits

The analysis of time series data with detection limits is challenging due to the high‐dimensional integral involved in the likelihood. Existing methods are either computationally demanding or rely on restrictive parametric distributional assumptions. We propose a semiparametric approach, where the temporal dependence is captured by parametric copula, while the marginal distribution is estimated non‐parametrically. Utilizing the properties of copulas, we develop a new copula‐based sequential sampling algorithm, which provides a convenient way to calculate the censored likelihood. Even without full parametric distributional assumptions, the proposed method still allows us to efficiently compute the conditional quantiles of the censored response at a future time point, and thus construct both point and interval predictions. We establish the asymptotic properties of the proposed pseudo maximum likelihood estimator, and demonstrate through simulation and the analysis of a water quality data that the proposed method is more flexible and leads to more accurate predictions than Gaussian‐based methods for non‐normal data. The Canadian Journal of Statistics 47: 438–454; 2019 © 2019 Statistical Society of Canada     

Linear approximate ML estimation in scaled Type I generalized logistic distribution based on Type-II censored samples

The scaled (two-parameter) Type I generalized logistic distribution (GLD) is considered with the known shape parameter. The ML method does not yield an explicit estimator for the scale parameter even in complete samples. In this article, we therefore construct a new linear estimator for scale parameter, based on complete and doubly Type-II censored samples, by making linear approximations to the intractable terms of the likelihood equation using least-squares (LS) method, a new approach of linearization. We call this as linear approximate maximum likelihood estimator (LAMLE). We also construct LAMLE based on Taylor series method of linear approximation and found that this estimator is slightly biased than that based on the LS method. A Monte Carlo simulation is used to investigate the performance of LAMLE and found that it is almost as efficient as MLE, though biased than MLE. We also compare unbiased LAMLE with BLUE based on the exact variances of the estimators and interestingly this new unbiased LAMLE is found just as efficient as the BLUE in both complete and Type-II censored samples. Since MLE is known as asymptotically unbiased, in large samples we compare unbiased LAMLE with MLE and found that this estimator is almost as efficient as MLE. We have also discussed interval estimation of the scale parameter from complete and Type-II censored samples. Finally, we present some numerical examples to illustrate the construction of the new estimators developed here.     

Classical and Bayesian estimation of stress-strength reliability in type II censored Pareto distributions

ABSTRACT

In this paper, the stress-strength reliability, R, is estimated in type II censored samples from Pareto distributions. The classical inference includes obtaining the maximum likelihood estimator, an exact confidence interval, and the confidence intervals based on Wald and signed log-likelihood ratio statistics. Bayesian inference includes obtaining Bayes estimator, equi-tailed credible interval, and highest posterior density (HPD) interval given both informative and non-informative prior distributions. Bayes estimator of R is obtained using four methods: Lindley's approximation, Tierney-Kadane method, Monte Carlo integration, and MCMC. Also, we compare the proposed methods by simulation study and provide a real example to illustrate them.     

An inverse-probability-weighted approach to the estimation of distribution function with middle-censored data

In this article, we propose an inverse-probability-weighted (IPW) estimator of distribution function for middle-censored data. By Jammalamadaka and Mangalam (2003), the IPW estimator is the nonparametric maximum likelihood estimator (NPMLE) when all censored intervals contain at least one uncensored observation. The asymptotic properties of the IPW estimator are derived. A simulation study is conducted to compare the performance between the IPW estimator and the self-consistent estimator (SCE). Simulation results indicate that the performance of the IPW estimator is close to that of the SCE.     

Estimation of the scale parameter of the Rayleigh distribution with multiply type–II censored sample

Based on a multiply type-II censored sample, the maximum likelihood estimator (MLE) and Bayes estimator for the scale parameter and the reliability function of the Rayleigh distribution are derived. However, since the MLE does not exist an explicit form, an approximate MLE which is the maximizer of an approximate likelihood function will be given. The comparisons among estimators are investigated through Monte Carlo simulations. An illustrative example with the real data concerning the 23 ball bearing in the life test is presented.     

Exact Bayesian variable sampling plans for the exponential distribution with progressive hybrid censoring

From the exact distribution of the maximum likelihood estimator of the average lifetime based on progressive hybrid exponential censored sample, we derive an explicit expression for the Bayes risk of a sampling plan when a quadratic loss function is used. The simulated annealing algorithm is then used to determine the optimal sampling plan. Some optimal Bayes solutions under progressive hybrid and ordinary hybrid censoring schemes are presented to illustrate the effectiveness of the proposed method.     

The nonparametric maximum likelihood estimator for middle-censored data

In this note, we consider data subjected to middle censoring where the variable of interest becomes unobservable when it falls within an interval of censorship. We demonstrate that the nonparametric maximum likelihood estimator (NPMLE) of distribution function can be obtained by using Turnbull's (1976) EM algorithm or self-consistent estimating equation (Jammalamadaka and Mangalam, 2003) with an initial estimator which puts mass only on the innermost intervals. The consistency of the NPMLE can be established based on the asymptotic properties of self-consistent estimators (SCE) with mixed interval-censored data ( [Yu et al., 2000] and [Yu et al., 2001]).     

Parameter estimation based upon nonparametric function estimators

By considering the solution to a linear approximation of a nonlinear regression problem, a procedure for developing a para¬meter estimator, based upon a nonpammetric estimator of a para¬metric function, is given. The resulting estimators, which are determinable in closed form, are asymptotically normally distri¬buted and are optimal among the class of estimators based upon the function estimator. Further, in many cases, the estimator will have the same asymptotic distribution theory as the correspond¬ing maximum likelihood estimator. Estimators based upon the Kaplan-Meier quantile function are developed for randomly censored samples.     

Inference for the two-parameter half-logistic distribution using pivotal quantities under progressively Type-II censoring schemes

This article addresses estimation and prediction problems for the two-parameter half-logistic distribution based on pivotal quantities when a sample is available from the progressively Type-II censoring scheme. An unbiased estimator of the location parameter based on a pivotal quantity is derived. To estimate the scale parameter, a new method based on a pivotal quantity is proposed. The proposed method provides a simpler estimation equation than the maximum likelihood equation. In addition, confidence intervals for the location and scale parameters are derived from these pivotal quantities. In the prediction of censored failure times, the shortest-length predictive intervals for the censored failure times are derived using a pivotal quantity. Finally, the validity of the proposed method is assessed through Monte Carlo simulations and a real data set is presented for illustration purposes.     

Non-parametric Hypothesis Testing and Confidence Intervals with Doubly Censored Data

The non-parametric maximum likelihood estimator (NPMLE) of the distribution function with doubly censored data can be computed using the self-consistent algorithm (Turnbull, 1974). We extend the self-consistent algorithm to include a constraint on the NPMLE. We then show how to construct confidence intervals and test hypotheses based on the NPMLE via the empirical likelihood ratio. Finally, we present some numerical comparisons of the performance of the above method with another method that makes use of the influence functions.     

Likelihood inference for the component lifetime distribution based on progressively censored parallel systems data

Point and interval estimators for the scale parameter of the component lifetime distribution of a k-component parallel system are obtained when the component lifetimes are assumed to be independently and identically exponentially distributed. We prove that the maximum likelihood estimator of the scale parameter based on progressively Type-II censored system lifetimes is unique and can be obtained by a fixed-point iteration procedure. In particular, we illustrate that the Newton–Raphson method does not converge for any initial value. Furthermore, exact confidence intervals are constructed by a transformation using normalized spacings and other component lifetime distributions including Weibull distribution are discussed.     

On hybrid censored Weibull distribution

A hybrid censoring is a mixture of Type-I and Type-II censoring schemes. This article presents the statistical inferences on Weibull parameters when the data are hybrid censored. The maximum likelihood estimators (MLEs) and the approximate maximum likelihood estimators are developed for estimating the unknown parameters. Asymptotic distributions of the MLEs are used to construct approximate confidence intervals. Bayes estimates and the corresponding highest posterior density credible intervals of the unknown parameters are obtained under suitable priors on the unknown parameters and using the Gibbs sampling procedure. The method of obtaining the optimum censoring scheme based on the maximum information measure is also developed. Monte Carlo simulations are performed to compare the performances of the different methods and one data set is analyzed for illustrative purposes.     

A Study of Interval Censoring in Parametric Regression Models

Parametric models for interval censored data can now easily be fitted with minimal programming in certain standard statistical software packages. Regression equations can be introduced, both for the location and for the dispersion parameters. Finite mixture models can also be fitted, with a point mass on right (or left) censored observations, to allow for individuals who cannot have the event (or already have it). This mixing probability can also be allowed to follow a regression equation.Here, models based on nine different distributions are compared for three examples of heavily censored data as well as a set of simulated data. We find that, for parametric models, interval censoring can often be ignored and that the density, at centres of intervals, can be used instead in the likelihood function, although the approximation is not always reliable. In the context of heavily interval censored data, the conclusions from parametric models are remarkably robust with changing distributional assumptions and generally more informative than the corresponding non-parametric models.     

Bayesian inference based on a jointly type-II censored sample from two exponential populations

In this paper, based on a jointly type-II censored sample from two exponential populations, the Bayesian inference for the two unknown parameters are developed with the use of squared-error, linear-exponential and general entropy loss functions. The problem of predicting the future failure times, both point and interval prediction, based on the observed joint type-II censored data, is also addressed from a Bayesian viewpoint. A Monte Carlo simulation study is conducted to compare the Bayesian estimators with the maximum likelihood estimator developed by Balakrishnan and Rasouli [Exact likelihood inference for two exponential populations under joint type-II censoring. Comput Stat Data Anal. 2008;52:2725–2738]. Finally, a numerical example is utilized for the purpose of illustration.     

Identity for the NPMLE in Censored Data Models

We derive an identity for nonparametric maximum likelihood estimators (NPMLE) and regularized MLEs in censored data models which expresses the standardized maximum likelihood estimator in terms of the standardized empirical process. This identity provides an effective starting point in proving both consistency and efficiency of NPMLE and regularized MLE. The identity and corresponding method for proving efficiency is illustrated for the NPMLE in the univariate right-censored data model, the regularized MLE in the current status data model and for an implicit NPMLE based on a mixture of right-censored and current status data. Furthermore, a general algorithm for estimation of the limiting variance of the NPMLE is provided. This revised version was published online in July 2006 with corrections to the Cover Date.     

Estimation with Interval Censored Data and Covariates

In biostatistical applications interest often focuses on the estimation of the distribution of time T between two consecutive events. If the initial event time is observed and the subsequent event time is only known to be larger or smaller than an observed monitoring time C, then the data conforms to the well understood singly-censored current status model, also known as interval censored data, case I. Additional covariates can be used to allow for dependent censoring and to improve estimation of the marginal distribution of T. Assuming a wrong model for the conditional distribution of T, given the covariates, will lead to an inconsistent estimator of the marginal distribution. On the other hand, the nonparametric maximum likelihood estimator of FT requires splitting up the sample in several subsamples corresponding with a particular value of the covariates, computing the NPMLE for every subsample and then taking an average. With a few continuous covariates the performance of the resulting estimator is typically miserable. In van der Laan, Robins (1996) a locally efficient one-step estimator is proposed for smooth functionals of the distribution of T, assuming nothing about the conditional distribution of T, given the covariates, but assuming a model for censoring, given the covariates. The estimators are asymptotically linear if the censoring mechanism is estimated correctly. The estimator also uses an estimator of the conditional distribution of T, given the covariates. If this estimate is consistent, then the estimator is efficient and if it is inconsistent, then the estimator is still consistent and asymptotically normal. In this paper we show that the estimators can also be used to estimate the distribution function in a locally optimal way. Moreover, we show that the proposed estimator can be used to estimate the distribution based on interval censored data (T is now known to lie between two observed points) in the presence of covariates. The resulting estimator also has a known influence curve so that asymptotic confidence intervals are directly available. In particular, one can apply our proposal to the interval censored data without covariates. In Geskus (1992) the information bound for interval censored data with two uniformly distributed monitoring times at the uniform distribution (for T has been computed. We show that the relative efficiency of our proposal w.r.t. this optimal bound equals 0.994, which is also reflected in finite sample simulations. Finally, the good practical performance of the estimator is shown in a simulation study. This revised version was published online in July 2006 with corrections to the Cover Date.