Nonparametric Bayes estimation of a distribution function with truncated data

A truncation bias affects the observation of a pair of variables (X,Y), so that data are available only if Y ⩽ X. In such a situation, the nonparametric maximum likelihood estimator (NPMLE) of the distribution function of Y may have unpleasant features (Woodroofe, Ann. Statist. 13 (1985) 163–177). As a possible alternative, a nonparametric Bayes estimator is obtained using a Dirichlet prior (Ferguson, Ann. Statist. 1 (1973) 209–230). Its frequentist asymptotic behavior is investigated and found to be the same as the asymptotic behavior of the NPMLE. The results are illustrated by an example, with astronomical data, where the NPMLE is clearly unacceptable.     

Estimation Of A Survival Function With Doubly Censored Data And Dirichlet Process Prior Knowledge On The Observable Variable

This article presents a nonparametric Bayesian procedure for estimating a survival curve in a proportional hazard model when some of the data are censored on the left and some are censored on the right. The method works under the assumption that there is a Dirichlet process prior knowledge on the observable variable. Strong consistency of the estimator is proved and an example is given. To finish some simulation is presented to analyze the estimator.     

Dimension reduced kernel estimation for distribution function with incomplete data

This work focuses on the estimation of distribution functions with incomplete data, where the variable of interest Y has ignorable missingness but the covariate X is always observed. When X is high dimensional, parametric approaches to incorporate X—information is encumbered by the risk of model misspecification and nonparametric approaches by the curse of dimensionality. We propose a semiparametric approach, which is developed under a nonparametric kernel regression framework, but with a parametric working index to condense the high dimensional X—information for reduced dimension. This kernel dimension reduction estimator has double robustness to model misspecification and is most efficient if the working index adequately conveys the X—information about the distribution of Y. Numerical studies indicate better performance of the semiparametric estimator over its parametric and nonparametric counterparts. We apply the kernel dimension reduction estimation to an HIV study for the effect of antiretroviral therapy on HIV virologic suppression.     

A Nonparametric Estimator of Survival Functions for Arbitrarily Truncated and Censored Data

It is well-known that the nonparametric maximum likelihood estimator (NPMLE) may severely under-estimate the survival function with left truncated data. Based on the Nelson estimator (for right censored data) and self-consistency we suggest a nonparametric estimator of the survival function, the iterative Nelson estimator (INE), for arbitrarily truncated and censored data, where only few nonparametric estimators are available. By simulation we show that the INE does well in overcoming the under-estimation of the survival function from the NPMLE for left-truncated and interval-censored data. An interesting application of the INE is as a diagnostic tool for other estimators, such as the monotone MLE or parametric MLEs. The methodology is illustrated by application to two real world problems: the Channing House and the Massachusetts Health Care Panel Study data sets.     

Nonparametric Bayes estimation of gap-time distribution with recurrent event data

Nonparametric Bayes (NPB) estimation of the gap-time survivor function governing the time to occurrence of a recurrent event in the presence of censoring is considered. In our Bayesian approach, the gap-time distribution, denoted by F, has a Dirichlet process prior with parameter α. We derive NPB and nonparametric empirical Bayes (NPEB) estimators of the survivor function F?=1?F and construct point-wise credible intervals. The resulting Bayes estimator of F? extends that based on single-event right-censored data, and the PL-type estimator is a limiting case of this Bayes estimator. Through simulation studies, we demonstrate that the PL-type estimator has smaller biases but higher root-mean-squared errors (RMSEs) than those of the NPB and the NPEB estimators. Even in the case of a mis-specified prior measure parameter α, the NPB and the NPEB estimators have smaller RMSEs than the PL-type estimator, indicating robustness of the NPB and NPEB estimators. In addition, the NPB and NPEB estimators are smoother (in some sense) than the PL-type estimator.     

Nonparametric hierarchical Bayes analysis of binomial data via Bernstein polynomial priors

For binomial data analysis, many methods based on empirical Bayes interpretations have been developed, in which a variance‐stabilizing transformation and a normality assumption are usually required. To achieve the greatest model flexibility, we conduct nonparametric Bayesian inference for binomial data and employ a special nonparametric Bayesian prior—the Bernstein–Dirichlet process (BDP)—in the hierarchical Bayes model for the data. The BDP is a special Dirichlet process (DP) mixture based on beta distributions, and the posterior distribution resulting from it has a smooth density defined on [0, 1]. We examine two Markov chain Monte Carlo procedures for simulating from the resulting posterior distribution, and compare their convergence rates and computational efficiency. In contrast to existing results for posterior consistency based on direct observations, the posterior consistency of the BDP, given indirect binomial data, is established. We study shrinkage effects and the robustness of the BDP‐based posterior estimators in comparison with several other empirical and hierarchical Bayes estimators, and we illustrate through examples that the BDP‐based nonparametric Bayesian estimate is more robust to the sample variation and tends to have a smaller estimation error than those based on the DP prior. In certain settings, the new estimator can also beat Stein's estimator, Efron and Morris's limited‐translation estimator, and many other existing empirical Bayes estimators. The Canadian Journal of Statistics 40: 328–344; 2012 © 2012 Statistical Society of Canada     

Asymptotic normality of a conditional hazard function estimate in the single index for quasi-associated data

Abstract

The main goal of this paper is to study the estimation of the conditional hazard function of a scalar response variable Y given a hilbertian random variable X in functional single-index model. We construct an estimator of this nonparametric function and we study its asymptotic properties, under quasi-associated structure. Precisely, we establish the asymptotic normality of the constructed estimator. We carried out simulation experiments to examine the behavior of this asymptotic property over finite sample data.     

Nonparametric bayes estimation of the survival function from a record of failures and follow-ups

In some observational studies, we have random censoring model. However, the data available may be partially observable censored data consisting of the observed failure times and only those nonfailure times which are subject to follow-up. Suzuki (1985) discussed the problem of nonparametric estimation of the survival function from such partially observable censored data. In this article, we derive a nonparametric Bayes estimator of the survival function for such data of failures and follow-ups under a Dirichlet process prior and squared error loss. The limiting properties such as the mean square consistency, weak convergence and strong consistency of the Bayes estimator are studied. Finally, the procedures developed are illustrated by means of an example.     

Nonparametric linear bayes estimation of survival curves from incomplete observations

A linear Bayes estimator of a survival curve is derived.The estimator has a relatively simple interpretation as a Kaplan-Meier estimator based on an augemented data base - prior information plus sampling information.It is Bayes if the prior is a Dirichlet process, and otherwise an approximation to the Bayes rule against any prior.     

Nonparametric location-scale models for censored successive survival times

Let (T₁,T₂) be gap times corresponding to two consecutive events, which are observed subject to (univariate) random right-censoring. The censoring variable corresponding to the second gap time T₂ will in general depend on this gap time. Suppose the vector (T₁,T₂) satisfies the nonparametric location-scale regression model T₂=m(T₁)+σ(T₁)?, where the functions m and σ are ‘smooth’, and ? is independent of T₁. The aim of this paper is twofold. First, we propose a nonparametric estimator of the distribution of the error variable under this model. This problem differs from others considered in the recent related literature in that the censoring acts not only on the response but also on the covariate, having no obvious solution. On the basis of the idea of transfer of tail information (Van Keilegom and Akritas, 1999), we then use the proposed estimator of the error distribution to introduce nonparametric estimators for important targets such as: (a) the conditional distribution of T₂ given T₁; (b) the bivariate distribution of the gap times; and (c) the so-called transition probabilities. The asymptotic properties of these estimators are obtained. We also illustrate through simulations, that the new estimators based on the location-scale model may behave much better than existing ones.     

Quantile inference for heteroscedastic regression models

Consider the nonparametric heteroscedastic regression model Y=m(X)+σ(X)?, where m(·) is an unknown conditional mean function and σ(·) is an unknown conditional scale function. In this paper, the limit distribution of the quantile estimate for the scale function σ(X) is derived. Since the limit distribution depends on the unknown density of the errors, an empirical likelihood ratio statistic based on quantile estimator is proposed. This statistics is used to construct confidence intervals for the variance function. Under certain regularity conditions, it is shown that the quantile estimate of the scale function converges to a Brownian motion and the empirical likelihood ratio statistic converges to a chi-squared random variable. Simulation results demonstrate the superiority of the proposed method over the least squares procedure when the underlying errors have heavy tails.     

An inverse-probability-weighted approach to estimation of the bivariate survival function under left-truncation and right-censoring

In this note, we consider estimating the bivariate survival function when both survival times are subject to random left truncation and one of the survival times is subject to random right censoring. Motivated by Satten and Datta [2001. The Kaplan–Meier estimator as an inverse-probability-of-censoring weighted average. Amer. Statist. 55, 207–210], we propose an inverse-probability-weighted (IPW) estimator. It involves simultaneous estimation of the bivariate survival function of the truncation variables and that of the censoring variable and the truncation variable of the uncensored components. We prove that (i) when there is no censoring, the IPW estimator reduces to NPMLE of van der Laan [1996a. Nonparametric estimation of the bivariate survival function with truncated data. J. Multivariate Anal. 58, 107–131] and (ii) when there is random left truncation and right censoring on only one of the components and the other component is always observed, the IPW estimator reduces to the estimator of Gijbels and Gürler [1998. Covariance function of a bivariate distribution function estimator for left truncated and right censored data. Statist. Sin. 1219–1232]. Based on Theorem 3.1 of van der Laan [1996a. Nonparametric estimation of the bivariate survival function with truncated data. J. Multivariate Anal. 58, 107–131, 1996b. Efficient estimation of the bivariate censoring model and repairing NPMLE. Ann. Statist. 24, 596–627], we prove that the IPW estimator is consistent under certain conditions. Finally, we examine the finite sample performance of the IPW estimator in some simulation studies. For the special case that censoring time is independent of truncation time, a simulation study is conducted to compare the performances of the IPW estimator against that of the estimator proposed by van der Laan [1996a. Nonparametric estimation of the bivariate survival function with truncated data. J. Multivariate Anal. 58, 107–131, 1996b. Efficient estimation of the bivariate censoring model and repairing NPMLE. Ann. Statist. 24, 596–627]. For the special case (i), a simulation study is conducted to compare the performances of the IPW estimator against that of the estimator proposed by Huang et al. (2001. Nonnparametric estimation of marginal distributions under bivariate truncation with application to testing for age-of-onset application. Statist. Sin. 11, 1047–1068).     

Simulations and computations of nonparametric density estimates for the deconvolution problem

The nonparametric density function estimation using sample observations which are contaminated with random noise is studied. The particular form of contamination under consideration is Y = X + Z, where Y is an observable random variableZ is a random noise variable with known distribution, and X is an absolutely continuous random variable which cannot be observed directly. The finite sample size performance of a strongly consistent estimator for the density function of the random variable X is illustrated for different distributions. The estimator uses Fourier and kernel function estimation techniques and allows the user to choose constants which relate to bandwidth windows and limits on integration and which greatly affect the appearance and properties of the estimates. Numerical techniques for computation of the estimated densities and for optimal selection of the constant are given.     

Empirical Bayes estimation of P(Y < X) and characterizations of Burr-type X model

This paper deals with the estimation of R = P(Y < X) when Y and X are two independent but not identically distributed Burr-type X random variables. Maximum likelihood, Bayes and empirical Bayes techniques are used for this purpose. Monte-Carlo simulation is carried out to compare the three methods of estimation. Also, two characterizations of the Burr-type X distribution are presented. The first characterization is based on the recurrence relationships between two successively conditional moments of a certain function of the random variable, whereas the second one is given by the conditional variance of that function.     

A study on the least squares estimator of multivariate isotonic regression function

Consider the problem of pointwise estimation of f in a multivariate isotonic regression model Z=f(X₁,…,X_d)+ϵ, where Z is the response variable, f is an unknown nonparametric regression function, which is isotonic with respect to each component, and ϵ is the error term. In this article, we investigate the behavior of the least squares estimator of f. We generalize the greatest convex minorant characterization of isotonic regression estimator for the multivariate case and use it to establish the asymptotic distribution of properly normalized version of the estimator. Moreover, we test whether the multivariate isotonic regression function at a fixed point is larger (or smaller) than a specified value or not based on this estimator, and the consistency of the test is established. The practicability of the estimator and the test are shown on simulated and real data as well.     

Nonparametric estimators of survival function under the mixed case interval-censored model with left truncation

It is well known that the nonparametric maximum likelihood estimator (NPMLE) can severely underestimate the survival probabilities at early times for left-truncated and interval-censored (LT-IC) data. For arbitrarily truncated and censored data, Pan and Chappel (JAMA Stat Probab Lett 38:49–57, 1998a, Biometrics 54:1053–1060, 1998b) proposed a nonparametric estimator of the survival function, called the iterative Nelson estimator (INE). Their simulation study showed that the INE performed well in overcoming the under-estimation of the survival function from the NPMLE for LT-IC data. In this article, we revisit the problem of inconsistency of the NPMLE. We point out that the inconsistency is caused by the likelihood function of the left-censored observations, where the left-truncated variables are used as the left endpoints of censoring intervals. This can lead to severe underestimation of the survival function if the NPMLE is obtained using Turnbull’s (JAMA 38:290–295, 1976) EM algorithm. To overcome this problem, we propose a modified maximum likelihood estimator (MMLE) based on a modified likelihood function, where the left endpoints of censoring intervals for left-censored observations are the maximum of left-truncated variables and the estimated left endpoint of the support of the left-censored times. Simulation studies show that the MMLE performs well for finite sample and outperforms both the INE and NPMLE.

     

Estimating survival curves with left truncated and interval censored data via the ems algorithm

It is well-known that the nonparametric maximum likelihood estimator (NPMLE) of a survival function may severely underestimate the survival probabilities at very early times for left truncated data. This problem might be overcome by instead computing a smoothed nonparametric estimator (SNE) via the EMS algorithm. The close connection between the SNE and the maximum penalized likelihood estimator is also established. Extensive Monte Carlo simulations demonstrate the superior performance of the SNE over that of the NPMLE, in terms of either bias or variance, even for moderately large Samples. The methodology is illustrated with an application to the Massachusetts Health Care Panel Study dataset to estimate the probability of being functionally independent for non-poor male and female groups rcspectively.     

Computationally simple estimation and improved efficiency for special cases of double truncation

Doubly truncated survival data arise when event times are observed only if they occur within subject specific intervals of times. Existing iterative estimation procedures for doubly truncated data are computationally intensive (Turnbull 38:290–295, 1976; Efron and Petrosian 94:824–825, 1999; Shen 62:835–853, 2010a). These procedures assume that the event time is independent of the truncation times, in the sample space that conforms to their requisite ordering. This type of independence is referred to as quasi-independence. In this paper we identify and consider two special cases of quasi-independence: complete quasi-independence and complete truncation dependence. For the case of complete quasi-independence, we derive the nonparametric maximum likelihood estimator in closed-form. For the case of complete truncation dependence, we derive a closed-form nonparametric estimator that requires some external information, and a semi-parametric maximum likelihood estimator that achieves improved efficiency relative to the standard nonparametric maximum likelihood estimator, in the absence of external information. We demonstrate the consistency and potentially improved efficiency of the estimators in simulation studies, and illustrate their use in application to studies of AIDS incubation and Parkinson’s disease age of onset.     

Bayesian Approach in Nonparametric Count Regression with Binomial Kernel

Recently, Kokonendji et al. have adapted the well-known Nadaraya–Watson kernel estimator for estimating the count function m in the context of nonparametric discrete regression. The authors have also investigated the bandwidth selection using the cross-validation method. In this article, we propose a Bayesian approach in the context of nonparametric count regression for estimating the bandwidth and the variance of the model error, which has not been estimated in Kokonendji et al. The model error is considered as Gaussian with mean of zero and a variance of σ². The Bayes estimates cannot be obtained in closed form and then, we use the well-known Markov chain Monte Carlo (MCMC) technique to compute the Bayes estimates under the squared errors loss function. The performance of this proposed approach and the cross-validation method are compared through simulation and real count data.