Estimation for the generalized pareto distribution with censored data

We present a methodology for computing the point and interval maximum likelihood parameter estimation for the two-parameter generalized Pareto distribution (GPD) with censored data. The basic idea underlying our method is a reduction of the two-dimensional numerical search for the zeros of the GPD log-likelihood gradient vector to a one-dimensional numerical search. We describe a computationally efficient algorithm which implement this approach. Two illustrative examples are presented. Simulation results indicate that the estimates derived by maximum likelihood estimation are more reliable against those of method of moments. An evaluation of the practical sample size requirements for the asymptotic normality is also included.     

Modeling censored data using modified lambda family

In the present article we propose the modified lambda family (MLF) which is the Freimer, Mudholkar, Kollia, and Lin (FMKL) parametrization of generalized lambda distribution (GLD) as a model for censored data. The expressions for probability weighted moments of MLF are derived and used to estimate the parameters of the distribution. We modified the estimation technique using probability weighted moments. It is shown that the distribution provides reasonable fit to a real censored data.     

Singly and Doubly Censored Current Status Data: Estimation, Asymptotics and Regression

In biostatistical applications interest often focuses on the estimation of the distribution of time between two consecutive events. If the initial event time is observed and the subsequent event time is only known to be larger or smaller than an observed point in time, then the data is described by the well-understood singly censored current status model, also known as interval censored data, case I. Jewell et al. (1994) extended this current status model by allowing the initial time to be unobserved, with its distribution over an observed interval [A, B] known; the data is referred to as doubly censored current status data. This model has applications in AIDS partner studies. If the initial time is known to be uniformly distribute d, the model reduces to a submodel of the current status model with the same asymptotic information bounds as in the current status model, but the distribution of interest is essentially the derivative of the distribution of interest in the current status model. As a consequence the non-parametric maximum likelihood estimator is inconsistent. Moreover, this submodel contains only smooth heavy tailed distributions for which no moments exist. In this paper, we discuss the connection between the singly censored current status model and the doubly censored current status model (for the uniform initial time) in detail and explain the difficulties in estimation which arise in the doubly censored case. We propose a regularized MLE corresponding with the current status model. We prove rate results, efficiency of smooth functionals of the regularized MLE, and present a generally applicable efficient method for estimation of regression parameters, which does not rely on the existence of moments. We also discuss extending these ideas to a non-uniform distribution for the initial time.     

On the heterogeneity of fecundability

We propose to use a general mixing distribution in modeling the heterogeneity of the fecundability of couples. We introduce a sequence of parameters called canonical moments, which is in one to one correspondence with the moments, to characterize the mixing distribution. By using the bootstrap method, we can estimate the standard errors of our estimates. Our method modifies the usual moment estimates so that the resulting mixing distribution is always supported on [0, 1]. Moreover, the downward bias of the moment estimate of the number of support points would be reduced. Our approach can be used for censored data. The application of our technique in finding the sterile subpopulation is also discussed. The theory is illustrated with several data examples and simulations.     

Quantile Regression For Longitudinal Biomarker Data Subject to Left Censoring and Dropouts

Quantile regression is increasingly used in biomarker analysis to handle nonnormal or heteroscedastic data. However, in some biomedical studies, the biomarker data can be censored by detection limits of the bioassay or missing when the subjects drop out from the study. Inappropriate handling of these two issues leads to biased estimation results. We consider the censored quantile regression approach to account for the censoring data and apply the inverse weighting technique to adjust for dropouts. In particular, we develop a weighted estimating equation for censored quantile regression, where an individual’s contribution is weighted by the inverse probability of dropout at the given occasion. We conduct simulation studies to evaluate the properties of the proposed estimators and demonstrate our method with a real data set from Genetic and Inflammatory Marker of Sepsis (GenIMS) study.     

Joint modeling of censored longitudinal and event time data

Censoring of a longitudinal outcome often occurs when data are collected in a biomedical study and where the interest is in the survival and or longitudinal experiences of a study population. In the setting considered herein, we encountered upper and lower censored data as the result of restrictions imposed on measurements from a kinetic model producing “biologically implausible” kidney clearances. The goal of this paper is to outline the use of a joint model to determine the association between a censored longitudinal outcome and a time to event endpoint. This paper extends Guo and Carlin's [6] paper to accommodate censored longitudinal data, in a commercially available software platform, by linking a mixed effects Tobit model to a suitable parametric survival distribution. Our simulation results showed that our joint Tobit model outperforms a joint model made up of the more naïve or “fill-in” method for the longitudinal component. In this case, the upper and/or lower limits of censoring are replaced by the limit of detection. We illustrated the use of this approach with example data from the hemodialysis (HEMO) study [3] and examined the association between doubly censored kidney clearance values and survival.     

On the exact distribution of the MLEs based on Type-II progressively hybrid censored data from exponential distributions

ABSTRACT

Distributions of the maximum likelihood estimators (MLEs) in Type-II (progressive) hybrid censoring based on two-parameter exponential distributions have been obtained using a moment generating function approach. Although resulting in explicit expressions, the representations are complicated alternating sums. Using the spacings-based approach of Cramer and Balakrishnan [On some exact distributional results based on Type-I progressively hybrid censored data from exponential distributions. Statist Methodol. 2013;10:128–150], we derive simple expressions for the exact density and distribution functions of the MLEs in terms of B-spline functions. These representations can be easily implemented on a computer and provide an efficient method to compute density and distribution functions as well as moments of Type-II (progressively) hybrid censored order statistics.     

A NOTE ON MOMENTS OF PROGRESSIVELY TYPE II CENSORED ORDER STATISTICS

ABSTRACT

Upper and lower bounds for moments of progressively Type II censored order statistics in terms of moments of (progressively Type II censored) order statistics are derived. In particular, this yields conditions for the existence of moments of progressively Type II censored order statistics based on an absolutely continuous distribution function.     

Testing equality of survival distributions when the population marks are missing

This paper introduces a nonparametric approach for testing the equality of two or more survival distributions based on right censored failure times with missing population marks for the censored observations. The standard log-rank test is not applicable here because the population membership information is not available for the right censored individuals. We propose to use the imputed population marks for the censored observations leading to fractional at-risk sets that can be used in a two sample censored data log-rank test. We demonstrate with a simple example that there could be a gain in power by imputing population marks (the proposed method) for the right censored individuals compared to simply removing them (which also would maintain the right size). Performance of the imputed log-rank tests obtained this way is studied through simulation. We also obtain an asymptotic linear representation of our test statistic. Our testing methodology is illustrated using a real data set.     

Estimation and goodness-of-fit for the Cox model with various types of censored data

The currently existing estimation methods and goodness-of-fit tests for the Cox model mainly deal with right censored data, but they do not have direct extension to other complicated types of censored data, such as doubly censored data, interval censored data, partly interval-censored data, bivariate right censored data, etc. In this article, we apply the empirical likelihood approach to the Cox model with complete sample, derive the semiparametric maximum likelihood estimators (SPMLE) for the Cox regression parameter and the baseline distribution function, and establish the asymptotic consistency of the SPMLE. Via the functional plug-in method, these results are extended in a unified approach to doubly censored data, partly interval-censored data, and bivariate data under univariate or bivariate right censoring. For these types of censored data mentioned, the estimation procedures developed here naturally lead to Kolmogorov-Smirnov goodness-of-fit tests for the Cox model. Some simulation results are presented.     

General results for the beta Weibull distribution

In this paper, we study some mathematical properties of the beta Weibull (BW) distribution, which is a quite flexible model in analysing positive data. It contains the Weibull, exponentiated exponential, exponentiated Weibull and beta exponential distributions as special sub-models. We demonstrate that the BW density can be expressed as a mixture of Weibull densities. We provide their moments and two closed-form expressions for their moment-generating function. We examine the asymptotic distributions of the extreme values. Explicit expressions are derived for the mean deviations, Bonferroni and Lorenz curves, reliability and two entropies. The density of the BW-order statistics is a mixture of Weibull densities and two closed-form expressions are derived for their moments. The estimation of the parameters is approached by two methods: moments and maximum likelihood. We compare the performances of the estimates obtained from both the methods by simulation. The expected information matrix is derived. For the first time, we introduce a log-BW regression model to analyse censored data. The usefulness of the BW distribution is illustrated in the analysis of three real data sets.     

The McDonald Weibull model

For the first time, we propose a five-parameter lifetime model called the McDonald Weibull distribution to extend the Weibull, exponentiated Weibull, beta Weibull and Kumaraswamy Weibull distributions, among several other models. We obtain explicit expressions for the ordinary moments, quantile and generating functions, mean deviations and moments of the order statistics. We use the method of maximum likelihood to fit the new distribution and determine the observed information matrix. We define the log-McDonald Weibull regression model for censored data. The potentiality of the new model is illustrated by means of two real data sets.     

Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

Semi-parametric hybrid empirical likelihood inference for two-sample comparison with censored data

Two-sample comparison problems are often encountered in practical projects and have widely been studied in literature. Owing to practical demands, the research for this topic under special settings such as a semiparametric framework have also attracted great attentions. Zhou and Liang (Biometrika 92:271–282, 2005) proposed an empirical likelihood-based semi-parametric inference for the comparison of treatment effects in a two-sample problem with censored data. However, their approach is actually a pseudo-empirical likelihood and the method may not be fully efficient. In this study, we develop a new empirical likelihood-based inference under more general framework by using the hazard formulation of censored data for two sample semi-parametric hybrid models. We demonstrate that our empirical likelihood statistic converges to a standard chi-squared distribution under the null hypothesis. We further illustrate the use of the proposed test by testing the ROC curve with censored data, among others. Numerical performance of the proposed method is also examined.     

A Laguerre polynomial approximation for a goodness-of-fit test for exponential distribution based on progressively censored data

This paper proposes an approximation to the distribution of a goodness-of-fit statistic proposed recently by Balakrishnan et al. [Balakrishnan, N., Ng, H.K.T. and Kannan, N., 2002, A test of exponentiality based on spacings for progressively Type-II censored data. In: C. Huber-Carol et al. (Eds.), Goodness-of-Fit Tests and Model Validity (Boston: Birkhäuser), pp. 89–111.] for testing exponentiality based on progressively Type-II right censored data. The moments of this statistic can be easily calculated, but its distribution is not known in an explicit form. We first obtain the exact moments of the statistic using Basu's theorem and then the density approximants based on these exact moments of the statistic, expressed in terms of Laguerre polynomials, are proposed. A comparative study of the proposed approximation to the exact critical values, computed by Balakrishnan and Lin [Balakrishnan, N. and Lin, C.T., 2003, On the distribution of a test for exponentiality based on progressively Type-II right censored spacings. Journal of Statistical Computation and Simulation, 73 (4), 277–283.], is carried out. This reveals that the proposed approximation is very accurate.     

Likelihood Ratio Confidence Bands in Non–parametric Regression with Censored Data

Let ( X , Y ) be a random vector, where Y denotes the variable of interest possibly subject to random right censoring, and X is a covariate. We construct confidence intervals and bands for the conditional survival and quantile function of Y given X using a non-parametric likelihood ratio approach. This approach was introduced by Thomas & Grunkemeier (1975 ), who estimated confidence intervals of survival probabilities based on right censored data. The method is appealing for several reasons: it always produces intervals inside [0, 1], it does not involve variance estimation, and can produce asymmetric intervals. Asymptotic results for the confidence intervals and bands are obtained, as well as simulation results, in which the performance of the likelihood ratio intervals and bands is compared with that of the normal approximation method. We also propose a bandwidth selection procedure based on the bootstrap and apply the technique on a real data set.     

A Spline‐Based Semiparametric Maximum Likelihood Estimation Method for the Cox Model with Interval‐Censored Data

Abstract. We propose a spline‐based semiparametric maximum likelihood approach to analysing the Cox model with interval‐censored data. With this approach, the baseline cumulative hazard function is approximated by a monotone B‐spline function. We extend the generalized Rosen algorithm to compute the maximum likelihood estimate. We show that the estimator of the regression parameter is asymptotically normal and semiparametrically efficient, although the estimator of the baseline cumulative hazard function converges at a rate slower than root‐n. We also develop an easy‐to‐implement method for consistently estimating the standard error of the estimated regression parameter, which facilitates the proposed inference procedure for the Cox model with interval‐censored data. The proposed method is evaluated by simulation studies regarding its finite sample performance and is illustrated using data from a breast cosmesis study.     

Log-Weibull extended regression model: Estimation,sensitivity and residual analysis

The failure rate function commonly has a bathtub shape in practice. In this paper we discuss a regression model considering new Weibull extended distribution developed by Xie et al. (2002) that can be used to model this type of failure rate function. Assuming censored data, we discuss parameter estimation: maximum likelihood method and a Bayesian approach where Gibbs algorithms along with Metropolis steps are used to obtain the posterior summaries of interest. We derive the appropriate matrices for assessing the local influence on the parameter estimates under different perturbation schemes, and we also present some ways to perform global influence. Also, some discussions on case deletion influence diagnostics are developed for the joint posterior distribution based on the Kullback–Leibler divergence. Besides, for different parameter settings, sample sizes and censoring percentages, are performed various simulations and display and compare the empirical distribution of the Martingale-type residual with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be straightforwardly extended to the martingale-type residual in log-Weibull extended models with censored data. Finally, we analyze a real data set under a log-Weibull extended regression model. We perform diagnostic analysis and model check based on the martingale-type residual to select an appropriate model.     

Accurate computation of single and product moments of order statistics under progressive censoring

An accurate procedure is proposed to calculate approximate moments of progressive order statistics in the context of statistical inference for lifetime models. The study analyses the performance of power series expansion to approximate the moments for location and scale distributions with high precision and smaller deviations with respect to the exact values. A comparative analysis between exact and approximate methods is shown using some tables and figures. The different approximations are applied in two situations. First, we consider the problem of computing the large sample variance–covariance matrix of maximum likelihood estimators. We also use the approximations to obtain progressively censored sampling plans for log-normal distributed data. These problems illustrate that the presented procedure is highly useful to compute the moments with precision for numerous censoring patterns and, in many cases, is the only valid method because the exact calculation may not be applicable.