Fitting linear regression models to censored data by least squares and maximum likelihood methods

Several approaches have been suggested for fitting linear regression models to censored data. These include Cox's propor­tional hazard models based on quasi-likelihoods. Methods of fitting based on least squares and maximum likelihoods have also been proposed. The methods proposed so far all require special purpose optimization routines. We describe an approach here which requires only a modified standard least squares routine.

We present methods for fitting a linear regression model to censored data by least squares and method of maximum likelihood. In the least squares method, the censored values are replaced by their expectations, and the residual sum of squares is minimized. Several variants are suggested in the ways in which the expect­ation is calculated. A parametric (assuming a normal error model) and two non-parametric approaches are described. We also present a method for solving the maximum likelihood equations in the estimation of the regression parameters in the censored regression situation. It is shown that the solutions can be obtained by a recursive algorithm which needs only a least squares routine for optimization. The suggested procesures gain considerably in computational officiency. The Stanford Heart Transplant data is used to illustrate the various methods.     

The likelihood ratio test for high-dimensional linear regression model

The paper considers a significance test of regression variables in the high-dimensional linear regression model when the dimension of the regression variables p, together with the sample size n, tends to infinity. Under two sightly different cases, we proved that the likelihood ratio test statistic will converge in distribution to a Gaussian random variable, and the explicit expressions of the asymptotical mean and covariance are also obtained. The simulations demonstrate that our high-dimensional likelihood ratio test method outperforms those using the traditional methods in analyzing high-dimensional data.     

Inference of δ = P(X < Y) for Burr XII distributions with record samples

Let X and Y follow independent Burr type XII distributions, which share a common inner shape parameter. The maximum likelihood estimator of the parameter δ = P(X < Y) is studied based on record samples. The existence and uniqueness of the maximum likelihood estimator of δ based on record samples are established. When the inner shape parameter is known, an exact confidence interval of δ is derived; otherwise, the Fisher information matrix and two bootstrap methods are used to obtain three approximate confidence intervals of δ. The performances of the proposed methods are evaluated via Monte Carlo simulation. Two examples are provided for illustration.     

Optimal estimating functions and wedderburn's quasi-likelihood

There are two inference methods which can be considered as developed from the classical least squares and maximum likelihood methods. One was put forward by Wedderburn (1974) and is called the quasi-likelihood method. Another was introduced by Godambe and others from the viewpoint of the estimating functions. This method is also called quasi-likelihood although there x are some differences between these two methods. In order to clarify the relationship, this paper provides a unified discussion of the two methods from the viewpoint of estimating functions.     

Higher order inference for stress–strength reliability with independent Burr-type X distributions

In this paper, a small-sample asymptotic method is proposed for higher order inference in the stress–strength reliability model, R=P(Y<X), where X and Y are distributed independently as Burr-type X distributions. In a departure from the current literature, we allow the scale parameters of the two distributions to differ, and the likelihood-based third-order inference procedure is applied to obtain inference for R. The difficulty of the implementation of the method is in obtaining the the constrained maximum likelihood estimates (MLE). A penalized likelihood method is proposed to handle the numerical complications of maximizing the constrained likelihood model. The proposed procedures are illustrated using a sample of carbon fibre strength data. Our results from simulation studies comparing the coverage probabilities of the proposed small-sample asymptotic method with some existing large-sample asymptotic methods show that the proposed method is very accurate even when the sample sizes are small.     

Target estimation for the logistic regression model

The method of target estimation developed by Cabrera and Fernholz [(1999). Target estimation for bias and mean square error reduction. The Annals of Statistics, 27(3), 1080–1104.] to reduce bias and variance is applied to logistic regression models of several parameters. The expectation functions of the maximum likelihood estimators for the coefficients in the logistic regression models of one and two parameters are analyzed and simulations are given to show a reduction in both bias and variability after targeting the maximum likelihood estimators. In addition to bias and variance reduction, it is found that targeting can also correct the skewness of the original statistic. An example based on real data is given to show the advantage of using target estimators for obtaining better confidence intervals of the corresponding parameters. The notion of the target median is also presented with some applications to the logistic models.     

Inference of R = P[X < Y] for skew normal distribution

This article studies the estimation of R = P[X < Y] when X and Y are two independent skew normal distribution with different parameters. When the scale parameter is unknown, the maximum likelihood estimator of R is proposed. The maximum likelihood estimator, uniformly minimum variance unbiased estimator, Bayes estimation, and confidence interval of R are obtained when the common scale parameter is known. In the general case, the maximum likelihood estimator of R is also discussed. To compare the different proposed methods, Monte Carlo simulations are performed. At last, the analysis of a real dataset has been presented for illustrative purposes too.     

On the performance of some new Liu parameters for the gamma regression model

The maximum likelihood (ML) method is used to estimate the unknown Gamma regression (GR) coefficients. In the presence of multicollinearity, the variance of the ML method becomes overstated and the inference based on the ML method may not be trustworthy. To combat multicollinearity, the Liu estimator has been used. In this estimator, estimation of the Liu parameter d is an important problem. A few estimation methods are available in the literature for estimating such a parameter. This study has considered some of these methods and also proposed some new methods for estimation of the d. The Monte Carlo simulation study has been conducted to assess the performance of the proposed methods where the mean squared error (MSE) is considered as a performance criterion. Based on the Monte Carlo simulation and application results, it is shown that the Liu estimator is always superior to the ML and recommendation about which best Liu parameter should be used in the Liu estimator for the GR model is given.     

Regularized proportional odds models

The proportional odds model (POM) is commonly used in regression analysis to predict the outcome for an ordinal response variable. The maximum likelihood estimation (MLE) approach is typically used to obtain the parameter estimates. The likelihood estimates do not exist when the number of parameters, p, is greater than the number of observations n. The MLE also does not exist if there are no overlapping observations in the data. In a situation where the number of parameters is less than the sample size but p is approaching to n, the likelihood estimates may not exist, and if they exist they may have quite large standard errors. An estimation method is proposed to address the last two issues, i.e. complete separation and the case when p approaches n, but not the case when p>n. The proposed method does not use any penalty term but uses pseudo-observations to regularize the observed responses by downgrading their effect so that they become close to the underlying probabilities. The estimates can be computed easily with all commonly used statistical packages supporting the fitting of POMs with weights. Estimates are compared with MLE in a simulation study and an application to the real data.     

On Ridge Parameters in Logistic Regression

This article applies and investigates a number of logistic ridge regression (RR) parameters that are estimable by using the maximum likelihood (ML) method. By conducting an extensive Monte Carlo study, the performances of ML and logistic RR are investigated in the presence of multicollinearity and under different conditions. The simulation study evaluates a number of methods of estimating the RR parameter k that has recently been developed for use in linear regression analysis. The results from the simulation study show that there is at least one RR estimator that has a lower mean squared error (MSE) than the ML method for all the different evaluated situations.     

Design considerations for small experiments and simple logistic regression

Inference for a generalized linear model is generally performed using asymptotic approximations for the bias and the covariance matrix of the parameter estimators. For small experiments, these approximations can be poor and result in estimators with considerable bias. We investigate the properties of designs for small experiments when the response is described by a simple logistic regression model and parameter estimators are to be obtained by the maximum penalized likelihood method of Firth [Firth, D., 1993, Bias reduction of maximum likelihood estimates. Biometrika, 80, 27–38]. Although this method achieves a reduction in bias, we illustrate that the remaining bias may be substantial for small experiments, and propose minimization of the integrated mean square error, based on Firth's estimates, as a suitable criterion for design selection. This approach is used to find locally optimal designs for two support points.     

Logistic regression with a partially observed covariate

We present results of a Monte Carlo study comparing four methods of estimating the parameters of the logistic model logit (pr (Y = 1 | X, Z)) = α₀ + α ₁ X + α ₂ Z where X and Z are continuous covariates and X is always observed but Z is sometimes missing. The four methods examined are 1) logistic regression using complete cases, 2) logistic regression with filled-in values of Z obtained from the regression of Z on X and Y, 3) logistic regression with filled-in values of Z and random error added, and 4) maximum likelihood estimation assuming the distribution of Z given X and Y is normal. Effects of different percent missing for Z and different missing value mechanisms on the bias and mean absolute deviation of the estimators are examined for data sets of N = 200 and N = 400.     

Pseudolikelihood estimation in a class of problems with response-related missing covariates

Many practical situations involve a response variable Y and covariates X , where data on (Y, X ) are incomplete for some portion of a sample of individuals. We consider two general types of pseudolikelihood estimation for problems in which missingness may be response-related. These are typically simpler to implement than ordinary maximum likelihood, which in this context is semiparametric. Asymptotics for the pseudolikelihood methods are presented, and simulations conducted to investigate the methods for an important class of problems involving lifetime data. Our results indicate that for these problems the two methods are effective and comparable with respect to efficiency.     

Nonparametric likelihood and doubly robust estimating equations for marginal and nested structural models

This article considers Robins's marginal and nested structural models in the cross‐sectional setting and develops likelihood and regression estimators. First, a nonparametric likelihood method is proposed by retaining a finite subset of all inherent and modelling constraints on the joint distributions of potential outcomes and covariates under a correctly specified propensity score model. A profile likelihood is derived by maximizing the nonparametric likelihood over these joint distributions subject to the retained constraints. The maximum likelihood estimator is intrinsically efficient based on the retained constraints and weakly locally efficient. Second, two regression estimators, named hat and tilde, are derived as first‐order approximations to the likelihood estimator under the propensity score model. The tilde regression estimator is intrinsically and weakly locally efficient and doubly robust. The methods are illustrated by data analysis for an observational study on right heart catheterization. The Canadian Journal of Statistics 38: 609–632; 2010 © 2010 Statistical Society of Canada     

A Spline‐Based Semiparametric Maximum Likelihood Estimation Method for the Cox Model with Interval‐Censored Data

Abstract. We propose a spline‐based semiparametric maximum likelihood approach to analysing the Cox model with interval‐censored data. With this approach, the baseline cumulative hazard function is approximated by a monotone B‐spline function. We extend the generalized Rosen algorithm to compute the maximum likelihood estimate. We show that the estimator of the regression parameter is asymptotically normal and semiparametrically efficient, although the estimator of the baseline cumulative hazard function converges at a rate slower than root‐n. We also develop an easy‐to‐implement method for consistently estimating the standard error of the estimated regression parameter, which facilitates the proposed inference procedure for the Cox model with interval‐censored data. The proposed method is evaluated by simulation studies regarding its finite sample performance and is illustrated using data from a breast cosmesis study.     

Estimation of P(Y < X) for Weibull Distribution Under Progressive Type-II Censoring

Based on progressively Type II censored samples, we consider the estimation of R = P(Y < X) when X and Y are two independent Weibull distributions with different shape parameters, but having the same scale parameter. The maximum likelihood estimator, approximate maximum likelihood estimator, and Bayes estimator of R are obtained. Based on the asymptotic distribution of R, the confidence interval of R are obtained. Two bootstrap confidence intervals are also proposed. Analysis of a real data set is given for illustrative purposes. Monte Carlo simulations are also performed to compare the different proposed methods.     

Stress-Strength Reliability of a Two-Parameter Bathtub-shaped Lifetime Distribution Based on Progressively Censored Samples

Based on progressively Type-II censored samples, this article deals with inference for the stress-strength reliability R = P(Y < X) when X and Y are two independent two-parameter bathtub-shape lifetime distributions with different scale parameters, but having the same shape parameter. Different methods for estimating the reliability are applied. The maximum likelihood estimate of R is derived. Also, its asymptotic distribution is used to construct an asymptotic confidence interval for R. Assuming that the shape parameter is known, the maximum likelihood estimator of R is obtained. Based on the exact distribution of the maximum likelihood estimator of R an exact confidence interval of that has been obtained. The uniformly minimum variance unbiased estimator are calculated for R. Bayes estimate of R and the associated credible interval are also got under the assumption of independent gamma priors. Monte Carlo simulations are performed to compare the performances of the proposed estimators. One data analysis has been performed for illustrative purpose. Finally, we will generalize this distribution to the proportional hazard family with two parameters and derive various estimators in this family.     

Comparison of estimation methods for the Weibull distribution

Weibull distributions have received wide ranging applications in many areas including reliability, hydrology and communication systems. Many estimation methods have been proposed for Weibull distributions. But there has not been a comprehensive comparison of these estimation methods. Most studies have focused on comparing the maximum likelihood estimation (MLE) with one of the other approaches. In this paper, we first propose an L-moment estimator for the Weibull distribution. Then, a comprehensive comparison is made of the following methods: the method of maximum likelihood estimation (MLE), the method of logarithmic moments, the percentile method, the method of moments and the method of L-moments.     

NONNORMAL REGRESSION. I. SKEW DISTRIBUTIONS

In a linear regression model of the type y= θ X+e, it is often assumed that the random error eis normally distributed. In numerous situations, e.g., when ymeasures life times or reaction times, etypically has a skew distribution. We consider two important families of skew distributions, (a) Weibull with support IR: (0, ∞) on the real line, and (b) generalised logistic with support IR: (?∞, ∞). Since the maximum likelihood estimators are intractable in these situations, we derive modified likelihood estimators which have explicit algebraic forms and are, therefore, easy to compute. We show that these estimators are remarkably efficient, and robust. We develop hypothesis testing procedures and give a real life example. Symmetric families of distributions, both long and short tailed, will be considered in a future paper.     

Inference of stress-strength for the Type-II generalized logistic distribution under progressively Type-II censored samples

This article studies the estimation of the reliability R = P[Y < X] when X and Y come from two independent generalized logistic distributions of Type-II with different parameters, based on progressively Type-II censored samples. When the common scale parameter is unknown, the maximum likelihood estimator and its asymptotic distribution are proposed. The asymptotic distribution is used to construct an asymptotic confidence interval of R. Bayes estimator of R and the corresponding credible interval using the Gibbs sampling technique have been proposed too. Assuming that the common scale parameter is known, the maximum likelihood estimator, uniformly minimum variance unbiased estimator, Bayes estimation, and confidence interval of R are extracted. Monte Carlo simulations are performed to compare the different proposed methods. Analysis of a real dataset is given for illustrative purposes. Finally, methods are extended for proportional hazard rate models.