Empirical likelihood method for non-ignorable missing data problems

Missing response problem is ubiquitous in survey sampling, medical, social science and epidemiology studies. It is well known that non-ignorable missing is the most difficult missing data problem where the missing of a response depends on its own value. In statistical literature, unlike the ignorable missing data problem, not many papers on non-ignorable missing data are available except for the full parametric model based approach. In this paper we study a semiparametric model for non-ignorable missing data in which the missing probability is known up to some parameters, but the underlying distributions are not specified. By employing Owen (1988)’s empirical likelihood method we can obtain the constrained maximum empirical likelihood estimators of the parameters in the missing probability and the mean response which are shown to be asymptotically normal. Moreover the likelihood ratio statistic can be used to test whether the missing of the responses is non-ignorable or completely at random. The theoretical results are confirmed by a simulation study. As an illustration, the analysis of a real AIDS trial data shows that the missing of CD4 counts around two years are non-ignorable and the sample mean based on observed data only is biased.     

A flexible marginal modelling strategy for non-monotone missing data

Summary.  Much research has been devoted to modelling strategies for longitudinal data with missingness, recently especially within the missingness not at random context. In this paper, the relatively unexplored but practically highly relevant domain of non-monotone missingness with multivariate ordinal responses is broached. For this, a dedicated version of the multivariate Dale model is formulated. Furthermore, we also assess the sensitivity of these models to their assumptions, by using the technique of global influence.     

Analysis of longitudinal data with non-ignorable non-monotone missing values

A full likelihood method is proposed to analyse continuous longitudinal data with non-ignorable (informative) missing values and non-monotone patterns. The problem arose in a breast cancer clinical trial where repeated assessments of quality of life were collected: patients rated their coping ability during and after treatment. We allow the missingness probabilities to depend on unobserved responses, and we use a multivariate normal model for the outcomes. A first-order Markov dependence structure for the responses is a natural choice and facilitates the construction of the likelihood; estimates are obtained via the Nelder–Mead simplex algorithm. Computations are difficult and become intractable with more than three or four assessments. Applying the method to the quality-of-life data results in easily interpretable estimates, confirms the suspicion that the data are non-ignorably missing and highlights the likely bias of standard methods. Although treatment comparisons are not affected here, the methods are useful for obtaining unbiased means and estimating trends over time.     

Lattice conditional independence models for contingency tables with non-monotone missing data patterns

In the analysis of non-monotone missing data patterns in multinomial distributions for contingency tables, it is known that explicit MLEs of the unknown parameters cannot be obtained. Iterative procedures such as the EM-algorithm are therefore required to obtain the MLEs. These iterative procedures, however, may offer several potential difficulties. Andersson and Perlman [Ann. Statist. 21 (1993) 1318–1358] introduced lattice conditional independence (LCI) models for multivariate normal distributions, which can be applied to the analysis of non-monotone missing observations in continuous data (Andersson and Perlman, Statist. Probab. Lett. 12 (1991) 465–486). In this paper, we show that LCI models may also be applied to the analysis of categorical data with non-monotone missing data patterns. Under a parsimonious set of LCI assumptions naturally determined by the observed data pattern, the likelihood function for the observed data can be factored as in the monotone case and explicit MLEs can be obtained for the unknown parameters. Furthermore, the LCI assumptions can be tested by explicit likelihood ratio tests.     

Efficiency of lattice conditional independence models for multinormal samples with non-monotone missing data

For multivariate normal data with non-monotone (i.e. arbitrary) missing data patterns, lattice conditional independence (LCI) models determined by the observed data patterns can be used to obtain closed-form MLEs (Andersson and Perlman, 1991, 1993). In this paper, three procedures — LCI models, the EM algorithm, and the complete-data method — are compared by means of a Monte Carlo experiment. When the LCI model is accepted by the LR test, the LCI estimate is more efficient than those based on the EM algorithm and the complete-data method. When the LCI model is not accepted, the LCI estimate may lose efficiency but still may be more efficient than the EM estimate if the observed data is sparse. When the LCI model appears too restrictive, it may be possible to obtain a less restrictive LCI model by.discarding only a small portion of the incomplete observations. LCI models appear to be especially useful when the observed data is sparse, even in cases where the suitability of the LCI model is uncertain.     

Cox regression with missing covariate data using a modified partial likelihood method

Missing covariate values is a common problem in survival analysis. In this paper we propose a novel method for the Cox regression model that is close to maximum likelihood but avoids the use of the EM-algorithm. It exploits that the observed hazard function is multiplicative in the baseline hazard function with the idea being to profile out this function before carrying out the estimation of the parameter of interest. In this step one uses a Breslow type estimator to estimate the cumulative baseline hazard function. We focus on the situation where the observed covariates are categorical which allows us to calculate estimators without having to assume anything about the distribution of the covariates. We show that the proposed estimator is consistent and asymptotically normal, and derive a consistent estimator of the variance–covariance matrix that does not involve any choice of a perturbation parameter. Moderate sample size performance of the estimators is investigated via simulation and by application to a real data example.     

Semi-empirical likelihood inference for the ROC curve with missing data

The receiver operating characteristic (ROC) curve is one of the most commonly used methods to compare the diagnostic performance of two or more laboratory or diagnostic tests. In this paper, we propose semi-empirical likelihood based confidence intervals for ROC curves of two populations, where one population is parametric and the other one is non-parametric and both have missing data. After imputing missing values, we derive the semi-empirical likelihood ratio statistic and the corresponding likelihood equations. It is shown that the log-semi-empirical likelihood ratio statistic is asymptotically scaled chi-squared. The estimating equations are solved simultaneously to obtain the estimated lower and upper bounds of semi-empirical likelihood confidence intervals. We conduct extensive simulation studies to evaluate the finite sample performance of the proposed empirical likelihood confidence intervals with various sample sizes and different missing probabilities.     

Empirical likelihood and Wilks phenomenon for data with nonignorable missing values

Wilks's theorem is useful for constructing confidence regions. When applying the popular empirical likelihood to data with nonignorable nonresponses, Wilks's phenomenon does not hold. This paper unveils that this is caused by the extra estimation of the nuisance parameter in the nonignorable nonresponse propensity. Motivated by this result, we propose an adjusted empirical likelihood for which Wilks's theorem holds. Asymptotic results are presented and supplemented by simulation results for finite sample performance of the point estimators and confidence regions. An analysis of a data set is included for illustration.     

Dimension-reduced empirical likelihood inference for response mean with data missing at random

To make efficient inference for mean of a response variable when the data are missing at random and the dimension of covariate is not low, we construct three bias-corrected empirical likelihood (EL) methods in conjunction with dimension-reduced kernel estimation of propensity or/and conditional mean response function. Consistency and asymptotic normality of the maximum dimension-reduced EL estimators are established. We further study the asymptotic properties of the resulting dimension-reduced EL ratio functions and the corresponding EL confidence intervals for the response mean are constructed. The finite-sample performance of the proposed estimators is studied through simulation, and an application to HIV-CD4 data set is also presented.     

Empirical likelihood for a partially linear model with covariate data missing at random

In this paper, we consider the partial linear model with the covariables missing at random. Empirical likelihood ratios for the regression coefficients and the baseline function are investigated, the empirical log-likelihood ratios are proven to be asymptotically chi-squared and the corresponding confidence regions for the parameters of interest are then constructed. The finite sample behavior of the proposed method is evaluated with simulation and illustrated with an AIDS clinical trial dataset.     

Smoothed empirical likelihood for quantile regression models with response data missing at random

This paper studies smoothed quantile linear regression models with response data missing at random. Three smoothed quantile empirical likelihood ratios are proposed first and shown to be asymptotically Chi-squared. Then, the confidence intervals for the regression coefficients are constructed without the estimation of the asymptotic covariance. Furthermore, a class of estimators for the regression parameter is presented to derive its asymptotic distribution. Simulation studies are conducted to assess the finite sample performance. Finally, a real-world data set is analyzed to illustrated the effectiveness of the proposed methods.     

Maximum likelihood estimation with missing spatial data and with an application to remotely sensed data

The paper examines the small and large lattice properties of the exact maximum likelihood estimator for a spatial model where parameter estimation and missing data estimation are tackled simultaneously, A first order conditional autoregressive model is examined in detail. The paper concludes with an empirical analysis of remotely sensed data.     

Empirical likelihood confidence intervals under imputation for missing survey data from stratified simple random sampling

Missing observations due to non‐response are commonly encountered in data collected from sample surveys. The focus of this article is on item non‐response which is often handled by filling in (or imputing) missing values using the observed responses (donors). Random imputation (single or fractional) is used within homogeneous imputation classes that are formed on the basis of categorical auxiliary variables observed on all the sampled units. A uniform response rate within classes is assumed, but that rate is allowed to vary across classes. We construct confidence intervals (CIs) for a population parameter that is defined as the solution to a smooth estimating equation with data collected using stratified simple random sampling. The imputation classes are assumed to be formed across strata. Fractional imputation with a fixed number of random draws is used to obtain an imputed estimating function. An empirical likelihood inference method under the fractional imputation is proposed and its asymptotic properties are derived. Two asymptotically correct bootstrap methods are developed for constructing the desired CIs. In a simulation study, the proposed bootstrap methods are shown to outperform traditional bootstrap methods and some non‐bootstrap competitors under various simulation settings. The Canadian Journal of Statistics 47: 281–301; 2019 © 2019 Statistical Society of Canada     

Empirical likelihood for nonlinear models with missing responses

In this paper, a nonlinear model with response variables missing at random is studied. In order to improve the coverage accuracy for model parameters, the empirical likelihood (EL) ratio method is considered. On the complete data, the EL statistic for the parameters and its approximation have a χ² asymptotic distribution. When the responses are reconstituted using a semi-parametric method, the empirical log-likelihood on the response variables associated with the imputed data is also asymptotically χ². The Wilks theorem for EL on the parameters, based on reconstituted data, is also satisfied. These results can be used to construct the confidence region for the model parameters and the response variables. It is shown via Monte Carlo simulations that the EL methods outperform the normal approximation-based method in terms of coverage probability for the unknown parameter, including on the reconstituted data. The advantages of the proposed method are exemplified on real data.     

Estimation of multi-state models with missing covariate values based on observed data likelihood

Abstract

Continuous-time multi-state models are commonly used to study diseases with multiple stages. Potential risk factors associated with the disease are added to the transition intensities of the model as covariates, but missing covariate measurements arise frequently in practice. We propose a likelihood-based method that deals efficiently with a missing covariate in these models. Our simulation study showed that the method performs well for both “missing completely at random” and “missing at random” mechanisms. We also applied our method to a real dataset, the Einstein Aging Study.     

Expedient universal robust likelihood method for general dispersed count data

ABSTRACT

We introduce a universal robust likelihood approach for regression analysis of general count data. The robust likelihood function is able to accommodate a wide range of dispersion and is insensitive to model failures. We use simulations and real data analysis to demonstrate the merit of the robust procedure.     

Two-step and likelihood methods for HIV viral dynamic models with covariate measurement errors and missing data

HIV viral dynamic models have received much attention in the literature. Long-term viral dynamics may be modelled by semiparametric nonlinear mixed-effect models, which incorporate large variation between subjects and autocorrelation within subjects and are flexible in modelling complex viral load trajectories. Time-dependent covariates may be introduced in the dynamic models to partially explain the between-individual variations. In the presence of measurement errors and missing data in time-dependent covariates, we show that the commonly used two-step method may give approximately unbiased estimates but may under-estimate standard errors. We propose a two-stage bootstrap method to adjust the standard errors in the two-step method and a likelihood method.     

On empirical likelihood for linear models with missing responses

Suppose that we have a linear regression model Y=X^′β+_ν0(X)ε$Y=X^{\prime }\beta +{\nu }_0(X)\epsilon$ with random error ε$\epsilon$, where X is a random design variable and is observed completely, and Y is the response variable and some Y-values are missing at random (MAR). In this paper, based on the ‘complete’ data set for Y after inverse probability weighted imputation, we construct empirical likelihood statistics on EY   and β$\beta$ which have the χ²${\chi }^2$-type limiting distributions under some new conditions compared with Xue (2009). Our results broaden the applicable scope of the approach combined with Xue (2009).