Corrected likelihood-ratio tests in logistic regression using small-sample data

Likelihood-ratio tests (LRTs) are often used for inferences on one or more logistic regression coefficients. Conventionally, for given parameters of interest, the nuisance parameters of the likelihood function are replaced by their maximum likelihood estimates. The new function created is called the profile likelihood function, and is used for inference from LRT. In small samples, LRT based on the profile likelihood does not follow χ² distribution. Several corrections have been proposed to improve LRT when used with small-sample data. Additionally, complete or quasi-complete separation is a common geometric feature for small-sample binary data. In this article, for small-sample binary data, we have derived explicitly the correction factors of LRT for models with and without separation, and proposed an algorithm to construct confidence intervals. We have investigated the performances of different LRT corrections, and the corresponding confidence intervals through simulations. Based on the simulation results, we propose an empirical rule of thumb on the use of these methods. Our simulation findings are also supported by real-world data.     

A tolerance interval approach for assessment of agreement in method comparison studies with repeated measurements

This paper generalizes the tolerance interval approach for assessing agreement between two methods of continuous measurement for repeated measurement data—a common scenario in applications. The repeated measurements may be longitudinal or they may be replicates of the same underlying measurement. Our approach is to first model the data using a mixed model and then construct a relevant asymptotic tolerance interval (or band) for the distribution of appropriately defined differences. We present the methodology in the general context of a mixed model that can incorporate covariates, heteroscedasticity and serial correlation in the errors. Simulation for the no-covariate case shows good small-sample performance of the proposed methodology. For the longitudinal data, we also describe an extension for the case when the observed time profiles are modelled nonparametrically through penalized splines. Two real data applications are presented.     

Penalized empirical likelihood inference for sparse additive hazards regression with a diverging number of covariates

High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by applications in high-throughput genomic data analysis. In this paper, we propose a class of regularization methods, integrating both the penalized empirical likelihood and pseudoscore approaches, for variable selection and estimation in sparse and high-dimensional additive hazards regression models. When the number of covariates grows with the sample size, we establish asymptotic properties of the resulting estimator and the oracle property of the proposed method. It is shown that the proposed estimator is more efficient than that obtained from the non-concave penalized likelihood approach in the literature. Based on a penalized empirical likelihood ratio statistic, we further develop a nonparametric likelihood approach for testing the linear hypothesis of regression coefficients and constructing confidence regions consequently. Simulation studies are carried out to evaluate the performance of the proposed methodology and also two real data sets are analyzed.     

Empirical Likelihood Intervals for Conditional Value‐at‐Risk in Heteroscedastic Regression Models

Abstract. Non‐parametric regression models have been studied well including estimating the conditional mean function, the conditional variance function and the distribution function of errors. In addition, empirical likelihood methods have been proposed to construct confidence intervals for the conditional mean and variance. Motivated by applications in risk management, we propose an empirical likelihood method for constructing a confidence interval for the pth conditional value‐at‐risk based on the non‐parametric regression model. A simulation study shows the advantages of the proposed method.     

Empirical likelihood-based inference in nonlinear regression models with missing responses at random

This paper investigates the estimations of regression parameters and response mean in nonlinear regression models in the presence of missing response variables that are missing with missingness probabilities depending on covariates. We propose four empirical likelihood (EL)-based estimators for the regression parameters and the response mean. The resulting estimators are shown to be consistent and asymptotically normal under some general assumptions. To construct the confidence regions for the regression parameters as well as the response mean, we develop four EL ratio statistics, which are proven to have the χ² distribution asymptotically. Simulation studies and an artificial data set are used to illustrate the proposed methodologies. Empirical results show that the EL method behaves better than the normal approximation method and that the coverage probabilities and average lengths depend on the selection probability function.     

Finite-Sample Properties of the Maximum Likelihood Estimator for the Poisson Regression Model With Random Covariates

We examine the small-sample behaviour of the maximum likelihood estimator for the Poisson regression model with random covariates. Analytic expressions for the second-order bias and mean squared error are derived, and we undertake some numerical evaluations to illustrate these results for the single covariate case. The properties of the bias-adjusted maximum likelihood estimator are investigated in a Monte Carlo experiment. Correcting the estimator for its second-order bias is found to be effective in the cases considered, and we recommend its use when the Poisson regression model is estimated by maximum likelihood with small samples.     

Empirical Likelihood-based Inference in Linear Models with Missing Data

The missing response problem in linear regression is studied. An adjusted empirical likelihood approach to inference on the mean of the response variable is developed. A non-parametric version of Wilks's theorem for the adjusted empirical likelihood is proved, and the corresponding empirical likelihood confidence interval for the mean is constructed. With auxiliary information, an empirical likelihood-based estimator with asymptotic normality is defined and an adjusted empirical log-likelihood function with asymptotic χ² is derived. A simulation study is conducted to compare the adjusted empirical likelihood methods and the normal approximation methods in terms of coverage accuracies and average lengths of the confidence intervals. Based on biases and standard errors, a comparison is also made between the empirical likelihood-based estimator and related estimators by simulation. Our simulation indicates that the adjusted empirical likelihood methods perform competitively and the use of auxiliary information provides improved inferences.     

Estimation and inference of combining quantile and least-square regressions with missing data

In this paper, we consider how to incorporate quantile information to improve estimator efficiency for regression model with missing covariates. We combine the quantile information with least-squares normal equations and construct an unbiased estimating equations (EEs). The lack of smoothness of the objective EEs is overcome by replacing them with smooth approximations. The maximum smoothed empirical likelihood (MSEL) estimators are established based on inverse probability weighted (IPW) smoothed EEs and their asymptotic properties are studied under some regular conditions. Moreover, we develop two novel testing procedures for the underlying model. The finite-sample performance of the proposed methodology is examined by simulation studies. A real example is used to illustrate our methods.     

Full likelihood inference in the Cox model with LTRC data when covariates are discrete

For the regression parameter β in the Cox model, there have been several estimates based on different types of approximated likelihood. For right-censored data, Ren and Zhou [Full likelihood inferences in the Cox model: an empirical approach. Ann Inst Statist Math. 2011;63:1005–1018] derive the full likelihood function for (β, F₀), where F₀ is the baseline distribution function in the Cox model. In this article, we extend their results to left-truncated and right-censored data with discrete covariates. Using the empirical likelihood parameterization, we obtain the full-profile likelihood function for β when covariates are discrete. Simulation results indicate that the maximum likelihood estimator outperforms Cox's partial likelihood estimator in finite samples.     

Asymptotic Normality of M-Estimators for Varying Coefficient Models with Longitudinal Data

This article considers a nonparametric varying coefficient regression model with longitudinal observations. The relationship between the dependent variable and the covariates is assumed to be linear at a specific time point, but the coefficients are allowed to change over time. A general formulation is used to treat mean regression, median regression, quantile regression, and robust mean regression in one setting. The local M-estimators of the unknown coefficient functions are obtained by local linear method. The asymptotic distributions of M-estimators of unknown coefficient functions at both interior and boundary points are established. Various applications of the main results, including estimating conditional quantile coefficient functions and robustifying the mean regression coefficient functions are derived. Finite sample properties of our procedures are studied through Monte Carlo simulations.     

Asymptotic Results of a Nonparametric Conditional Quantile Estimator for Functional Time Series

We consider the estimation of the conditional quantile function when the covariates take values in some abstract function space. The main goal of this article is to establish the almost complete convergence and the asymptotic normality of the kernel estimator of the conditional quantile under the α-mixing assumption and on the concentration properties on small balls of the probability measure of the functional regressors. Some applications and particular cases are studied. This approach can be applied in time series analysis to the prediction and building of confidence bands. We illustrate our methodology with El Niño data.     

Higher order inference for stress–strength reliability with independent Burr-type X distributions

In this paper, a small-sample asymptotic method is proposed for higher order inference in the stress–strength reliability model, R=P(Y<X), where X and Y are distributed independently as Burr-type X distributions. In a departure from the current literature, we allow the scale parameters of the two distributions to differ, and the likelihood-based third-order inference procedure is applied to obtain inference for R. The difficulty of the implementation of the method is in obtaining the the constrained maximum likelihood estimates (MLE). A penalized likelihood method is proposed to handle the numerical complications of maximizing the constrained likelihood model. The proposed procedures are illustrated using a sample of carbon fibre strength data. Our results from simulation studies comparing the coverage probabilities of the proposed small-sample asymptotic method with some existing large-sample asymptotic methods show that the proposed method is very accurate even when the sample sizes are small.     

Regularized robust estimation in binary regression models

In this paper, we investigate robust parameter estimation and variable selection for binary regression models with grouped data. We investigate estimation procedures based on the minimum-distance approach. In particular, we employ minimum Hellinger and minimum symmetric chi-squared distances criteria and propose regularized minimum-distance estimators. These estimators appear to possess a certain degree of automatic robustness against model misspecification and/or for potential outliers. We show that the proposed non-penalized and penalized minimum-distance estimators are efficient under the model and simultaneously have excellent robustness properties. We study their asymptotic properties such as consistency, asymptotic normality and oracle properties. Using Monte Carlo studies, we examine the small-sample and robustness properties of the proposed estimators and compare them with traditional likelihood estimators. We also study two real-data applications to illustrate our methods. The numerical studies indicate the satisfactory finite-sample performance of our procedures.     

Comparisons of approximate confidence intervals for the smallest extreme value distribution simple linear regression model under time censoring

Exact confidence interval estimation for accelerated life regression models with censored smallest extreme value (or Weibull) data is often impractical. This paper evaluates the accuracy of approximate confidence intervals based on the asymptotic normality of the maximum likelihood estimator, the asymptotic X²distribution of the likelihood ratio statistic, mean and variance correction to the likelihood ratio statistic, and the so-called Bartlett correction to the likelihood ratio statistic. The Monte Carlo evaluations under various degrees of time censoring show that uncorrected likelihood ratio intervals are very accurate in situations with heavy censoring. The benefits of mean and variance correction to the likelihood ratio statistic are only realized with light or no censoring. Bartlett correction tends to result in conservative intervals. Intervals based on the asymptotic normality of maximum likelihood estimators are anticonservative and should be used with much caution.     

On the estimation of normal copula discrete regression models using the continuous extension and simulated likelihood

The continuous extension of a discrete random variable is amongst the computational methods used for estimation of multivariate normal copula-based models with discrete margins. Its advantage is that the likelihood can be derived conveniently under the theory for copula models with continuous margins, but there has not been a clear analysis of the adequacy of this method. We investigate the asymptotic and small-sample efficiency of two variants of the method for estimating the multivariate normal copula with univariate binary, Poisson, and negative binomial regressions, and show that they lead to biased estimates for the latent correlations, and the univariate marginal parameters that are not regression coefficients. We implement a maximum simulated likelihood method, which is based on evaluating the multidimensional integrals of the likelihood with randomized quasi-Monte Carlo methods. Asymptotic and small-sample efficiency calculations show that our method is nearly as efficient as maximum likelihood for fully specified multivariate normal copula-based models. An illustrative example is given to show the use of our simulated likelihood method.     

Nonnull asymptotic distributions of the LR,Wald, score and gradient statistics in generalized linear models with dispersion covariates

The class of generalized linear models with dispersion covariates, which allows us to jointly model the mean and dispersion parameters, is a natural extension to the classical generalized linear models. In this paper, we derive the asymptotic expansions under a sequence of Pitman alternatives (up to order n ^?1/2) for the nonnull distribution functions of the likelihood ratio, Wald, Rao score and gradient statistics in this class of models. The asymptotic distributions of these statistics are obtained for testing a subset of regression parameters and for testing a subset of dispersion parameters. Based on these nonnull asymptotic expansions, the power of all four tests, which are equivalent to first order, are compared. Furthermore, we consider Monte Carlo simulations in order to compare the finite-sample performance of these tests in this class of models. We present two empirical applications to two real data sets for illustrative purposes.     

Inference in Semi‐Parametric Dynamic Models for Repeated Count Data

This paper deals with a longitudinal semi‐parametric regression model in a generalised linear model setup for repeated count data collected from a large number of independent individuals. To accommodate the longitudinal correlations, we consider a dynamic model for repeated counts which has decaying auto‐correlations as the time lag increases between the repeated responses. The semi‐parametric regression function involved in the model contains a specified regression function in some suitable time‐dependent covariates and a non‐parametric function in some other time‐dependent covariates. As far as the inference is concerned, because the non‐parametric function is of secondary interest, we estimate this function consistently using the independence assumption‐based well‐known quasi‐likelihood approach. Next, the proposed longitudinal correlation structure and the estimate of the non‐parametric function are used to develop a semi‐parametric generalised quasi‐likelihood approach for consistent and efficient estimation of the regression effects in the parametric regression function. The finite sample performance of the proposed estimation approach is examined through an intensive simulation study based on both large and small samples. Both balanced and unbalanced cluster sizes are incorporated in the simulation study. The asymptotic performances of the estimators are given. The estimation methodology is illustrated by reanalysing the well‐known health care utilisation data consisting of counts of yearly visits to a physician by 180 individuals for four years and several important primary and secondary covariates.     

Joint estimation of the mean and dispersion parameters in the analysis of proportions: A comparison of efficiency and bias

This paper deals with joint estimation of the mean and dispersion parameters in the analysis of proportions. We consider a parametric model, namely the extended beta-binomial model, and several semiparametric procedures. We study large-sample efficiency and small-sample bias and efficiency properties of the estimates of the mean and intraclass correlation parameters. Estimation and efficiency calculations are présentés for the regression model. However, for simplicity, numerical large-sample efficiency and small-sample bias and efficiency calculations are performed for the two-parameter model only. Numerical efficiency results are présentés in terms of graphs. Estimated asymptotic efficiencies of various estimates are also compared for two data sets. Our findings suggest that for the estimation of the mean (regression) parameters the quasilikelihood procedure performs best. However, for the joint estimation, the Gaussian likelihood estimates perform best.     

A Mixed Model Approach for Geoadditive Hazard Regression

Abstract.  Mixed model based approaches for semiparametric regression have gained much interest in recent years, both in theory and application. They provide a unified and modular framework for penalized likelihood and closely related empirical Bayes inference. In this article, we develop mixed model methodology for a broad class of Cox-type hazard regression models where the usual linear predictor is generalized to a geoadditive predictor incorporating non-parametric terms for the (log-)baseline hazard rate, time-varying coefficients and non-linear effects of continuous covariates, a spatial component, and additional cluster-specific frailties. Non-linear and time-varying effects are modelled through penalized splines, while spatial components are treated as correlated random effects following either a Markov random field or a stationary Gaussian random field prior. Generalizing existing mixed model methodology, inference is derived using penalized likelihood for regression coefficients and (approximate) marginal likelihood for smoothing parameters. In a simulation we study the performance of the proposed method, in particular comparing it with its fully Bayesian counterpart using Markov chain Monte Carlo methodology, and complement the results by some asymptotic considerations. As an application, we analyse leukaemia survival data from northwest England.     

Doubly robust empirical likelihood inference in covariate-missing data problems

Missing covariate data occurs often in regression analysis. We study methods for estimating the regression coefficients in an assumed conditional mean function when some covariates are completely observed but other covariates are missing for some subjects. We adopt the semiparametric perspective of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866] on regression analyses with missing covariates, in which they pioneered the use of two working models, the working propensity score model and the working conditional score model. A recent approach to missing covariate data analysis is the empirical likelihood method of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503], which effectively combines unbiased estimating equations. In this paper, we consider an alternative likelihood approach based on the full likelihood of the observed data. This full likelihood-based method enables us to generate estimators for the vector of the regression coefficients that are (a) asymptotically equivalent to those of Qin et al. [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the working propensity score model is correctly specified, and (b) doubly robust, like the augmented inverse probability weighting (AIPW) estimators of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Am Statist Assoc. 1994;89:846–866]. Thus, the proposed full likelihood-based estimators improve on the efficiency of the AIPW estimators when the working propensity score model is correct but the working conditional score model is possibly incorrect, and also improve on the empirical likelihood estimators of Qin, Zhang and Leung [Empirical likelihood in missing data problems. J Amer Statist Assoc. 2009;104:1492–1503] when the reverse is true, that is, the working conditional score model is correct but the working propensity score model is possibly incorrect. In addition, we consider a regression method for estimation of the regression coefficients when the working conditional score model is correctly specified; the asymptotic variance of the resulting estimator is no greater than the semiparametric variance bound characterized by the theory of Robins et al. [Estimation of regression coefficients when some regressors are not always observed. J Amer Statist Assoc. 1994;89:846–866]. Finally, we compare the finite-sample performance of various estimators in a simulation study.