Empirical likelihood inference for a semiparametric hazards regression model

ABSTRACT

We investigated the empirical likelihood inference approach under a general class of semiparametric hazards regression models with survival data subject to right-censoring. An empirical likelihood ratio for the full 2p regression parameters involved in the model is obtained. We showed that it converged weakly to a random variable which could be written as a weighted sum of 2p independent chi-squared variables with one degree of freedom. Using this, we could construct a confidence region for parameters. We also suggested an adjusted version for the preceding statistic, whose limit followed a standard chi-squared distribution with 2p degrees of freedom.     

A comparison of two approaches for power and sample size calculations in logistic regression models

Whittemore (1981) proposed an approach for calculating the sample size needed to test hypotheses with specified significance and power against a given alternative for logistic regression with small response probability. Based on the distribution of covariate, which could be either discrete or continuous, this approach first provides a simple closed-form approximation to the asymptotic covariance matrix of the maximum likelihood estimates, and then uses it to calculate the sample size needed to test a hypothesis about the parameter. Self et al. (1992) described a general approach for power and sample size calculations within the framework of generalized linear models, which include logistic regression as a special case. Their approach is based on an approximation to the distribution of the likelihood ratio statistic. Unlike the Whittemore approach, their approach is not limited to situations of small response probability. However, it is restricted to models with a finite number of covariate configurations. This study compares these two approaches to see how accurate they would be for the calculations of power and sample size in logistic regression models with various response probabilities and covariate distributions. The results indicate that the Whittemore approach has a slight advantage in achieving the nominal power only for one case with small response probability. It is outperformed for all other cases with larger response probabilities. In general, the approach proposed in Self et al. (1992) is recommended for all values of the response probability. However, its extension for logistic regression models with an infinite number of covariate configurations involves an arbitrary decision for categorization and leads to a discrete approximation. As shown in this paper, the examined discrete approximations appear to be sufficiently accurate for practical purpose.     

A test for the linearity of the nonparametric part of a semiparametric logistic regression model

A semiparametric logistic regression model is proposed in which its nonparametric component is approximated with fixed-knot cubic B-splines. To assess the linearity of the nonparametric component, we construct a penalized likelihood ratio test statistic. When the number of knots is fixed, the null distribution of the test statistic is shown to be asymptotically the distribution of a linear combination of independent chi-squared random variables, each with one degree of freedom. We set the asymptotic null expectation of this test statistic equal to a value to determine the smoothing parameter value. Monte Carlo experiments are conducted to investigate the performance of the proposed test. Its practical use is illustrated with a real-life example.     

Graphics for studying logistic regression models

In this article we focus on logistic regression models for binary responses. An existing result shows that the log-odds can be modelled depending on the log of the ratio between the conditional densities of the predictors given the response variable. This suggests that relevant statistical information could be extracted investigating the inverse problem. Thus, we present different methods for studying the log-density ratio through graphs, which allow us to select which predictors are needed, and how they should be included in a logistic regression model. We also discuss data analysis examples based on real datasets available in literature in order to provide further insights into the methodology proposed.     

Optimal robust influence functions in semiparametric regression

Robust statistics allows the distribution of the observations to be any member of a suitable neighborhood about an ideal model distribution. In this paper, the ideal models are semiparametric with finite-dimensional parameter of interest and a possibly infinite-dimensional nuisance parameter.In the asymptotic setup of shrinking neighborhoods, we derive and study the Hampel-type problem and the minmax MSE-problem. We show that, for all common types of neighborhood systems, the optimal influence function can be approximated by the optimal influence functions for certain parametric models.For general semiparametric regression models, we determine in case of error-in-variables and in case of error-free-variables.Finally, the results are applied to Cox regression where we compare our approach to that of Bednarski [1993. Robust estimation in Cox's regression model. Scand. J. Statist. 20, 213–225] in a small simulation study and on a real data set.     

A Goodness-of-Fit Test for Logistic Regression Models in Stratified Case-Control Studies via Empirical Likelihood

In the literature, there were only a few reports on goodness-of-fit tests on logistic regression models specifically derived for case-control studies. In this article, we propose a goodness-of-fit test for logistic regression models in stratified case-control studies using an empirical likelihood approach. The proposed statistic is an alternative to the statistic G _o , recently proposed by Arbigast and Lin (2005 Arbigast , P. G. , Lin , D. Y. ( 2005 ). Model-checking techniques for stratified case-control studies . Statist. Med. 24 : 229 – 247 . [Google Scholar]). Simulation results show that the proposed statistic is often slightly more powerful than G _o , although their performances are always close to each other. Moreover, implementation of our method is easy since the usual stratified logistic regression procedures in many statistical softwares can be employed. Some asymptotic results and application of the proposed statistic to two real datasets are also presented.     

Methods for missing covariates in logistic regression

Various methods have been suggested in the literature to handle a missing covariate in the presence of surrogate covariates. These methods belong to one of two paradigms. In the imputation paradigm, Pepe and Fleming (1991) and Reilly and Pepe (1995) suggested filling in missing covariates using the empirical distribution of the covariate obtained from the observed data. We can proceed one step further by imputing the missing covariate using nonparametric maximum likelihood estimates (NPMLE) of the density of the covariate. Recently Murphy and Van der Vaart (1998a) showed that such an approach yields a consistent, asymptotically normal, and semiparametric efficient estimate for the logistic regression coefficient. In the weighting paradigm, Zhao and Lipsitz (1992) suggested an estimating function using completely observed records after weighting inversely by the probability of observation. An extension of this weighting approach designed to achieve semiparametric efficient bound is considered by Robins, Hsieh and Newey (RHN) (1995). The two ends of each paradigm (NPMLE and RHN) attain the efficiency bound and are asymptotically equivalent. However, both require a substantial amount of computation. A question arises whether and when, in practical situations, this extensive computation is worthwhile. In this paper we investigate the performance of single and multiple imputation estimates, weighting estimates, semiparametric efficient estimates, and two new imputation estimates. Simulation studies suggest that the sample size should be substantially large (e.g. n=2000) for NPMLE and RHN to be more efficient than simpler imputation estimates. When the sample size is moderately large (n≤ 1500), simpler imputation estimates have as small a variance as semiparametric efficient estimates.     

Robust test for means when population variances are unequal

We consider the problem of testing the equality of two population means when the population variances are not necessarily equal. We propose a Welch-type statistic, say T^* _c, based on Tiku!s ‘1967, 1980’ modified maximum likelihood estimators, and show that this statistic is robust to symmetric and moderately skew distributions. We investigate the power properties of the statistic T^* _c; T^* _c clearly seems to be more powerful than Yuen's ‘1974’ Welch-type robust statistic based on the trimmed sample means and the matching sample variances. We show that the analogous statistics based on the ‘adaptive’ robust estimators give misleading Type I errors. We generalize the results to testing linear contrasts among k population means     

The extreme residuals in logistic regression models

Goodness-of-fit tests for logistic regression models using extreme residuals are considered. Approximations to the moments of the Pearson residuals are given for model fits made by maximum likelihood, minimum chi-square and weighted least squares and used to define modified residuals. Approximations to the critical values of the extreme statistics based on the ordinary and modified Pearson residuals are developed and assessed for the case of a single explanatory variable.     

Empirical likelihood based diagnostics for heteroscedasticity in semiparametric varying-coefficient partially linear errors-in-variables models

In this paper, we propose an empirical likelihood based diagnostic technique for heteroscedasticity in the semiparametric varying-coefficient partially linear errors-in-variables models. Under mild conditions, a nonparametric version of Wilk’s theorem is derived. Simulation results reveal that our test performs well in both size and power.     

Orthogonal weighted empirical likelihood-based variable selection for semiparametric instrumental variable models

In this article, we consider the variable selection for a class of semiparametric instrumental variable models. By combining orthogonal weighting technology and empirical likelihood method, we propose an orthogonal weighted empirical likelihood-based variable selection procedure. Under some mild conditions, the consistency and sparsity of the variable selection procedure are studied. Furthermore, some simulation studies and a real data analysis are carried out to examine the finite-sample performance of the proposed method.     

Using Adjacent-category Logits Procedure for Estimating Receiver Operating Characteristic Surface

This article presents a semiparametric method for estimating receiver operating characteristic surface under density ratio model. The construction of the proposed method is based on the adjacent-category logit model and the empirical likelihood approach. A bootstrap approach for the VUS estimator inference is presented. In a simulation study, the proposed estimator is compared with the existing parametric and nonparametric estimators in terms of bias, standard error, and mean square error. Finally, a real data example and some discussions on the proposed method are provided.     

Improving predictive accuracy of logistic regression model using ranked set samples

Logistic regression is often confronted with separation of likelihood problem, especially with unbalanced success–failure distribution. We propose to address this issue by drawing a ranked set sample (RSS). Simulation studies illustrated the advantages of logistic regression models fitted with RSS samples with small sample size regardless of the distribution of the binary response. As sample size increases, RSS eventually becomes comparable to SRS, but still has the advantage over SRS in mitigating the problem of separation of likelihood. Even in the presence of ranking errors, models from RSS samples yield higher predictive ability than its SRS counterpart.     

A two sample ordered alternative test for means and variances

A two sample test of likelihood ratio type is proposed, assuming normal distribution theory, for testing the hypothesis that two samples come from identical normal populations versus the alternative that the populations are normal but vary in mean value and variance with one population having a smaller mean and smaller variance than the other. The small sample and large sample distribution of the proposed statistic are derived assuming normality. Some computations are presented which show the speed of convergence of small sample critical values to their asymptotic counterparts. Comparisons of local power of the proposed test are made with several potential competing tests. Asymptotics for the test statistic are derived when underlying distributions are not necessarily normal.     

An improved and efficient biased estimation technique in logistic regression model

Abstract

In this article, we propose a new improved and efficient biased estimation method which is a modified restricted Liu-type estimator satisfying some sub-space linear restrictions in the binary logistic regression model. We study the properties of the new estimator under the mean squared error matrix criterion and our results show that under certain conditions the new estimator is superior to some other estimators. Moreover, a Monte Carlo simulation study is conducted to show the performance of the new estimator in the simulated mean squared error and predictive median squared errors sense. Finally, a real application is considered.     

On the existence of maximum likelihood estimates for weighted logistic regression

The problems of existence and uniqueness of maximum likelihood estimates for logistic regression were completely solved by Silvapulle in 1981 and Albert and Anderson in 1984. In this paper, we extend the well-known results by Silvapulle and by Albert and Anderson to weighted logistic regression. We analytically prove the equivalence between the overlap condition used by Albert and Anderson and that used by Silvapulle. We show that the maximum likelihood estimate of weighted logistic regression does not exist if there is a complete separation or a quasicomplete separation of the data points, and exists and is unique if there is an overlap of data points. Our proofs and results for weighted logistic apply to unweighted logistic regression.     

Semiparametric ROC surface estimation for continuous diagnostic tests via polytomous logistic regression procedures

In this paper, we propose a semiparametric method of estimating receiver operating characteristic (ROC) surfaces for continuous diagnostic tests under density ratio models. Implementation of our method is easy since the usual polytomous logistic regression procedures in many statistical software packages can be employed. A simulated example is provided to facilitate the implementation of our method. Simulation results show that the proposed semiparametric ROC surface estimator is more efficient than the nonparametric counterpart and the parametric counterpart whether the normality assumption of data holds or not. Moreover, some simulation results on the underlying semiparametric distribution function estimators are also reported. In addition, some discussions on the proposed method as well as analysis of a real data set are provided.     

Invariant properties of logistic regression model in credit scoring under monotonic transformations

Monotonic transformations of explanatory continuous variables are often used to improve the fit of the logistic regression model to the data. However, no analytic studies have been done to study the impact of such transformations. In this paper, we study invariant properties of the logistic regression model under monotonic transformations. We prove that the maximum likelihood estimates, information value, mutual information, Kolmogorov–Smirnov (KS) statistics, and lift table are all invariant under certain monotonic transformations.     

Estimating the density of a possibly missing response variable in nonlinear regression

This paper considers linear and nonlinear regression with a response variable that is allowed to be “missing at random”. The only structural assumptions on the distribution of the variables are that the errors have mean zero and are independent of the covariates. The independence assumption is important. It enables us to construct an estimator for the response density that uses all the observed data, in contrast to the usual local smoothing techniques, and which therefore permits a faster rate of convergence. The idea is to write the response density as a convolution integral which can be estimated by an empirical version, with a weighted residual-based kernel estimator plugged in for the error density. For an appropriate class of regression functions, and a suitably chosen bandwidth, this estimator is consistent and converges with the optimal parametric rate n^1/2. Moreover, the estimator is proved to be efficient (in the sense of Hájek and Le Cam) if an efficient estimator is used for the regression parameter.     

Preliminary phi-divergence test estimators for linear restrictions in a logistic regression model

The problem of estimation of the parameters in a logistic regression model is considered under multicollinearity situation when it is suspected that the parameter of the logistic regression model may be restricted to a subspace. We study the properties of the preliminary test based on the minimum ϕ -divergence estimator as well as in the ϕ -divergence test statistic. The minimum ϕ -divergence estimator is a natural extension of the maximum likelihood estimator and the ϕ -divergence test statistics is a family of the test statistics for testing the hypothesis that the regression coefficients may be restricted to a subspace.