Consistency of non parametric estimators of the area under the ROC curve

ABSTRACT

The area under the receiver operating characteristic (ROC) curve is a popular summary index that measures the accuracy of a continuous-scale diagnostic test to measure its accuracy. Under certain conditions on estimators of distribution functions, we prove a theorem on strong consistency of the non parametric “plugin” estimators of the area under the ROC curve. Based on this theorem, we construct some new “plugin” consistent estimators. The performance of the non parametric estimators considered is illustrated numerically and the estimators are compared in terms of bias, variance, and mean square error.     

Cumulative/dynamic ROC curve estimation

ABSTRACT

Receiver operating-characteristic (ROC) curve is a popular graphical method frequently used in order to study the diagnostic capacity of continuous (bio)markers. When the considered outcome is a time-dependent variable, the direct generalization is known as cumulative/dynamic ROC curve. For a fixed point of time, t, one subject is allocated into the positive group if the event happens before t and into the negative group if the event is not happened at t. The presence of censored subject, which can not be directly assigned into a group, is the main handicap of this approach. The proposed cumulative/dynamic ROC curve estimator assigns a probability to belong to the negative (positive) group to the subjects censored previously to t. The performance of the resulting estimator is studied from Monte Carlo simulations. Some real-world applications are reported. Results suggest that the new estimators provide a good approximation to the real cumulative/dynamic ROC curve.     

An application of lomax distributions in receiver operating characteristic(roc)curve analysis

Receiver operating characteristic(ROC)curves are useful for studying the performance of diagnostic tests. ROC curves occur in many fields of applications including psychophysics, quality control and medical diagnostics. In practical situations, often the responses to a diagnostic test are classified into a number of ordered categories. Such data are referred to as ratings data. It is typically assumed that the underlying model is based on a continuous probability distribution. The ROC curve is then constructed from such data using this probability model. Properties of the ROC curve are inherited from the model. Therefore, understanding the role of different probability distributions in ROC modeling is an interesting and important area of research. In this paper the Lomax distribution is considered as a model for ratings data and the corresponding ROC curve is derived. The maximum likelihood estimation procedure for the related parameters is discussed. This procedure is then illustrated in the analysis of a neurological data example.     

Statistical analysis of non-inferiority via non-zero risk difference in stratified matched-pair studies

A stratified study is often designed for adjusting several independent trials in modern medical research. We consider the problem of non-inferiority tests and sample size determinations for a nonzero risk difference in stratified matched-pair studies, and develop the likelihood ratio and Wald-type weighted statistics for testing a null hypothesis of non-zero risk difference for each stratum in stratified matched-pair studies on the basis of (1) the sample-based method and (2) the constrained maximum likelihood estimation (CMLE) method. Sample size formulae for the above proposed statistics are derived, and several choices of weights for Wald-type weighted statistics are considered. We evaluate the performance of the proposed tests according to type I error rates and empirical powers via simulation studies. Empirical results show that (1) the likelihood ratio and the Wald-type CMLE test based on harmonic means of the stratum-specific sample size (SSIZE) weight (the Cochran's test) behave satisfactorily in the sense that their significance levels are much closer to the prespecified nominal level; (2) the likelihood ratio test is better than Nam's [2006. Non-inferiority of new procedure to standard procedure in stratified matched-pair design. Biometrical J. 48, 966–977] score test; (3) the sample sizes obtained by using SSIZE weight are smaller than other weighted statistics in general; (4) the Cochran's test statistic is generally much better than other weighted statistics with CMLE method. A real example from a clinical laboratory study is used to illustrate the proposed methodologies.     

Compare diagnostic tests using transformation-invariant smoothed ROC curves()

Receiver operating characteristic (ROC) curve, plotting true positive rates against false positive rates as threshold varies, is an important tool for evaluating biomarkers in diagnostic medicine studies. By definition, ROC curve is monotone increasing from 0 to 1 and is invariant to any monotone transformation of test results. And it is often a curve with certain level of smoothness when test results from the diseased and non-diseased subjects follow continuous distributions. Most existing ROC curve estimation methods do not guarantee all of these properties. One of the exceptions is Du and Tang (2009) which applies certain monotone spline regression procedure to empirical ROC estimates. However, their method does not consider the inherent correlations between empirical ROC estimates. This makes the derivation of the asymptotic properties very difficult. In this paper we propose a penalized weighted least square estimation method, which incorporates the covariance between empirical ROC estimates as a weight matrix. The resulting estimator satisfies all the aforementioned properties, and we show that it is also consistent. Then a resampling approach is used to extend our method for comparisons of two or more diagnostic tests. Our simulations show a significantly improved performance over the existing method, especially for steep ROC curves. We then apply the proposed method to a cancer diagnostic study that compares several newly developed diagnostic biomarkers to a traditional one.     

Combining ROC curves using MAMSE weighted distributions

The receiver operating characteristic (ROC) curve can be used to evaluate the properties of a diagnostic test from the distribution of a variable on the healthy and diseased populations. The minimum averaged mean squared error (MAMSE) weights were developed to handle data from different sources by adjusting the relative contribution of each data source. The authors use the MAMSE weights to infer the ROC curve of a diagnostic test based on raw data from multiple studies. The proposed estimates are consistent and Monte Carlo simulations show favourable finite sample performance. The method is illustrated in a case study where progesterone level is used to detect ectopic pregnancies and abortions from other natural causes. The Canadian Journal of Statistics 46: 298–315; 2018 © 2018 Statistical Society of Canada     

ROC Curves in Non‐Parametric Location‐Scale Regression Models

Abstract. The receiver operating characteristic (ROC) curve is a tool of extensive use to analyse the discrimination capability of a diagnostic variable in medical studies. In certain situations, the presence of a covariate related to the diagnostic variable can increase the discriminating power of the ROC curve. In this article, we model the effect of the covariate over the diagnostic variable by means of non‐parametric location‐scale regression models. We propose a new non‐parametric estimator of the conditional ROC curve and study its asymptotic properties. We also present some simulations and an illustration to a data set concerning diagnosis of diabetes.     

Confidence intervals and bands for the binormal ROC curve revisited

Two types of confidence intervals (CIs) and confidence bands (CBs) for the receiver operating characteristic (ROC) curve are studied: pointwise CIs and simultaneous CBs. An optimized version of the pointwise CI with the shortest width is developed. A new ellipse-envelope simultaneous CB for the ROC curve is suggested as an adaptation of the Working-Hotelling-type CB implemented in a paper by Ma and Hall (1993). Statistical simulations show that our ellipse-envelope CB covers the true ROC curve with a probability close to nominal while the coverage probability of the Ma and Hall CB is significantly smaller. Simulations also show that our CI for the area under the ROC curve is close to nominal while the coverage probability of the CI suggested by Hanley and McNail (1982) uniformly overestimates the nominal value. Two examples illustrate our simultaneous ROC bands: radiation dose estimation from time to vomiting and discrimination of breast cancer from benign abnormalities using electrical impedance measurements.     

Testing upper and lower outlier paris in gamma samples

The problem of discordancy testing for an upper and lower outlier pair in a sample from a gamma distribution with known shape is considered. Three statistics are investigated: the well-known extremal quotient and two likelihood-based procedures. Approximations t o the null distributions of the statistics are obtained where appropriate. The non-null properties are investigated by sensitivity contours and simulation.     

A test for crossing receiver operating characteristic (roc) curves

The receiver operating characteristic (ROC) curve gives a graphical representation of sensitivity and specificity of a prediction model when varying the decision treshold on a diagnostic criterion. A classical test for comparing the overall accuracies for two models -1 and 2- is based on the difference between ROC curves areas - related to its standard error. This test is designed for the situation where ROC curve 1 caps ROC curve 2. Often both curves cross :in this paper, a new test, based on the integrated difference between the curves, is proposed to deal with this situation. In a simulation experiment, the new test was less powerful than the old test for detecting an overall superiority, but much more powerfull against the crossing alternative.     

Adjusting ROC curves for covariates in the presence of verification bias

The ROC (receiver operating characteristic) curve is frequently used for describing effectiveness of a diagnostic marker or test. Classical estimation of the ROC curve uses independent identically distributed samples taken randomly from the healthy and diseased populations. Frequently not all subjects undergo a definitive gold standard assessment of disease status (verification). Estimation of the ROC curve based on data only from subjects with verified disease status may be badly biased (verification bias). In this work we investigate the properties of the doubly robust (DR) method for estimating the ROC curve adjusted for covariates (ROC regression) under verification bias. We develop the estimator's asymptotic distribution and examine its finite sample size properties via a simulation study. We apply this procedure to fingerstick postprandial blood glucose measurement data adjusting for age.     

Small sample power of multivariate nonparametric tests for monotone trend

Bhattacharyya and Kioiz (1966) propose two multivariate nonparametric tests for monotone trend, one involving coordinate-wise Mann statistics and the other, coordinate-wise Spearman statistics. Dietz and Killeen (1981) propose a different test statistic based on coordinate-wise Mann statistics. The Pitman asymptotic relative efficiency of all three tests with respect to a normal theory competitor equals the cube root of the efficiency of a multivariate signed rank test with respect to Hotelling's T2. In this article, the small sample power of the nonparametric tests, the normal theory test, and a Bonferroni approach involving coordinate-wise univariate Mann or Spearman tests is examined in a simulation study. The Mann statistic of Dietz and Killeen and the Spearman statistic of Bhattacharyya and Klotz are found to perform well under both null and alternative hypotheses     

ROC analysis for multiple markers with tree-based classification

Multiple biomarkers are frequently observed or collected for detecting or understanding a disease. The research interest of this article is to extend tools of receiver operating characteristic (ROC) analysis from univariate marker setting to multivariate marker setting for evaluating predictive accuracy of biomarkers using a tree-based classification rule. Using an arbitrarily combined and-or classifier, an ROC function together with a weighted ROC function (WROC) and their conjugate counterparts are introduced for examining the performance of multivariate markers. Specific features of the ROC and WROC functions and other related statistics are discussed in comparison with those familiar properties for univariate marker. Nonparametric methods are developed for estimating the ROC and WROC functions, and area under curve and concordance probability. With emphasis on population average performance of markers, the proposed procedures and inferential results are useful for evaluating marker predictability based on multivariate marker measurements with different choices of markers, and for evaluating different and-or combinations in classifiers.     

Robust confidence regions for multinomial probabilities

Generally, confidence regions for the probabilities of a multinomial population are constructed based on the Pearson χ² statistic. Morales et al. (Bootstrap confidence regions in multinomial sampling. Appl Math Comput. 2004;155:295–315) considered the bootstrap and asymptotic confidence regions based on a broader family of test statistics known as power-divergence test statistics. In this study, we extend their work and propose penalized power-divergence test statistics-based confidence regions. We only consider small sample sizes where asymptotic properties fail and alternative methods are needed. Both bootstrap and asymptotic confidence regions are constructed. We consider the percentile and the bias corrected and accelerated bootstrap confidence regions. The latter confidence region has not been studied previously for the power-divergence statistics much less for the penalized ones. Designed simulation studies are carried out to calculate average coverage probabilities. Mean absolute deviation between actual and nominal coverage probabilities is used to compare the proposed confidence regions.     

On the wald,lagrangian multiplier and likelihood ratio tests when the information matrix is singular

Summary Modified formulas for the Wald and Lagrangian multiplier statistics are introduced and considered together with the likelihood ratio statistics for testing a typical null hypothesisH ₀ stated in terms of equality constraints. It is demonstrated, subject to known standard regularity conditions, that each of these statistics and the known Wald statistic has the asymptotic chi-square distribution with degrees of freedom equal to the number of equality constraints specified byH ₀ whether the information matrix is singular or nonsingular. The results of this paper include a generalization of the results of Sively (1959) concerning the equivalence of the Wald, Lagrange multiplier and likelihood ratio tests to the case of singular information matrices.     

Approximations to the p-values of tests for a change-point under non-standard conditions

Three test statistics for a change-point in a linear model, variants of those considered by Andrews and Ploberger [Optimal tests when a nusiance parameter is present only under the alternative. Econometrica. 1994;62:1383–1414]: the sup-likelihood ratio (LR) statistic; a weighted average of the exponential of LR-statistics and a weighted average of LR-statistics, are studied. Critical values for the statistics with time trend regressors, obtained via simulation, are found to vary considerably, depending on conditions on the error terms. The performance of the bootstrap in approximating p-values of the distributions is assessed in a simulation study. A sample approximation to asymptotic analytical expressions extending those of Kim and Siegmund [The likelihood ratio test for a change-point in simple linear regression. Biometrika. 1989;76:409–423] in the case of the sup-LR test is also assessed. The approximations and bootstrap are applied to the Quandt data [The estimation of a parameter of a linear regression system obeying two separate regimes. J Amer Statist Assoc. 1958;53:873–880] and real data concerning a change-point in oxygen uptake during incremental exercise testing and the bootstrap gives reasonable results.     

Rank statistics expressible as integrals under P–P-plots and receiver operating characteristic curves

Given a probability measure on the unit square, the measure of the region under an empirical P – P -plot defines a two-sample rank statistic. Instances include trimmed and censored versions of the Mann–Whitney–Wilcoxon statistic and a class of statistics with applications in the analysis of receiver operating characteristic (ROC) curves. A large sample distribution for such a statistic is obtained, which is valid under sampling from general populations. Explicit results are presented for comparing arbitrary quantile segments of two populations. The results are not restricted to continuous data and incorporate adjustments for tied values in the discrete case. A multivariate version of the large sample distribution extends the class of tractable statistics in ROC analysis and facilitates the use of methods based on partial areas when the data are discrete.     

Asymptotic relative efficiency of wald tests in measurement error models

In this paper, asymptotic relative efficiency (ARE) of Wald tests for the Tweedie class of models with log-linear mean, is considered when the aux¬iliary variable is measured with error. Wald test statistics based on the naive maximum likelihood estimator and on a consistent estimator which is obtained by using Nakarnura's (1990) corrected score function approach are defined. As shown analytically, the Wald statistics based on the naive and corrected score function estimators are asymptotically equivalents in terms of ARE. On the other hand, the asymptotic relative efficiency of the naive and corrected Wald statistic with respect to the Wald statistic based on the true covariate equals to the square of the correlation between the unobserved and the observed co-variate. A small scale numerical Monte Carlo study and an example illustrate the small sample size situation.     

Identification of the minimum effective dose based on weighted Kaplan–Meier statistics

Testing procedures are considered for identifying the minimum effective dose (MED) in a dose–response study with randomly right-censored survival data, where the MED is defined to be the smallest dose level under study that has survival advantage over the zero dose control. The proposed testing procedures are implemented in a step-down manner together with three different types of weighted Kaplan–Meier statistics. Comparative results of a Monte Carlo error rate and power/bias study for a variety of survival and censoring distributions are then presented and discussed. The application of the proposed procedures is finally illustrated for identifying the MED of the diethylstilbestrol in the treatment of prostate cancer.