Exact confidence interval estimation for the difference in diagnostic accuracy with three ordinal diagnostic groups

In the cases with three ordinal diagnostic groups, the important measures of diagnostic accuracy are the volume under surface (VUS) and the partial volume under surface (PVUS) which are the extended forms of the area under curve (AUC) and the partial area under curve (PAUC). This article addresses confidence interval estimation of the difference in paired VUS s and the difference in paired PVUS s. To focus especially on studies with small to moderate sample sizes, we propose an approach based on the concepts of generalized inference. A Monte Carlo study demonstrates that the proposed approach generally can provide confidence intervals with reasonable coverage probabilities even at small sample sizes. The proposed approach is compared to a parametric bootstrap approach and a large sample approach through simulation. Finally, the proposed approach is illustrated via an application to a data set of blood test results of anemia patients.     

Generalized Confidence Interval Estimation for the Difference in Paired Areas Under the ROC Curves in the Absence of a Gold Standard

Receiver operating characteristic (ROC) curves can be used to assess the accuracy of tests measured on ordinal or continuous scales. The most commonly used measure for the overall diagnostic accuracy of diagnostic tests is the area under the ROC curve (AUC). A gold standard (GS) test on the true disease status is required to estimate the AUC. However, a GS test may be too expensive or infeasible. In many medical researches, the true disease status of the subjects may remain unknown. Under the normality assumption on test results from each disease group of subjects, we propose a heuristic method of estimating confidence intervals for the difference in paired AUCs of two diagnostic tests in the absence of a GS reference. This heuristic method is a three-stage method by combining the expectation-maximization (EM) algorithm, bootstrap method, and an estimation based on asymptotic generalized pivotal quantities (GPQs) to construct generalized confidence intervals for the difference in paired AUCs in the absence of a GS. Simulation results show that the proposed interval estimation procedure yields satisfactory coverage probabilities and expected interval lengths. The numerical example using a published dataset illustrates the proposed method.     

Generalized inferences on the common mean vector of several multivariate normal populations

The hypothesis testing and confidence region are considered for the common mean vector of several multivariate normal populations when the covariance matrices are unknown and possibly unequal. A generalized confidence region is derived using the concepts of generalized method based on the generalized p$p$-value. The generalized confidence region is illustrated with two numerical examples. The merits of the proposed method are numerically compared with those of existing methods with respect to their expected area or expected d-dimensional volumes and coverage probabilities under different scenarios.     

Simultaneous Small Sample Inference for Linear Combinations of Generalized Linear Model Parameters

A method is proposed to construct simultaneous confidence intervals for multiple linear combinations of generalized linear model parameters, that uses a multivariate normal- or t-distribution together with the signed likelihood root statistic. In an application to a case study simultaneous confidence bands for logistic regression are calculated. A simulation study based on the example evaluation suggests superior performance compared to the common Wald-type approaches. The proposed methods are readily implemented in the R extension package mcprofile.     

Approximating the risk score for disease diagnosis using MARS

In disease screening and diagnosis, often multiple markers are measured and combined to improve the accuracy of diagnosis. McIntosh and Pepe [Combining several screening tests: optimality of the risk score, Biometrics 58 (2002), pp. 657–664] showed that the risk score, defined as the probability of disease conditional on multiple markers, is the optimal function for classification based on the Neyman–Pearson lemma. They proposed a two-step procedure to approximate the risk score. However, the resulting receiver operating characteristic (ROC) curve is only defined in a subrange (L, h) of false-positive rates in (0,1) and the determination of the lower limit L needs extra prior information. In practice, most diagnostic tests are not perfect, and it is usually rare that a single marker is uniformly better than the other tests. Using simulation, I show that multivariate adaptive regression spline is a useful tool to approximate the risk score when combining multiple markers, especially when ROC curves from multiple tests cross. The resulting ROC is defined in the whole range of (0,1) and is easy to implement and has intuitive interpretation. The sample code of the application is shown in the appendix.     

Assessing Occupational Exposure via the Unbalanced One-Way Random Effects Model

In this article, an unbalanced one-way random effects model is considered for the log-transformed shift-long exposure measurements. Exact test and confidence interval for the proportion of workers whose mean exposure exceeds the occupational exposure limit are developed based on the concepts of generalized p-value and generalized confidence interval. Some simulation results to compare the performance of the proposed test with that of the existing method are reported. The simulation results indicate that the proposed method appears to have significant gain in the size and power.     

Detecting diagnostic accuracy of two biomarkers through a bivariate log-normal ROC curve

In biomedical research, two or more biomarkers may be available for diagnosis of a particular disease. Selecting one single biomarker which ideally discriminate a diseased group from a healthy group is confront in a diagnostic process. Frequently, most of the people use the accuracy measure, area under the receiver operating characteristic (ROC) curve to choose the best diagnostic marker among the available markers for diagnosis. Some authors have tried to combine the multiple markers by an optimal linear combination to increase the discriminatory power. In this paper, we propose an alternative method that combines two continuous biomarkers by direct bivariate modeling of the ROC curve under log-normality assumption. The proposed method is applied to simulated data set and prostate cancer diagnostic biomarker data set.     

Combination of multiple functional markers to improve diagnostic accuracy

Combination of multiple biomarkers to improve diagnostic accuracy is meaningful for practitioners and clinicians, and are attractive to lots of researchers. Nowadays, with development of modern techniques, functional markers such as curves or images, play an important role in diagnosis. There exists rich literature developing combination methods for continuous scalar markers. Unfortunately, only sporadic works have studied how functional markers affect diagnosis in the literature. Moreover, no publication can be found to do combination of multiple functional markers to improve the diagnostic accuracy. It is impossible to apply scalar combination methods to the multiple functional markers directly because of infinite dimensionality of functional markers. In this article, we propose a one-dimension scalar feature motivated by square loss distance, as an alternative of the original functional curve in the sense that, it can retain information to the most extent. The square loss distance is defined as the function of projection scores generated from functional principal component decomposition. Then existing variety of scalar combination methods can be applied to scalar features of functional markers after dimension reduction to improve the diagnostic accuracy. Area under the receiver operating characteristic curve and Youden index are used to assess performances of various methods in numerical studies. We also analyzed the high- or low- hospital admissions due to respiratory diseases between 2010 and 2017 in Hong Kong by combining weather conditions and media information, which are regarded as functional markers. Finally, we provide an R function for convenient application.     

Inferences on the Among-Group Variance Component in Unbalanced Heteroscedastic One-Fold Nested Design

This article studies the hypothesis testing and interval estimation for the among-group variance component in unbalanced heteroscedastic one-fold nested design. Based on the concepts of generalized p-value and generalized confidence interval, tests and confidence intervals for the among-group variance component are developed. Furthermore, some simulation results are presented to compare the performance of the proposed approach with those of existing approaches. It is found that the proposed approach and one of the existing approaches can maintain the nominal confidence level across a wide array of scenarios, and therefore are recommended to use in practical problems. Finally, a real example is illustrated.     

COMBINING MULTIPLE MARKERS FOR MULTI‐CATEGORY CLASSIFICATION: AN ROC SURFACE APPROACH

The Receiver Operating Characteristic (ROC) curve and the Area Under the ROC Curve (AUC) are effective statistical tools for evaluating the accuracy of diagnostic tests for binary‐class medical data. However, many real‐world biomedical problems involve more than two categories. The Volume Under the ROC Surface (VUS) and Hypervolume Under the ROC Manifold (HUM) measures are extensions for the AUC under three‐class and multiple‐class models. Inference methods for such measures have been proposed recently. We develop a method of constructing a linear combination of markers for which the VUS or HUM of the combined markers is maximized. Asymptotic validity of the estimator is justified by extending the results for maximum rank correlation estimation that are well known in econometrics. A bootstrap resampling method is then applied to estimate the sampling variability. Simulations and examples are provided to demonstrate our methods.     

Reliable confidence intervals for assessing normal process incapability

This study aims to provide a reliable confidence interval for assessing the process incapability index [C_pp]. The concept of the generalized pivotal quantities is utilized for constructing the generalized confidence interval for [C_pp]. And, simulations are performed for demonstrating our proposed method and one existent method. The results show that the empirical confidences of these two methods are significantly affected by the degree of process departure. Therefore, we suggest the practitioners to select proper one for capability testing purpose based on the information of degree of process departure.     

Subgroup specific incremental value of new markers for risk prediction

In many clinical applications, understanding when measurement of new markers is necessary to provide added accuracy to existing prediction tools could lead to more cost effective disease management. Many statistical tools for evaluating the incremental value (IncV) of the novel markers over the routine clinical risk factors have been developed in recent years. However, most existing literature focuses primarily on global assessment. Since the IncVs of new markers often vary across subgroups, it would be of great interest to identify subgroups for which the new markers are most/least useful in improving risk prediction. In this paper we provide novel statistical procedures for systematically identifying potential traditional-marker based subgroups in whom it might be beneficial to apply a new model with measurements of both the novel and traditional markers. We consider various conditional time-dependent accuracy parameters for censored failure time outcome to assess the subgroup-specific IncVs. We provide non-parametric kernel-based estimation procedures to calculate the proposed parameters. Simultaneous interval estimation procedures are provided to account for sampling variation and adjust for multiple testing. Simulation studies suggest that our proposed procedures work well in finite samples. The proposed procedures are applied to the Framingham Offspring Study to examine the added value of an inflammation marker, C-reactive protein, on top of the traditional Framingham risk score for predicting 10-year risk of cardiovascular disease.     

Confidence intervals for a two-parameter exponential distribution: One- and two-sample problems

The problems of interval estimating the mean, quantiles, and survival probability in a two-parameter exponential distribution are addressed. Distribution function of a pivotal quantity whose percentiles can be used to construct confidence limits for the mean and quantiles is derived. A simple approximate method of finding confidence intervals for the difference between two means and for the difference between two location parameters is also proposed. Monte Carlo evaluation studies indicate that the approximate confidence intervals are accurate even for small samples. The methods are illustrated using two examples.     

Inferences on the Coefficients of Variation in a Multivariate Normal Population

In this article, the problem of testing the equality of coefficients of variation in a multivariate normal population is considered, and an asymptotic approach and a generalized p-value approach based on the concepts of generalized test variable are proposed. Monte Carlo simulation studies show that the proposed generalized p-value test has good empirical sizes, and it is better than the asymptotic approach. In addition, the problem of hypothesis testing and confidence interval for the common coefficient variation of a multivariate normal population are considered, and a generalized p-value and a generalized confidence interval are proposed. Using Monte Carlo simulation, we find that the coverage probabilities and expected lengths of this generalized confidence interval are satisfactory, and the empirical sizes of the generalized p-value are close to nominal level. We illustrate our approaches using a real data.     

Assessing case influence on confidence intervals in nonlinear regression

Diagnostic techniques are proposed for assessing the influence of individual cases on confidence intervals in nonlinear regression. The technique proposed uses the method of profile t-plots applied to the case-deletion model. The effect of the geometry of the statistical model on the influence measures is assessed, and an algorithm for computing case-deleted confidence intervals is described. This algorithm provides a direct method for constructing a simple diagnostic measure based on the ratio of the lengths of confidence intervals. The generalization of these methods to multiresponse models is discussed.     

Joint confidence region estimation on predictive values

For evaluating diagnostic accuracy of inherently continuous diagnostic tests/biomarkers, sensitivity and specificity are well-known measures both of which depend on a diagnostic cut-off, which is usually estimated. Sensitivity (specificity) is the conditional probability of testing positive (negative) given the true disease status. However, a more relevant question is “what is the probability of having (not having) a disease if a test is positive (negative)?”. Such post-test probabilities are denoted as positive predictive value (PPV) and negative predictive value (NPV). The PPV and NPV at the same estimated cut-off are correlated, hence it is desirable to make the joint inference on PPV and NPV to account for such correlation. Existing inference methods for PPV and NPV focus on the individual confidence intervals and they were developed under binomial distribution assuming binary instead of continuous test results. Several approaches are proposed to estimate the joint confidence region as well as the individual confidence intervals of PPV and NPV. Simulation results indicate the proposed approaches perform well with satisfactory coverage probabilities for normal and non-normal data and, additionally, outperform existing methods with improved coverage as well as narrower confidence intervals for PPV and NPV. The Alzheimer's Disease Neuroimaging Initiative (ADNI) data set is used to illustrate the proposed approaches and compare them with the existing methods.     

Evaluation of incomplete multiple diagnostic tests,with an application in the colon cancer family registry study

Accurate diagnosis of a molecularly defined subtype of cancer is often an important step toward its effective control and treatment. For the diagnosis of some subtypes of a cancer, a gold standard with perfect sensitivity and specificity may be unavailable. In those scenarios, tumor subtype status is commonly measured by multiple imperfect diagnostic markers. Additionally, in many such studies, some subjects are only measured by a subset of diagnostic tests and the missing probabilities may depend on the unknown disease status. In this paper, we present statistical methods based on the EM algorithm to evaluate incomplete multiple imperfect diagnostic tests under a missing at random assumption and one missing not at random scenario. We apply the proposed methods to a real data set from the National Cancer Institute (NCI) colon cancer family registry on diagnosing microsatellite instability for hereditary non-polyposis colorectal cancer to estimate diagnostic accuracy parameters (i.e. sensitivities and specificities), prevalence, and potential differential missing probabilities for 11 biomarker tests. Simulations are also conducted to evaluate the small-sample performance of our methods.     

Performance of Confidence Intervals in Regression Models with Unbalanced One-Fold Nested Error Structures

Abstract

In this article we consider the problem of constructing confidence intervals for a linear regression model with unbalanced nested error structure. A popular approach is the likelihood-based method employed by PROC MIXED of SAS. In this article, we examine the ability of MIXED to produce confidence intervals that maintain the stated confidence coefficient. Our results suggest that intervals for the regression coefficients work well, but intervals for the variance component associated with the primary level cannot be recommended. Accordingly, we propose alternative methods for constructing confidence intervals on the primary level variance component. Computer simulation is used to compare the proposed methods. A numerical example and SAS code are provided to demonstrate the methods.     

A statistical method for synthesizing mediation analyses using the product of coefficient approach across multiple trials

Mediation analysis often requires larger sample sizes than main effect analysis to achieve the same statistical power. Combining results across similar trials may be the only practical option for increasing statistical power for mediation analysis in some situations. In this paper, we propose a method to estimate: (1) marginal means for mediation path a, the relation of the independent variable to the mediator; (2) marginal means for path b, the relation of the mediator to the outcome, across multiple trials; and (3) the between-trial level variance–covariance matrix based on a bivariate normal distribution. We present the statistical theory and an R computer program to combine regression coefficients from multiple trials to estimate a combined mediated effect and confidence interval under a random effects model. Values of coefficients a and b, along with their standard errors from each trial are the input for the method. This marginal likelihood based approach with Monte Carlo confidence intervals provides more accurate inference than the standard meta-analytic approach. We discuss computational issues, apply the method to two real-data examples and make recommendations for the use of the method in different settings.     

A ray method of confidence band construction for multiple linear regression models

This paper addresses the problem of confidence band construction for a standard multiple linear regression model. A “ray” method of construction is developed which generalizes the method of Graybill and Bowden [1967. Linear segment confidence bands for simple linear regression models. J. Amer. Statist. Assoc. 62, 403–408] for a simple linear regression model to a multiple linear regression model. By choosing suitable directions for the rays this method requires only critical points from t-distributions so that the confidence bands are easy to construct. Both one-sided and two-sided confidence bands can be constructed using this method. An illustration of the new method is provided.