The Validation of Screening Tests: Meet the New Screen same as the Old Screen?

The focus of this report is to examine the process of validation of new screening tests designed to detect the problem gambler in research and practice settings. A hierarchical or phases of evaluation model is presented as a conceptual framework to describe the basic features of the validation process and its implications for application and interpretation of test results. The report describes a number of threats to validity in the form of sources of unintended bias that when unrecognized may lead to incorrect interpretations of study results and the drawing of incorrect conclusions about the usefulness of the new screening tests. Examples drawn from the gambling literature on problem gambling are used to illustrate some of the more important concepts including spectrum bias and clinical variation in test accuracy. The concept of zones of severity and the bias inherent in selecting criterion thresholds are reviewed. A definition of reference or study gold standard is provided. The use of 2-stage designs to establish validity by efficiently using reference standards to determine indices of accuracy and prevalence is recommended.     

An Epidemiologic Note on Verification Bias: Implications for Estimation of Rates

Two-stage designs offer several advantages for purposes of test validation and prevalence estimation. These include enhanced precision and increased cost-efficiency. Cost efficiency is obtained when the best available verification criterion is too expensive to employ in a large-scale epidemiologic study. The use of two-stage (or double-sampling) designs permit the same inferences from application to only a subset of those who were screened (with the less expensive criterion) during stage one. The retesting of only some of the first stage respondents introduces a bias, however, if these are sampled on the basis of first stage screening results. The form of this bias is described and solutions for correcting estimates are provided. These solutions are applied to the data reported by Abbott and Volberg (1996) in their study of the New Zealand general population. Corrected estimates of the sensitivity and specificity of the South Oaks Gambling Screen are obtained and used to adjust the reported lifetime and current prevalence estimates. The value of multi-stage designs for validity assessment and prevalence estimation are briefly described.     

Reflections on Accuracy

The difference between test accuracy and predictive accuracy is presented and defined. The failure to distinguish between these two types of measures is shown to have led to a misguided debate over the interpretation of prevalence estimates. The distinction between test accuracy defined as sensitivity and specificity, and predictive accuracy defined as positive and negative predictive value is shown to reflect the choice of the denominator used to calculate true positive, false positive, false negative, and true negative rates. It is further shown that any instrument will tend to overestimate prevalence in low base rate populations and underestimate it in those populations where prevalence is high. The implications of these observations are then discussed in terms of the need to define diagnostic thresholds that have clinical and policy relevance.     

Perceptions by treatment staff of critical tasks in the treatment of the compulsive gambler

Seventy-five clinicians who treat compulsive gamblers were surveyed. Each rated 89 clinical tasks and responsibilities for importance in the treatment of this population. Analysis of those items for which a plurality of clinicians rated the item as critical was chosen as the criteria of importance. A principal components analysis was conducted to determine the underlying structure of clinical perceptions of importance. An eight-dimensional model was found to describe perceptions in the most satisfactory way. The analysis revealed five major and three minor clusters of tasks and responsibilities. The major dimensions were labeled as (1) self-help/social support, (2) crisis interventions, (3) behavioral resources for change, (4) psychodynamics of treatment, and (5) crisis severity. The minor dimensions were (6) knowledge and training, (7) ethics and sensitivity to needs, and (8) confidentiality and regulations. A brief discussion of the implications are presented.This research was supported in part by a grant from the Massachusetts Department of Public Health.     

Perceived family history of problem gambling and scores on SOGS

A sample of 93 veterans (92.4% males), with a median age of 41, (Mean=43.5) attending clinics for problem drinking, drug abuse and other mental disorders was screened for problems associated with the diagnosis of pathological gambling. The diagnostic instrument employed was the South Oaks Gambling Screen developed by Lesieur and Blume. The data replicate earlier findings indicating a link between parental problem gambling and pathological gambling. The results extended this association to include grandparents thus firming the familial relationship. Several epidemiological measures were defined and illustrated. These included relative risk, the odds ratio, attributable risk percent and population attributable risk percent. The data were consistent with previous research that substance abusers are about six times as likely to be addicted to gambling as the general population.This research was supported in part by a grant from the Massachusetts Department of Public Health.     

A Comment on the Utility of Prevalence Estimates of Pathological Gambling

The case is presented that researchers interested in policy aimed at treating the pathological gambler need to shift focus to improving the utility of prevalence estimates. It is argued that researchers supplement prevalence estimates with practical and well-defined measures of severity and other predictors and correlates of help-seeking. The dimension of severity is emphasized as one means of providing estimates that are relevant to policy makers when placed in the context of additional measures that improve their meaning and utility. Estimates may then be partitioned along these dimensions to ascertain the proportion of gamblers most likely to need or seek treatment for gambling-related disorders. The recommendations provided are subject to a number of possible objections and are presented in the interest of stimulating further discussion such as the distinction between symptom assessment and the measurement of severity.     

Estimating Confidence Intervals and Sampling Proportions in Two-Stage Prevalence Designs

In response to Abbott and Volberg's (in press) rejoinder to my epidemiologic note on verification bias and estimation of prevalence rates (Gambino, in press), I provide the formulas for computing confidence intervals for the results of second-stage verification. In addition, I provide the appropriate equation for determining confidence intervals when prevalence is near zero or one. Finally, we present formulas for determining the most efficient sample sizes needed to minimize second-stage variance estimates. These allow the investigator working under a fixed budget to determine the relative value of sampling negative screens to test for false negatives. We close with an observation on the interpretability of evidence.     

Test Performance Variation Between Settings and Populations

The present paper is the third in a series on the evaluation of new tests designed to detect the disordered gambler. The present paper has two objectives. First, the observed variation in test performance between settings and populations is described in general terms and an explanation of the observed variation is presented in terms of what is referred to as spectrum effects. Second the expected variation in test performance between settings and populations is illustrated with several examples and the implications emphasized for the purpose of test evaluation.     

A bayesian approach to prediction in analysis of variance settings

This paper takes the results of Lindley and smith ( 1972 ) one step further, by finding the predictive distribution of an observation y^* whose distribution is normal, and centred at A^* ₁θ₁ We then apply this distribution to the case of prediction based on data obtained in one and two wau ANOVA situations. For Example, it turns out that for two way ANOVA with interaction, the predictive mean, (which we would use as the predictor) is a weighted combination of sample main effects and interaction effects