Understanding increments in model performance metrics

The area under the receiver operating characteristic curve (AUC) is the most commonly reported measure of discrimination for prediction models with binary outcomes. However, recently it has been criticized for its inability to increase when important risk factors are added to a baseline model with good discrimination. This has led to the claim that the reliance on the AUC as a measure of discrimination may miss important improvements in clinical performance of risk prediction rules derived from a baseline model. In this paper we investigate this claim by relating the AUC to measures of clinical performance based on sensitivity and specificity under the assumption of multivariate normality. The behavior of the AUC is contrasted with that of discrimination slope. We show that unless rules with very good specificity are desired, the change in the AUC does an adequate job as a predictor of the change in measures of clinical performance. However, stronger or more numerous predictors are needed to achieve the same increment in the AUC for baseline models with good versus poor discrimination. When excellent specificity is desired, our results suggest that the discrimination slope might be a better measure of model improvement than AUC. The theoretical results are illustrated using a Framingham Heart Study example of a model for predicting the 10-year incidence of atrial fibrillation.     

Tests of hypotheses based on ranks in the general linear model

A unified approach is developed for testing hypotheses in the general linear model based on the ranks of the residuals. It complements the nonparametric estimation procedures recently reported in the literature. The testing and estimation procedures together provide a robust alternative to least squares. The methods are similar in spirit to least squares so that results are simple to interpret. Hypotheses concerning a subset of specified parameters can be tested, while the remaining parameters are treated as nuisance parameters. Asymptotically, the test statistic is shown to have a chi-square distribution under the null hypothesis. This result is then extended to cover a sequence of contiguous alternatives from which the Pitman efficacy is derived. The general application of the test requires the consistent estimation of a functional of the underlying distribution and one such estimate is furnished.     

A Survey of the Teaching of Statistics in M.B.A. Programs

A survey of business schools was conducted to obtain information about the current state of the teaching of business statistics to students enrolled in M.B.A. degree programs. The survey was undertaken for and presented at a June 1986 conference on “Making Statistics More Effective in Schools of Business,” held at the University of Chicago's Graduate School of Business. Information was elicited concerning both the required statistics sequence and elective statistics courses for M.B.A. students, as well as computer usage in these courses. This article summarizes the information obtained from the survey.     

Developing “A Framework for Planning U.S. Federal Statistics, 1978–1989”

The Statistical Policy Division of the Office of Management and Budget has the overall responsibility for the planning and coordination of U.S. government statistics. The present staff of the Statistical Policy Division is attempting, through an integrated publication entitled “A Framework for Planning U.S. Federal Statistics, 1978–1989,” to state its perspective on necessary developments in the coming years. This article illustrates the character of the Framework materials and outlines the process for public review and comment on this undertaking.     

Generalized Linear Models

The importance of random number generators has increased over the years. This follows from the fact that contemporary research methods rely more and more on simulation and the increased importance of encryption technology. The output of a random number generator is created by either an algorithm or a physical device. The most popular method for random number generation is through the use of an algorithm. This article presents a new category of physical random bit generator that is packaged by several manufacturers. A statistical analysis of the output from the generators is given.     

On constructing T 2 control charts for retrospective examination

In this paper we consider the issue of constructing retrospective T ² control chart limits so as to control the overall probability of a false alarm at a specified value. We describe an exact method for constructing the control limits for retrospective examination. We then consider Bonferroni-adjustments to Alt's control limit and to the standard x ² control limit as alternatives to the exact limit since it is computationally cumbersome to find the exact limit. We present the results of some simulation experiments that are carried out to compare the performance of these control limits. The results indicate that the Bonferroni-adjusted Alt's control limit performs better that the Bonferroni-adjusted x ² control limit. Furthermore, it appears that the Bonferroni-adjusted Alt's control limit is more than adequate for controlling the overall false alarm probability at a specified value.     

Age-period-cohort analysis: an illustration of the problems in assessing interaction in one observation per cell data

This paper discusses the specific problems of age-period-cohort (A-P-C) analysis within the general framework of interaction assessment for two-way cross-classified data with one observation per cell. The A-P-C multiple classification model containing the effects of age groups (rows), periods of observation (columns), and birth cohorts (diagonals of the two-way table) is characterized as one of a special class of models involving interaction terms assumed to have very specific forms. The so-called A-P-C identification problem, which results from the use of a particular interaction structure for detecting cohort effects, is shown to manifest itself in the form of an exact linear dependency among the columns of the design matrix. The precise relationship holding among these columns is derived, as is an explicit formula for the bias in the parameter estimates resulting from an incorrect specification of an assumed restriction on the parameters required to solve the normal equations. Current methods for modeling A-P-C data are critically reviewed, an illustrative numerical example is presented, and one potentially promising analysis strategy is discussed. However, gien the large number of possible sources for error in A-P-C analyses, it is strongly recommended that the results of such analyses be interpreted with a great deal of caution.     

Testing Symmetry of the Error Distribution in Nonlinear Heteroscedastic Models

Using the empirical characteristic function, we derive a Cramér-von Mises test for symmetry of the error distribution in a class of nonlinear parametric heteroscedastic models. We study the convergence of the residual-based empirical distribution function. We establish a functional limit theorem for an empirical process of residuals, and investigate the asymptotic null distribution function of our test statistic. A simulation experiment is conducted to evaluate small-sample performances of our test.     

A survey of statistical methods on remote sensing in russian literature from 1976 to 1985

In this article we review the major areas of remote sensing in the Russian literature for the period 1976 to 1985 that use statistical methods to analyze the observed data. For each of the areas, the problems that have been studied and the statistical techniques that have been used are briefly described