Evaluating Probabilistic Forecasts with Bayesian Signal Detection Models

We propose the use of signal detection theory (SDT) to evaluate the performance of both probabilistic forecasting systems and individual forecasters. The main advantage of SDT is that it provides a principled way to distinguish the response from system diagnosticity, which is defined as the ability to distinguish events that occur from those that do not. There are two challenges in applying SDT to probabilistic forecasts. First, the SDT model must handle judged probabilities rather than the conventional binary decisions. Second, the model must be able to operate in the presence of sparse data generated within the context of human forecasting systems. Our approach is to specify a model of how individual forecasts are generated from underlying representations and use Bayesian inference to estimate the underlying latent parameters. Given our estimate of the underlying representations, features of the classic SDT model, such as the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC), follow immediately. We show how our approach allows ROC curves and AUCs to be applied to individuals within a group of forecasters, estimated as a function of time, and extended to measure differences in forecastability across different domains. Among the advantages of this method is that it depends only on the ordinal properties of the probabilistic forecasts. We conclude with a brief discussion of how this approach might facilitate decision making.     

On the robustness of the extreme deviate test for a single multivariate outlier against heavy-tailed distributions

In this paper we assess the sensitivity of the multivariate extreme deviate test for a single multivariate outlier to non-normality in the form of heavy tails. We find that the empirical significance levels can be markedly affected by even modest departures from multivariate normality. The effects are particularly severe when the sample size is large relative to the dimension. Finally, by way of example we demonstrate that certain graphical techniques may prove useful in identifying the source of rejection for the multivariate extreme deviate test.     

South Korean College Students' Holland Types and Career Compromise Processes

This study explored South Korean college students' career compromise processes and examined whether preferences for sex type, prestige, or interests would be differentiated by Holland theme interest types, gender, or college major. Participants were South Korean undergraduate students from 2 universities in Seoul, South Korea. They were asked to choose 1 occupation from each of 168 pairs of occupations using a forced‐choice format. A total of 376 surveys were analyzed. There were significant main effects for Holland interest types and for gender but not for college major on their career compromise processes. Implications for career counselors and researchers are discussed.     

Causes of Disaster: Sloppy Management

The loss of control associated with accidents and disasters can have severe negative consequences for the organizations involved. Some disasters are caused by inadequacies of management, or by unprofessional behaviour but others arise when disaster preconditions are generated as a result of the normal functioning of larger managerial and technical systems. During the incubation period preceding a major incident, therefore, two levels of correction need to be considered to avoid disasters. ‘Sloppy management’ of various kinds can be tackled by establishing, strengthening and asserting appropriate forms of management control. The system properties which constitute disaster preconditions are less immediately accessible to management control. At both levels managers in pursuit of high reliability should expect to have to make a range of trade-offs.