Bayesian Treatment of Model Uncertainty for Partially Applicable Models

This article discusses how analyst's or expert's beliefs on the credibility and quality of models can be assessed and incorporated into the uncertainty assessment of an unknown of interest. The proposed methodology is a specialization of the Bayesian framework for the assessment of model uncertainty presented in an earlier paper. This formalism treats models as sources of information in assessing the uncertainty of an unknown, and it allows the use of predictions from multiple models as well as experimental validation data about the models’ performances. In this article, the methodology is extended to incorporate additional types of information about the model, namely, subjective information in terms of credibility of the model and its applicability when it is used outside its intended domain of application. An example in the context of fire risk modeling is also provided.     

The Assessment of Probability Distributions from Expert Opinions with an Application to Seismic Fragility Curves

A method is developed for estimating a probability distribution using estimates of its percentiles provided by experts. The analyst's judgment concerning the credibility of these expert opinions is quantified in the likelihood function of Bayes'Theorem. The model considers explicitly the random variability of each expert estimate, the dependencies among the estimates of each expert, the dependencies among experts, and potential systematic biases. The relation between the results of the formal methods of this paper and methods used in practice is explored. A series of sensitivity studies provides insights into the significance of the parameters of the model. The methodology is applied to the problem of estimation of seismic fragility curves (i.e., the conditional probability of equipment failure given a seismically induced stress).     

Bayesian Methodology for Model Uncertainty Using Model Performance Data

A simple and useful characterization of many predictive models is in terms of model structure and model parameters. Accordingly, uncertainties in model predictions arise from uncertainties in the values assumed by the model parameters (parameter uncertainty) and the uncertainties and errors associated with the structure of the model (model uncertainty). When assessing uncertainty one is interested in identifying, at some level of confidence, the range of possible and then probable values of the unknown of interest. All sources of uncertainty and variability need to be considered. Although parameter uncertainty assessment has been extensively discussed in the literature, model uncertainty is a relatively new topic of discussion by the scientific community, despite being often the major contributor to the overall uncertainty. This article describes a Bayesian methodology for the assessment of model uncertainties, where models are treated as sources of information on the unknown of interest. The general framework is then specialized for the case where models provide point estimates about a single‐valued unknown, and where information about models are available in form of homogeneous and nonhomogeneous performance data (pairs of experimental observations and model predictions). Several example applications for physical models used in fire risk analysis are also provided.     

Vector and recurrent singular spectrum analysis: which is better at forecasting?

Singular spectrum analysis (SSA) is an increasingly popular and widely adopted filtering and forecasting technique which is currently exploited in a variety of fields. Given its increasing application and superior performance in comparison to other methods, it is pertinent to study and distinguish between the two forecasting variations of SSA. These are referred to as Vector SSA (SSA-V) and Recurrent SSA (SSA-R). The general notion is that SSA-V is more robust and provides better forecasts than SSA-R. This is especially true when faced with time series which are non-stationary and asymmetric, or affected by unit root problems, outliers or structural breaks. However, currently there exists no empirical evidence for proving the above notions or suggesting that SSA-V is better than SSA-R. In this paper, we evaluate out-of-sample forecasting capabilities of the optimised SSA-V and SSA-R forecasting algorithms via a simulation study and an application to 100 real data sets with varying structures, to provide a statistically reliable answer to the question of which SSA algorithm is best for forecasting at both short and long run horizons based on several important criteria.     

Discourses about children's mental health and developmental disorders in North American women's magazines 1990–2012

This paper describes and analyses the portrayal of children's mental health and developmental issues (CMHI) in articles located in a random sample of a selection of available high‐circulating English language North American ‘women's magazines’ indexed in the Reader's Guide to Periodical Literature from 1990 to 2012. It is based upon a qualitative discourse analysis. CMHI are portrayed as materially, biologically real, prevalent and growing in incidence, and severity. They are also portrayed as located in the ‘non‐normal’, ‘non‐nice’, ‘disliked’ and ‘non‐successful’ individual child. Neither the facticity nor the biomedicalization of CMHI is questioned. The psy‐scientists and practitioners cited as experts for the ‘disorders’ offer contradictory and confusing information and advice. The possible theoretical and pragmatic explanations and consequences of this portrayal are discussed.     

A Hybrid Methodology for Modeling Risk of Adverse Events in Complex Health‐Care Settings

In spite of increased attention to quality and efforts to provide safe medical care, adverse events (AEs) are still frequent in clinical practice. Reports from various sources indicate that a substantial number of hospitalized patients suffer treatment‐caused injuries while in the hospital. While risk cannot be entirely eliminated from health‐care activities, an important goal is to develop effective and durable mitigation strategies to render the system “safer.” In order to do this, though, we must develop models that comprehensively and realistically characterize the risk. In the health‐care domain, this can be extremely challenging due to the wide variability in the way that health‐care processes and interventions are executed and also due to the dynamic nature of risk in this particular domain. In this study, we have developed a generic methodology for evaluating dynamic changes in AE risk in acute care hospitals as a function of organizational and nonorganizational factors, using a combination of modeling formalisms. First, a system dynamics (SD) framework is used to demonstrate how organizational‐level and policy‐level contributions to risk evolve over time, and how policies and decisions may affect the general system‐level contribution to AE risk. It also captures the feedback of organizational factors and decisions over time and the nonlinearities in these feedback effects. SD is a popular approach to understanding the behavior of complex social and economic systems. It is a simulation‐based, differential equation modeling tool that is widely used in situations where the formal model is complex and an analytical solution is very difficult to obtain. Second, a Bayesian belief network (BBN) framework is used to represent patient‐level factors and also physician‐level decisions and factors in the management of an individual patient, which contribute to the risk of hospital‐acquired AE. BBNs are networks of probabilities that can capture probabilistic relations between variables and contain historical information about their relationship, and are powerful tools for modeling causes and effects in many domains. The model is intended to support hospital decisions with regard to staffing, length of stay, and investments in safety, which evolve dynamically over time. The methodology has been applied in modeling the two types of common AEs: pressure ulcers and vascular‐catheter‐associated infection, and the models have been validated with eight years of clinical data and use of expert opinion.     

On Decomposition and Aggregation Error in Estimation: Some Basic Principles and Examples

A common strategy in estimation is to decompose the quantity being estimated into several factors, estimate a value for each factor, and then reaggregate these values to obtain an overall estimate. Until now, the extent of decomposition to use has been considered largely a matter of judgment. However, both the extent of decomposition and the manner in which the results are reaggregated can have a significant effect on the resulting estimate. This paper discusses factors affecting the optimum level of decomposition, and presents examples showing that the use of nonoptimal decomposition strategies can lead to significant inaccuracies.     

Improving Default Risk Prediction Using Bayesian Model Uncertainty Techniques

Credit risk is the potential exposure of a creditor to an obligor's failure or refusal to repay the debt in principal or interest. The potential of exposure is measured in terms of probability of default. Many models have been developed to estimate credit risk, with rating agencies dating back to the 19th century. They provide their assessment of probability of default and transition probabilities of various firms in their annual reports. Regulatory capital requirements for credit risk outlined by the Basel Committee on Banking Supervision have made it essential for banks and financial institutions to develop sophisticated models in an attempt to measure credit risk with higher accuracy. The Bayesian framework proposed in this article uses the techniques developed in physical sciences and engineering for dealing with model uncertainty and expert accuracy to obtain improved estimates of credit risk and associated uncertainties. The approach uses estimates from one or more rating agencies and incorporates their historical accuracy (past performance data) in estimating future default risk and transition probabilities. Several examples demonstrate that the proposed methodology can assess default probability with accuracy exceeding the estimations of all the individual models. Moreover, the methodology accounts for potentially significant departures from “nominal predictions” due to “upsetting events” such as the 2008 global banking crisis.