Combining Quantitative and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System

This article discusses recent experiences with the Numeral Unit Spread Assessment Pedigree (NUSAP) system for multidimensional uncertainty assessment, based on four case studies that vary in complexity. We show that the NUSAP method is applicable not only to relatively simple calculation schemes but also to complex models in a meaningful way and that NUSAP is useful to assess not only parameter uncertainty but also (model) assumptions. A diagnostic diagram can be used to synthesize results of quantitative analysis of parameter sensitivity and qualitative review (pedigree analysis) of parameter strength. It provides an analytic tool to prioritize uncertainties according to quantitative and qualitative insights in the limitations of available knowledge. We show that extension of the pedigree scheme to include societal dimensions of uncertainty, such as problem framing and value-laden assumptions, further promotes reflexivity and collective learning. When used in a deliberative setting, NUSAP pedigree assessment has the potential to foster a deeper social debate and a negotiated management of complex environmental problems.     

Uncertainty of Population Risk Estimates for Pathogens Based on QMRA or Epidemiology: A Case Study of Campylobacter in the Netherlands

Epidemiology and quantitative microbiological risk assessment are disciplines in which the same public health measures are estimated, but results differ frequently. If large, these differences can confuse public health policymakers. This article aims to identify uncertainty sources that explain apparent differences in estimates for Campylobacter spp. incidence and attribution in the Netherlands, based on four previous studies (two for each discipline). An uncertainty typology was used to identify uncertainty sources and the NUSAP method was applied to characterize the uncertainty and its influence on estimates. Model outcomes were subsequently calculated for alternative scenarios that simulated very different but realistic alternatives in parameter estimates, modeling, data handling, or analysis to obtain impressions of the total uncertainty. For the epidemiological assessment, 32 uncertainty sources were identified and for QMRA 67. Definitions (e.g., of a case) and study boundaries (e.g., of the studied pathogen) were identified as important drivers for the differences between the estimates of the original studies. The range in alternatively calculated estimates usually overlapped between disciplines, showing that proper appreciation of uncertainty can explain apparent differences between the initial estimates from both disciplines. Uncertainty was not estimated in the original QMRA studies and underestimated in the epidemiological studies. We advise to give appropriate attention to uncertainty in QMRA and epidemiological studies, even if only qualitatively, so that scientists and policymakers can interpret reported outcomes more correctly. Ideally, both disciplines are joined by merging their strong respective properties, leading to unified public health measures.     

Development of a Quantitative Microbial Risk Assessment for Human Salmonellosis Through Household Consumption of Fresh Minced Pork Meat in Belgium

A quantitative microbial risk assessment (QMRA) according to the Codex Alimentarius Principles is conducted to evaluate the risk of human salmonellosis through household consumption of fresh minced pork meat in Belgium. The quantitative exposure assessment is carried out by building a modular risk model, called the METZOON-model, which covers the pork production from farm to fork. In the METZOON-model, the food production pathway is split up in six consecutive modules: (1) primary production, (2) transport and lairage, (3) slaughterhouse, (4) postprocessing, (5) distribution and storage, and (6) preparation and consumption. All the modules are developed to resemble as closely as possible the Belgian situation, making use of the available national data. Several statistical refinements and improved modeling techniques are proposed. The model produces highly realistic results. The baseline predicted number of annual salmonellosis cases is 20,513 ( SD 9061.45). The risk is estimated higher for the susceptible population (estimate  4.713 × 10⁻⁵; SD 1.466 × 10⁻⁵  ) compared to the normal population  (estimate 7.704 × 10⁻⁶; SD 5.414 × 10⁻⁶)  and is mainly due to undercooking and to a smaller extent to cross-contamination in the kitchen via cook's hands.     

A Quantitative Microbial Risk Assessment Model for Legionnaires'' Disease: Animal Model Selection and Dose-Response Modeling

Legionnaires' disease (LD), first reported in 1976, is an atypical pneumonia caused by bacteria of the genus Legionella, and most frequently by L. pneumophila (Lp). Subsequent research on exposure to the organism employed various animal models, and with quantitative microbial risk assessment (QMRA) techniques, the animal model data may provide insights on human dose-response for LD. This article focuses on the rationale for selection of the guinea pig model, comparison of the dose-response model results, comparison of projected low-dose responses for guinea pigs, and risk estimates for humans. Based on both in vivo and in vitro comparisons, the guinea pig (Cavia porcellus) dose-response data were selected for modeling human risk. We completed dose-response modeling for the beta-Poisson (approximate and exact), exponential, probit, logistic, and Weibull models for Lp inhalation, mortality, and infection (end point elevated body temperature) in guinea pigs. For mechanistic reasons, including low-dose exposure probability, further work on human risk estimates for LD employed the exponential and beta-Poisson models. With an exposure of 10 colony-forming units (CFU) (retained dose), the QMRA model predicted a mild infection risk of 0.4 (as evaluated by seroprevalence) and a clinical severity LD case (e.g., hospitalization and supportive care) risk of 0.0009. The calculated rates based on estimated human exposures for outbreaks used for the QMRA model validation are within an order of magnitude of the reported LD rates. These validation results suggest the LD QMRA animal model selection, dose-response modeling, and extension to human risk projections were appropriate.     

A Practical Framework for the Construction of a Biotracing Model: Application to Salmonella in the Pork Slaughter Chain

A novel purpose of the use of mathematical models in quantitative microbial risk assessment (QMRA) is to identify the sources of microbial contamination in a food chain (i.e., biotracing). In this article we propose a framework for the construction of a biotracing model, eventually to be used in industrial food production chains where discrete numbers of products are processed that may be contaminated by a multitude of sources. The framework consists of steps in which a Monte Carlo model, simulating sequential events in the chain following a modular process risk modeling (MPRM) approach, is converted to a Bayesian belief network (BBN). The resulting model provides a probabilistic quantification of concentrations of a pathogen throughout a production chain. A BBN allows for updating the parameters of the model based on observational data, and global parameter sensitivity analysis is readily performed in a BBN. Moreover, a BBN enables “backward reasoning” when downstream data are available and is therefore a natural framework for answering biotracing questions. The proposed framework is illustrated with a biotracing model of Salmonella in the pork slaughter chain, based on a recently published Monte Carlo simulation model. This model, implemented as a BBN, describes the dynamics of Salmonella in a Dutch slaughterhouse and enables finding the source of contamination of specific carcasses at the end of the chain.     

Advice to Risk Assessors Modeling Viral Health Risk Associated with Household Graywater

Quantitative microbial risk assessment (QMRA) is a valuable tool that can be used to predict the risk associated with human exposure to specific microbial contaminants in water sources. The transparency inherent in the QMRA process benefits discussions between multidisciplinary teams because members of such teams have different expertise and their confidence in the risk assessment output will depend upon whether they regard the selected input data and assumptions as being suitable and/or plausible. Selection of input data requires knowledge of the availability of appropriate data sets, the limitations of using a particular data set, and the logic of using alternative approaches. In performing QMRA modeling and in the absence of directly relevant data, compromises must be made. One such compromise made is to use available Escherichia coli data and apply a ratio of enteric viruses to indicator E. coli in wastewater obtained from prior studies to estimate the concentration of enteric viruses in other wastewater types/sources. In this article, we have provided an argument for why we do not recommend the use of a pathogen to E. coli ratio to estimate virus concentrations in single household graywater and additionally suggested circumstances in which use of such a ratio may be justified.     

Modeling Logistic Performance in Quantitative Microbial Risk Assessment

In quantitative microbial risk assessment (QMRA), food safety in the food chain is modeled and simulated. In general, prevalences, concentrations, and numbers of microorganisms in media are investigated in the different steps from farm to fork. The underlying rates and conditions (such as storage times, temperatures, gas conditions, and their distributions) are determined. However, the logistic chain with its queues (storages, shelves) and mechanisms for ordering products is usually not taken into account. As a consequence, storage times—mutually dependent in successive steps in the chain—cannot be described adequately. This may have a great impact on the tails of risk distributions. Because food safety risks are generally very small, it is crucial to model the tails of (underlying) distributions as accurately as possible. Logistic performance can be modeled by describing the underlying planning and scheduling mechanisms in discrete-event modeling. This is common practice in operations research, specifically in supply chain management. In this article, we present the application of discrete-event modeling in the context of a QMRA for  Listeria monocytogenes  in fresh-cut iceberg lettuce. We show the potential value of discrete-event modeling in QMRA by calculating logistic interventions (modifications in the logistic chain) and determining their significance with respect to food safety.     

Inferring an Augmented Bayesian Network to Confront a Complex Quantitative Microbial Risk Assessment Model with Durability Studies: Application to Bacillus Cereus on a Courgette Purée Production Chain

The Monte Carlo (MC) simulation approach is traditionally used in food safety risk assessment to study quantitative microbial risk assessment (QMRA) models. When experimental data are available, performing Bayesian inference is a good alternative approach that allows backward calculation in a stochastic QMRA model to update the experts’ knowledge about the microbial dynamics of a given food‐borne pathogen. In this article, we propose a complex example where Bayesian inference is applied to a high‐dimensional second‐order QMRA model. The case study is a farm‐to‐fork QMRA model considering genetic diversity of Bacillus cereus in a cooked, pasteurized, and chilled courgette purée. Experimental data are Bacillus cereus concentrations measured in packages of courgette purées stored at different time‐temperature profiles after pasteurization. To perform a Bayesian inference, we first built an augmented Bayesian network by linking a second‐order QMRA model to the available contamination data. We then ran a Markov chain Monte Carlo (MCMC) algorithm to update all the unknown concentrations and unknown quantities of the augmented model. About 25% of the prior beliefs are strongly updated, leading to a reduction in uncertainty. Some updates interestingly question the QMRA model.     

The Impact of Consumer Phase Models in Microbial Risk Analysis

In quantitative microbiological risk assessment (QMRA), the consumer phase model (CPM) describes the part of the food chain between purchase of the food product at retail and exposure. Construction of a CPM is complicated by the large variation in consumer food handling practices and a limited availability of data. Therefore, several subjective (simplifying) assumptions have to be made when a CPM is constructed, but with a single CPM their impact on the QMRA results is unclear. We therefore compared the performance of eight published CPMs for Campylobacter in broiler meat in an example of a QMRA, where all the CPMs were analyzed using one single input distribution of concentrations at retail, and the same dose‐response relationship. It was found that, between CPMs, there may be a considerable difference in the estimated probability of illness per serving. However, the estimated relative risk reductions are less different for scenarios modeling the implementation of control measures. For control measures affecting the Campylobacter prevalence, the relative risk is proportional irrespective of the CPM used. However, for control measures affecting the concentration the CPMs show some difference in the estimated relative risk. This difference is largest for scenarios where the aim is to remove the highly contaminated portion from human exposure. Given these results, we conclude that for many purposes it is not necessary to develop a new detailed CPM for each new QMRA. However, more observational data on consumer food handling practices and their impact on microbial transfer and survival are needed to generalize this conclusion.     

Combining QMRA and Epidemiology to Estimate Campylobacteriosis Incidence

The disease burden of pathogens as estimated by QMRA (quantitative microbial risk assessment) and EA (epidemiological analysis) often differs considerably. This is an unsatisfactory situation for policymakers and scientists. We explored methods to obtain a unified estimate using campylobacteriosis in the Netherlands as an example, where previous work resulted in estimates of 4.9 million (QMRA) and 90,600 (EA) cases per year. Using the maximum likelihood approach and considering EA the gold standard, the QMRA model could produce the original EA estimate by adjusting mainly the dose‐infection relationship. Considering QMRA the gold standard, the EA model could produce the original QMRA estimate by adjusting mainly the probability that a gastroenteritis case is caused by Campylobacter. A joint analysis of QMRA and EA data and models assuming identical outcomes, using a frequentist or Bayesian approach (using vague priors), resulted in estimates of 102,000 or 123,000 campylobacteriosis cases per year, respectively. These were close to the original EA estimate, and this will be related to the dissimilarity in data availability. The Bayesian approach further showed that attenuating the condition of equal outcomes immediately resulted in very different estimates of the number of campylobacteriosis cases per year and that using more informative priors had little effect on the results. In conclusion, EA was dominant in estimating the burden of campylobacteriosis in the Netherlands. However, it must be noted that only statistical uncertainties were taken into account here. Taking all, usually difficult to quantify, uncertainties into account might lead to a different conclusion.     

Validation of Quantitative Microbial Risk Assessment Using Epidemiological Data from Outbreaks of Waterborne Gastrointestinal Disease

The assumptions underlying quantitative microbial risk assessment (QMRA) are simple and biologically plausible, but QMRA predictions have never been validated for many pathogens. The objective of this study was to validate QMRA predictions against epidemiological measurements from outbreaks of waterborne gastrointestinal disease. I screened 2,000 papers and identified 12 outbreaks with the necessary data: disease rates measured using epidemiological methods and pathogen concentrations measured in the source water. Eight of the 12 outbreaks were caused by Cryptosporidium, three by Giardia, and one by norovirus. Disease rates varied from 5.5 × 10^?6 to 1.1 × 10^?2 cases/person‐day, and reported pathogen concentrations varied from 1.2 × 10^?4 to 8.6 × 10² per liter. I used these concentrations with single‐hit dose–response models for all three pathogens to conduct QMRA, producing both point and interval predictions of disease rates for each outbreak. Comparison of QMRA predictions to epidemiological measurements showed good agreement; interval predictions contained measured disease rates for 9 of 12 outbreaks, with point predictions off by factors of 1.0–120 (median = 4.8). Furthermore, 11 outbreaks occurred at mean doses of less than 1 pathogen per exposure. Measured disease rates for these outbreaks were clearly consistent with a single‐hit model, and not with a “two‐hit” threshold model. These results demonstrate the validity of QMRA for predicting disease rates due to Cryptosporidium and Giardia.     

Bayesian Treatment of Model Uncertainty for Partially Applicable Models

This article discusses how analyst's or expert's beliefs on the credibility and quality of models can be assessed and incorporated into the uncertainty assessment of an unknown of interest. The proposed methodology is a specialization of the Bayesian framework for the assessment of model uncertainty presented in an earlier paper. This formalism treats models as sources of information in assessing the uncertainty of an unknown, and it allows the use of predictions from multiple models as well as experimental validation data about the models’ performances. In this article, the methodology is extended to incorporate additional types of information about the model, namely, subjective information in terms of credibility of the model and its applicability when it is used outside its intended domain of application. An example in the context of fire risk modeling is also provided.     

Producing ratio measures of effect with quantitative microbial risk assessment

Estimating the risk of infections or other outcomes incident to pathogen exposure is a primary goal of quantitative microbial risk assessment (QMRA). Such estimates are useful to predict population-level risks, to evaluate exposures based on normative or tolerable risk guidelines, and to interpret the likely public health relevance of microbial measurements in environmental media. To evaluate alternative control measures (interventions), ratio estimates of effect (e.g., odds and risk ratios) are needed that are more broadly interpretable in the health sciences and consistent with convention in epidemiology. In this paper, we propose a general method for estimating widely used ratio measures of effect derived from stochastic QMRA approaches, including the generation of appropriate confidence intervals. Such QMRA-derived ratios can be used as a basis for evaluating interventions via hypothesis testing and for inclusion in systematic reviews and meta-analyses in a form consistent with risk estimation approaches commonly used in epidemiology.     

Application of a QMRA Framework to Inform Selection of Drinking Water Interventions in the Developing Context

The aim of this study was to develop a modified quantitative microbial risk assessment (QMRA) framework that could be applied as a decision support tool to choose between alternative drinking water interventions in the developing context. The impact of different household water treatment (HWT) interventions on the overall incidence of diarrheal disease and disability adjusted life years (DALYs) was estimated, without relying on source water pathogen concentration as the starting point for the analysis. A framework was developed and a software tool constructed and then implemented for an illustrative case study for Nepal based on published scientific data. Coagulation combined with free chlorine disinfection provided the greatest estimated health gains in the short term; however, when long‐term compliance was incorporated into the calculations, the preferred intervention was porous ceramic filtration. The model demonstrates how the QMRA framework can be used to integrate evidence from different studies to inform management decisions, and in particular to prioritize the next best intervention with respect to estimated reduction in diarrheal incidence. This study only considered HWT interventions; it is recognized that a systematic consideration of sanitation, recreation, and drinking water pathways is important for effective management of waterborne transmission of pathogens, and the approach could be expanded to consider the broader water‐related context.     

Regional Risk Assessment for Point Source Pollution Based on a Water Quality Model of the Taipu River,China

Point source pollution is one of the main threats to regional environmental health. Based on a water quality model, a methodology to assess the regional risk of point source pollution is proposed. The assessment procedure includes five parts: (1) identifying risk source units and estimating source emissions using Monte Carlo algorithms; (2) observing hydrological and water quality data of the assessed area, and evaluating the selected water quality model; (3) screening out the assessment endpoints and analyzing receptor vulnerability with the Choquet fuzzy integral algorithm; (4) using the water quality model introduced in the second step to predict pollutant concentrations for various source emission scenarios and analyzing hazards of risk sources; and finally, (5) using the source hazard values and receptor vulnerability scores to estimate overall regional risk. The proposed method, based on the Water Quality Analysis Simulation Program (WASP), was applied in the region of the Taipu River, which is in the Taihu Basin, China. Results of source hazard and receptor vulnerability analysis allowed us to describe aquatic ecological, human health, and socioeconomic risks individually, and also integrated risks in the Taipu region, from a series of risk curves. Risk contributions of sources to receptors were ranked, and the spatial distribution of risk levels was presented. By changing the input conditions, we were able to estimate risks for a range of scenarios. Thus, the proposed procedure may also be used by decisionmakers for long‐term dynamic risk prediction.     

Assessing the Benefits and Costs of Homeland Security Research: A Risk-Informed Methodology with Applications for the U.S. Coast Guard

This article describes a methodology for risk-informed benefit–cost analyses of homeland security research products. The methodology is field-tested with 10 research products developed for the U.S. Coast Guard. Risk-informed benefit–cost analysis is a tool for risk management that integrates elements of risk analysis, decision analysis, and benefit–cost analysis. The cost analysis methodology includes a full-cost accounting of research projects, starting with initial fundamental research costs and extending to the costs of implementation of the research products and, where applicable, training, maintenance, and upgrade costs. The benefits analysis methodology is driven by changes in costs and risks leading to five alternative models: cost savings at the same level of security, increased security at the same cost, signal detection improvements, risk reduction by deterrence, and value of information. The U.S. Coast Guard staff selected 10 research projects to test and generalize the methodology. Examples include tools to improve the detection of explosives, reduce the costs of harbor patrols, and provide better predictions of hurricane wind speeds and floods. Benefits models and estimates varied by research project and many input parameters of the benefit estimates were highly uncertain, so risk analysis for sensitivity testing and simulation was important. Aggregating across the 10 research products, we found an overall median net present value of about $385 million, with a range from $54 million (5th percentile) to $877 million (95th percentile). Lessons learned are provided for future applications.     

Development of an Interspecies Nested Dose‐Response Model for Mycobacterium avium subspecies paratuberculosis

Mycobacterium avium subspecies paratuberculosis (MAP) causes chronic inflammation of the intestines in humans, ruminants, and other species. It is the causative agent of Johne's disease in cattle, and has been implicated as the causative agent of Crohn's disease in humans. To date, no quantitative microbial risk assessment (QMRA) for MAP utilizing a dose‐response function exists. The objective of this study is to develop a nested dose‐response model for infection from oral exposure to MAP utilizing data from the peer‐reviewed literature. Four studies amenable to dose‐response modeling were identified in the literature search and optimized to the one‐parameter exponential or two‐parameter beta‐Poisson dose‐response models. A nesting analysis was performed on all permutations of the candidate data sets to determine the acceptability of pooling data sets across host species. Three of four data sets exhibited goodness of fit to at least one model. All three data sets exhibited good fit to the beta‐Poisson model, and one data set exhibited goodness of fit, and best fit, to the exponential model. Two data sets were successfully nested using the beta‐Poisson model with parameters α = 0.0978 and N₅₀ = 2.70 × 10² CFU. These data sets were derived from sheep and red deer host species, indicating successful interspecies nesting, and demonstrate the highly infective nature of MAP. The nested dose‐response model described should be used for future QMRA research regarding oral exposure to MAP.     

A Poultry-Processing Model for Quantitative Microbiological Risk Assessment

A poultry-processing model for a quantitative microbiological risk assessment (QMRA) of campylobacter is presented, which can also be applied to other QMRAs involving poultry processing. The same basic model is applied in each consecutive stage of industrial processing. It describes the effects of inactivation and removal of the bacteria, and the dynamics of cross-contamination in terms of the transfer of campylobacter from the intestines to the carcass surface and the environment, from the carcasses to the environment, and from the environment to the carcasses. From the model it can be derived that, in general, the effect of inactivation and removal is dominant for those carcasses with high initial bacterial loads, and cross-contamination is dominant for those with low initial levels. In other QMRA poultry-processing models, the input-output relationship between the numbers of bacteria on the carcasses is usually assumed to be linear on a logarithmic scale. By including some basic mechanistics, it is shown that this may not be realistic. As nonlinear behavior may affect the predicted effects of risk mitigations; this finding is relevant for risk management. Good knowledge of the variability of bacterial loads on poultry entering the process is important. The common practice in microbiology to only present geometric mean of bacterial counts is insufficient: arithmetic mean are more suitable, in particular, to describe the effect of cross-contamination. The effects of logistic slaughter (scheduled processing) as a risk mitigation strategy are predicted to be small. Some additional complications in applying microbiological data obtained in processing plants are discussed.     

A Critical Discussion and Practical Recommendations on Some Issues Relevant to the Nonprobabilistic Treatment of Uncertainty in Engineering Risk Assessment

Models for the assessment of the risk of complex engineering systems are affected by uncertainties due to the randomness of several phenomena involved and the incomplete knowledge about some of the characteristics of the system. The objective of this article is to provide operative guidelines to handle some conceptual and technical issues related to the treatment of uncertainty in risk assessment for engineering practice. In particular, the following issues are addressed: (1) quantitative modeling and representation of uncertainty coherently with the information available on the system of interest; (2) propagation of the uncertainty from the input(s) to the output(s) of the system model; (3) (Bayesian) updating as new information on the system becomes available; and (4) modeling and representation of dependences among the input variables and parameters of the system model. Different approaches and methods are recommended for efficiently tackling each of issues (1)?(4) above; the tools considered are derived from both classical probability theory as well as alternative, nonfully probabilistic uncertainty representation frameworks (e.g., possibility theory). The recommendations drawn are supported by the results obtained in illustrative applications of literature.     

Policies for knowledge refreshing in databases

Knowledge discovery in databases (KDD) provides organizations necessary tools to sift through vast data stores to extract knowledge. This process supports and improves decision making in organizations. In this paper, we introduce and define the concept of knowledge refreshing, a critical step to ensure the quality and timeliness of knowledge discovered in a KDD process. This has been unfortunately overlooked by prior researchers. Specifically, we study knowledge refreshing from the perspective of when to refresh knowledge so that the total system cost over a time horizon is minimized. We propose a model for knowledge refreshing, and a dynamic programming methodology for developing optimal strategies. We demonstrate the effectiveness of the proposed methodology using data from a real world application. The proposed methodology provides decision makers guidance in running KDD effectively and efficiently.