Structured expert elicitation about Listeria monocytogenes cross-contamination in the environment of retail deli operations in the United States

Listeria monocytogenes is among the foodborne pathogens with the highest death toll in the United States. Ready‐to‐eat foods contaminated at retail are an important source of infection. Environmental sites in retail deli operations can be contaminated. However, commonly contaminated sites are unlikely to come into direct contact with food and the public health relevance of environmental contamination has remained unclear. To identify environmental sites that may pose a considerable cross‐contamination risk, to elucidate potential transmission pathways, and to identify knowledge gaps, we performed a structured expert elicitation of 41 experts from state regulatory agencies and the food retail industry with practical experience in retail deli operations. Following the “Delphi” method, the elicitation was performed in three consecutive steps: questionnaire, review and discussion of results, second questionnaire. Hands and gloves were identified as important potential contamination sources. However, bacterial transfers to and from hands or gloves represented a major data gap. Experts agreed about transfer probabilities from cutting boards, scales, deli cases, and deli preparation sinks to product, and about transfer probabilities from floor drains, walk‐in cooler floors, and knife racks to food contact surfaces. Comparison of experts' opinions to observational data revealed a tendency among experts with certain demographic characteristics and professional opinions to overestimate prevalence. Experts’ votes clearly clustered into separate groups not defined by place of employment, even though industry experts may have been somewhat overrepresented in one cluster. Overall, our study demonstrates the value and caveats of expert elicitation to identify data gaps and prioritize research efforts.     

An Agent‐Based Model for Pathogen Persistence and Cross‐Contamination Dynamics in a Food Facility

We used an agent‐based modeling (ABM) framework and developed a mathematical model to explain the complex dynamics of microbial persistence and spread within a food facility and to aid risk managers in identifying effective mitigation options. The model explicitly considered personal hygiene practices by food handlers as well as their activities and simulated a spatially explicit dynamic system representing complex interaction patterns among food handlers, facility environment, and foods. To demonstrate the utility of the model in a decision‐making context, we created a hypothetical case study and used it to compare different risk mitigation strategies for reducing contamination and spread of Listeria monocytogenes in a food facility. Model results indicated that areas with no direct contact with foods (e.g., loading dock and restroom) can serve as contamination niches and recontaminate areas that have direct contact with food products. Furthermore, food handlers’ behaviors, including, for example, hygiene and sanitation practices, can impact the persistence of microbial contamination in the facility environment and the spread of contamination to prepared foods. Using this case study, we also demonstrated benefits of an ABM framework for addressing food safety in a complex system in which emergent system‐level responses are predicted using a bottom‐up approach that observes individual agents (e.g., food handlers) and their behaviors. Our model can be applied to a wide variety of pathogens, food commodities, and activity patterns to evaluate efficacy of food‐safety management practices and quantify contamination reductions associated with proposed mitigation strategies in food facilities.     

Application of Whole-Genome Sequences and Machine Learning in Source Attribution of Salmonella Typhimurium

Prevention of the emergence and spread of foodborne diseases is an important prerequisite for the improvement of public health. Source attribution models link sporadic human cases of a specific illness to food sources and animal reservoirs. With the next generation sequencing technology, it is possible to develop novel source attribution models. We investigated the potential of machine learning to predict the animal reservoir from which a bacterial strain isolated from a human salmonellosis case originated based on whole-genome sequencing. Machine learning methods recognize patterns in large and complex data sets and use this knowledge to build models. The model learns patterns associated with genetic variations in bacteria isolated from the different animal reservoirs. We selected different machine learning algorithms to predict sources of human salmonellosis cases and trained the model with Danish Salmonella Typhimurium isolates sampled from broilers (n = 34), cattle (n = 2), ducks (n = 11), layers (n = 4), and pigs (n = 159). Using cgMLST as input features, the model yielded an average accuracy of 0.783 (95% CI: 0.77–0.80) in the source prediction for the random forest and 0.933 (95% CI: 0.92–0.94) for the logit boost algorithm. Logit boost algorithm was most accurate (valid accuracy: 92%, CI: 0.8706–0.9579) and predicted the origin of 81% of the domestic sporadic human salmonellosis cases. The most important source was Danish produced pigs (53%) followed by imported pigs (16%), imported broilers (6%), imported ducks (2%), Danish produced layers (2%), Danish produced cattle and imported cattle (<1%) while 18% was not predicted. Machine learning has potential for improving source attribution modeling based on sequence data. Results of such models can inform risk managers to identify and prioritize food safety interventions.     

面向专家的知识库优化

知识库的质量是影响智能系统性能的主要因素 ,而知识获取一直是设计智能系统的瓶颈问题 ,这是由于目前人类认识的局限性 ,导致知识工程师和专家之间的不协调关系造成的 .为克服上述不利局面 ,本文利用粗糙集等理论 ,得到含有噪声的初始知识库 ,然后采用遗传算法、可视化技术和知识校验等技术对规则库和案例库进行了优化 .从而在知识获取过程中建立了知识工程师和专家之间的新型的关系 ,其中专家处于中心地位 ,知识工程师只是起辅助作用 ,即整个知识获取过程是面向专家的     

Listeria monocytogenes Dose Response Revisited—Incorporating Adjustments for Variability in Strain Virulence and Host Susceptibility

Evaluations of Listeria monocytogenes dose‐response relationships are crucially important for risk assessment and risk management, but are complicated by considerable variability across population subgroups and L. monocytogenes strains. Despite difficulties associated with the collection of adequate data from outbreak investigations or sporadic cases, the limitations of currently available animal models, and the inability to conduct human volunteer studies, some of the available data now allow refinements of the well‐established exponential L. monocytogenes dose response to more adequately represent extremely susceptible population subgroups and highly virulent L. monocytogenes strains. Here, a model incorporating adjustments for variability in L. monocytogenes strain virulence and host susceptibility was derived for 11 population subgroups with similar underlying comorbidities using data from multiple sources, including human surveillance and food survey data. In light of the unique inherent properties of L. monocytogenes dose response, a lognormal‐Poisson dose‐response model was chosen, and proved able to reconcile dose‐response relationships developed based on surveillance data with outbreak data. This model was compared to a classical beta‐Poisson dose‐response model, which was insufficiently flexible for modeling the specific case of L. monocytogenes dose‐response relationships, especially in outbreak situations. Overall, the modeling results suggest that most listeriosis cases are linked to the ingestion of food contaminated with medium to high concentrations of L. monocytogenes. While additional data are needed to refine the derived model and to better characterize and quantify the variability in L. monocytogenes strain virulence and individual host susceptibility, the framework derived here represents a promising approach to more adequately characterize the risk of listeriosis in highly susceptible population subgroups.     

Ensemble Habitat Mapping of Invasive Plant Species

Ensemble species distribution models combine the strengths of several species environmental matching models, while minimizing the weakness of any one model. Ensemble models may be particularly useful in risk analysis of recently arrived, harmful invasive species because species may not yet have spread to all suitable habitats, leaving species‐environment relationships difficult to determine. We tested five individual models (logistic regression, boosted regression trees, random forest, multivariate adaptive regression splines (MARS), and maximum entropy model or Maxent) and ensemble modeling for selected nonnative plant species in Yellowstone and Grand Teton National Parks, Wyoming; Sequoia and Kings Canyon National Parks, California, and areas of interior Alaska. The models are based on field data provided by the park staffs, combined with topographic, climatic, and vegetation predictors derived from satellite data. For the four invasive plant species tested, ensemble models were the only models that ranked in the top three models for both field validation and test data. Ensemble models may be more robust than individual species‐environment matching models for risk analysis.     

Machine Learning Methods as a Tool for Predicting Risk of Illness Applying Next‐Generation Sequencing Data

Next‐generation sequencing (NGS) data present an untapped potential to improve microbial risk assessment (MRA) through increased specificity and redefinition of the hazard. Most of the MRA models do not account for differences in survivability and virulence among strains. The potential of machine learning algorithms for predicting the risk/health burden at the population level while inputting large and complex NGS data was explored with Listeria monocytogenes as a case study. Listeria data consisted of a percentage similarity matrix from genome assemblies of 38 and 207 strains of clinical and food origin, respectively. Basic Local Alignment (BLAST) was used to align the assemblies against a database of 136 virulence and stress resistance genes. The outcome variable was frequency of illness, which is the percentage of reported cases associated with each strain. These frequency data were discretized into seven ordinal outcome categories and used for supervised machine learning and model selection from five ensemble algorithms. There was no significant difference in accuracy between the models, and support vector machine with linear kernel was chosen for further inference (accuracy of 89% [95% CI: 68%, 97%]). The virulence genes FAM002725, FAM002728, FAM002729, InlF, InlJ, Inlk, IisY, IisD, IisX, IisH, IisB, lmo2026, and FAM003296 were important predictors of higher frequency of illness. InlF was uniquely truncated in the sequence type 121 strains. Most important risk predictor genes occurred at highest prevalence among strains from ready‐to‐eat, dairy, and composite foods. We foresee that the findings and approaches described offer the potential for rethinking the current approaches in MRA.     

Framework for Microbial Food‐Safety Risk Assessments Amenable to Bayesian Modeling

Regulatory agencies often perform microbial risk assessments to evaluate the change in the number of human illnesses as the result of a new policy that reduces the level of contamination in the food supply. These agencies generally have regulatory authority over the production and retail sectors of the farm‐to‐table continuum. Any predicted change in contamination that results from new policy that regulates production practices occurs many steps prior to consumption of the product. This study proposes a framework for conducting microbial food‐safety risk assessments; this framework can be used to quantitatively assess the annual effects of national regulatory policies. Advantages of the framework are that estimates of human illnesses are consistent with national disease surveillance data (which are usually summarized on an annual basis) and some of the modeling steps that occur between production and consumption can be collapsed or eliminated. The framework leads to probabilistic models that include uncertainty and variability in critical input parameters; these models can be solved using a number of different Bayesian methods. The Bayesian synthesis method performs well for this application and generates posterior distributions of parameters that are relevant to assessing the effect of implementing a new policy. An example, based on Campylobacter and chicken, estimates the annual number of illnesses avoided by a hypothetical policy; this output could be used to assess the economic benefits of a new policy. Empirical validation of the policy effect is also examined by estimating the annual change in the numbers of illnesses observed via disease surveillance systems.     

Estimating Listeria monocytogenes Growth in Ready-to-Eat Chicken Salad Using a Challenge Test for Quantitative Microbial Risk Assessment

Currently, there is a growing preference for convenience food products, such as ready-to-eat (RTE) foods, associated with long refrigerated shelf-lives, not requiring a heat treatment prior to consumption. Because Listeria monocytogenes is able to grow at refrigeration temperatures, inconsistent temperatures during production, distribution, and at consumer's household may allow for the pathogen to thrive, reaching unsafe limits. L. monocytogenes is the causative agent of listeriosis, a rare but severe human illness, with high fatality rates, transmitted almost exclusively by food consumption. With the aim of assessing the quantitative microbial risk of L. monocytogenes in RTE chicken salads, a challenge test was performed. Salads were inoculated with a three-strain mixture of cold-adapted L. monocytogenes and stored at 4, 12, and 16 °C for eight days. Results revealed that the salad was able to support L. monocytogenes’ growth, even at refrigeration temperatures. The Baranyi primary model was fitted to microbiological data to estimate the pathogen's growth kinetic parameters. Temperature effect on the maximum specific growth rate (μ_max) was modeled using a square-root-type model. Storage temperature significantly influenced μ_max of L. monocytogenes (p < 0.05). These predicted growth models for L. monocytogenes were subsequently used to develop a quantitative microbial risk assessment, estimating a median number of 0.00008726 listeriosis cases per year linked to the consumption of these RTE salads. Sensitivity analysis considering different time–temperature scenarios indicated a very low median risk per portion (<−7 log), even if the assessed RTE chicken salad was kept in abuse storage conditions.     

New Data,Strategies, and Insights for Listeria monocytogenes Dose‐Response Models: Summary of an Interagency Workshop, 2011

Listeria monocytogenes is a leading cause of hospitalization, fetal loss, and death due to foodborne illnesses in the United States. A quantitative assessment of the relative risk of listeriosis associated with the consumption of 23 selected categories of ready‐to‐eat foods, published by the U.S. Department of Health and Human Services and the U.S. Department of Agriculture in 2003, has been instrumental in identifying the food products and practices that pose the greatest listeriosis risk and has guided the evaluation of potential intervention strategies. Dose‐response models, which quantify the relationship between an exposure dose and the probability of adverse health outcomes, were essential components of the risk assessment. However, because of data gaps and limitations in the available data and modeling approaches, considerable uncertainty existed. Since publication of the risk assessment, new data have become available for modeling L. monocytogenes dose‐response. At the same time, recent advances in the understanding of L. monocytogenes pathophysiology and strain diversity have warranted a critical reevaluation of the published dose‐response models. To discuss strategies for modeling L. monocytogenes dose‐response, the Interagency Risk Assessment Consortium (IRAC) and the Joint Institute for Food Safety and Applied Nutrition (JIFSAN) held a scientific workshop in 2011 (details available at http://foodrisk.org/irac/events/ ). The main findings of the workshop and the most current and relevant data identified during the workshop are summarized and presented in the context of L. monocytogenes dose‐response. This article also discusses new insights on dose‐response modeling for L. monocytogenes and research opportunities to meet future needs.     

Social Contagion of Risk Perceptions in Environmental Management Networks

An important requisite for improving risk communication practice related to contentious environmental issues is having a better theoretical understanding of how risk perceptions function in real‐world social systems. Our study applied Scherer and Cho's social network contagion theory of risk perception (SNCTRP) to cormorant management (a contentious environmental management issue) in the Great Lakes Basin to: (1) assess contagion effects on cormorant‐related risk perceptions and individual factors believed to influence those perceptions and (2) explore the extent of social contagion in a full network (consisting of interactions between and among experts and laypeople) and three “isolated” models separating different types of interactions from the full network (i.e., expert‐to‐expert, layperson‐to‐layperson, and expert‐to‐layperson). We conducted interviews and administered questionnaires with experts (e.g., natural resource professionals) and laypeople (e.g., recreational and commercial anglers, business owners, bird enthusiasts) engaged in cormorant management in northern Lake Huron (n = 115). Our findings generally support the SNCTRP; however, the scope and scale of social contagion varied considerably based on the variables (e.g., individual risk perception factors), actors (i.e., experts or laypeople), and interactions of interest. Contagion effects were identified more frequently, and were stronger, in the models containing interactions between experts and laypeople than in those models containing only interactions among experts or laypeople.     

Validating Expert Systems: A Demonstration Using Personal Choice Expert,A Flexible Employee Benefit System*

A method for validating expert systems, based on validation approaches from psychology and Turing's “imitation game,” is demonstrated using a flexible employee benefits expert system. Psychometric validation has three aspects: the extent to which the system and expert decisions agree (criterionrelated validity), the inputs and processes used by experts compared to the system (content validity), and differences between expert and novice decisions (construct validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies the Turing Test. Personal Choice Expert (PCE) was designed to help employees of a Fortune 500 firm choose benefits in their flexible benefits system. Its recommendations do not significantly differ from those given by independent experts. Hence, if the system-independent expert agreement (criterion-related validity) were the only standard, PCE could be considered valid. However, construct analysis suggests that re-engineering may be required. High intra-expert agreement exists only for some benefit recommendations (e.g., dental care and long-term disability) and not for others (e.g., short-term disability, accidental death and dismemberment, and life insurance). Insights offered by these methods are illustrated and examined.     

Risk‐Based Approach for Microbiological Food Safety Management in the Dairy Industry: The Case of Listeria monocytogenes in Soft Cheese Made from Pasteurized Milk

According to Codex Alimentarius Commission recommendations, management options applied at the process production level should be based on good hygiene practices, HACCP system, and new risk management metrics such as the food safety objective. To follow this last recommendation, the use of quantitative microbiological risk assessment is an appealing approach to link new risk‐based metrics to management options that may be applied by food operators. Through a specific case study, Listeria monocytogenes in soft cheese made from pasteurized milk, the objective of the present article is to practically show how quantitative risk assessment could be used to direct potential intervention strategies at different food processing steps. Based on many assumptions, the model developed estimates the risk of listeriosis at the moment of consumption taking into account the entire manufacturing process and potential sources of contamination. From pasteurization to consumption, the amplification of a primo‐contamination event of the milk, the fresh cheese or the process environment is simulated, over time, space, and between products, accounting for the impact of management options, such as hygienic operations and sampling plans. A sensitivity analysis of the model will help orientating data to be collected prioritarily for the improvement and the validation of the model. What‐if scenarios were simulated and allowed for the identification of major parameters contributing to the risk of listeriosis and the optimization of preventive and corrective measures.     

Probabilistic Modeling of the Fate of Listeria Monocytogenes in Diced Bacon During the Manufacturing Process

To assess the impact of the manufacturing process on the fate of Listeria monocytogenes, we built a generic probabilistic model intended to simulate the successive steps in the process. Contamination evolution was modeled in the appropriate units (breasts, dice, and then packaging units through the successive steps in the process). To calibrate the model, parameter values were estimated from industrial data, from the literature, and based on expert opinion. By means of simulations, the model was explored using a baseline calibration and alternative scenarios, in order to assess the impact of changes in the process and of accidental events. The results are reported as contamination distributions and as the probability that the product will be acceptable with regards to the European regulatory safety criterion. Our results are consistent with data provided by industrial partners and highlight that tumbling is a key step for the distribution of the contamination at the end of the process. Process chain models could provide an important added value for risk assessment models that basically consider only the outputs of the process in their risk mitigation strategies. Moreover, a model calibrated to correspond to a specific plant could be used to optimize surveillance.     

Testing Multiple Forecasters

We consider a cross‐calibration test of predictions by multiple potential experts in a stochastic environment. This test checks whether each expert is calibrated conditional on the predictions made by other experts. We show that this test is good in the sense that a true expert—one informed of the true distribution of the process—is guaranteed to pass the test no matter what the other potential experts do, and false experts will fail the test on all but a small (category I) set of true distributions. Furthermore, even when there is no true expert present, a test similar to cross‐calibration cannot be simultaneously manipulated by multiple false experts, but at the cost of failing some true experts.     

Dengue Disease Risk Mental Models in the City of Dhaka,Bangladesh: Juxtapositions and Gaps Between the Public and Experts

Worldwide, more than 50 million cases of dengue fever are reported every year in at least 124 countries, and it is estimated that approximately 2.5 billion people are at risk for dengue infection. In Bangladesh, the recurrence of dengue has become a growing public health threat. Notably, knowledge and perceptions of dengue disease risk, particularly among the public, are not well understood. Recognizing the importance of assessing risk perception, we adopted a comparative approach to examine a generic methodology to assess diverse sets of beliefs related to dengue disease risk. Our study mapped existing knowledge structures regarding the risk associated with dengue virus, its vector (Aedes mosquitoes), water container use, and human activities in the city of Dhaka, Bangladesh. “Public mental models” were developed from interviews and focus group discussions with diverse community groups; “expert mental models” were formulated based on open‐ended discussions with experts in the pertinent fields. A comparative assessment of the public's and experts’ knowledge and perception of dengue disease risk has revealed significant gaps in the perception of: (a) disease risk indicators and measurements; (b) disease severity; (c) control of disease spread; and (d) the institutions responsible for intervention. This assessment further identifies misconceptions in public perception regarding: (a) causes of dengue disease; (b) dengue disease symptoms; (c) dengue disease severity; (d) dengue vector ecology; and (e) dengue disease transmission. Based on these results, recommendations are put forward for improving communication of dengue risk and practicing local community engagement and knowledge enhancement in Bangladesh.     

Subjective machines: Probabilistic risk assessment based on deep learning of soft information

For several years machine learning methods have been proposed for risk classification. While machine learning methods have also been used for failure diagnosis and condition monitoring, to the best of our knowledge, these methods have not been used for probabilistic risk assessment. Probabilistic risk assessment is a subjective process. The problem of how well machine learning methods can emulate expert judgments is challenging. Expert judgments are based on mental shortcuts, heuristics, which are susceptible to biases. This paper presents a process for developing natural language-based probabilistic risk assessment models, applying deep learning algorithms to emulate experts’ quantified risk estimates. This allows the risk analyst to obtain an a priori risk assessment when there is limited information in the form of text and numeric data. Universal sentence embedding (USE) with gradient boosting regression (GBR) trees trained over limited structured data presented the most promising results. When we apply these models’ outputs to generate survival distributions for autonomous systems’ likelihood of loss with distance, we observe that for open water and ice shelf operating environments, the differences between the survival distributions generated by the machine learning algorithm and those generated by the experts are not statistically significant.     

基于信息熵的群组聚类组合赋权法

在多属性群决策方法的研究中,为了科学地确定专家的权重,提出一种基于信息熵的群组聚类组合赋权法。依据各个专家的判断矩阵归一化得到的排序向量,利用相关系数法构造相关矩阵。通过分析阀值变化率选取最优聚类阀值,对相似程度较高的排序向量给出合理的聚类。运用信息熵为类内专家赋权,综合聚类结果和排序向量的信息熵,确定专家的总权重。算例表明该方法可以对较为相近的专家评价结果进行有效分类,并准确衡量每位专家评价信息量的大小,能够有效提高专家赋权的合理性和群组决策的科学性。     

Combining Quantitative Risk Assessment of Human Health,Food Waste,and Energy Consumption: The Next Step in the Development of the Food Cold Chain?

The preservation of perishable food via refrigeration in the supply chain is essential to extend shelf life and provide consumers with safe food. However, electricity consumed in refrigeration processes has an economical and an environmental impact. This study focuses on the cold chain of cooked ham, including transport, cold room in supermarket, display cabinet, transport by consumer, and domestic refrigerator, and aims to predict the risk for human health associated with Listeria monocytogenes, the amount of food wasted due to the growth of spoilage bacteria, and the electrical consumption to maintain product temperature through the cold chain. A set of eight intervention actions were tested to evaluate their impact on the three criteria. Results show that the modification of the thermostat of the domestic refrigerator has a high impact on food safety and food waste and a limited impact on the electrical consumption. Inversely, the modification of the airflow rate in the display cabinet has a high impact on electrical consumption and a limited impact on food safety and food waste. A cost–benefit analysis approach and two multicriteria decision analysis methods were used to rank the intervention actions. These three methodologies show that setting the thermostat of the domestic refrigerator to 4 °C presents the best compromise between the three criteria. The impact of decisionmaker preferences (criteria weight) and limitations of these three approaches are discussed. The approaches proposed by this study may be useful in decision making to evaluate global impact of intervention actions in issues involving conflicting outputs.     

Genetically Modified Organisms and the U.S. Retail Food Labeling Controversy: Consumer Perceptions,Regulation, and Public Policy

In this article, we address the public issue of mandatory Genetically Modified Organism (GMO) retail food labeling in the U.S., first by reviewing the policy arguments both in support and against labeling food containing GMOs; second, by describing the existing U.S. federal regulatory system pertaining to GMO labeling, and why it does not presently require labeling of food containing GMOs; third, by reviewing and interpreting the results of studies of American consumer attitudes toward mandatory GMO retail food labeling; fourth, by evaluating, through the utilization of issue life cycle analysis in the nonmarket environment, where the issue of GMO retail food labeling stands in the national public policy process; fifth, summarizing the state of scientific evidence addressing the safety of GMO foods and the existing regulatory and public policy environment for this issue; and sixth, offering legislative and litigation strategies for the mainstream food nonmarket strategy framework to formally assess the GMO industry to protect their interests and those of American consumers not concerned with GMO food ingredients, while offering a voluntary labeling strategy for firms responding to and recognizing the “rights” of American consumers who “choose” to purchase non‐GMO food products.