Risk Analysis for Invasive Species: General Framework and Research Needs

A joint workshop was convened by the Society for Risk Analysis Ecological Risk Assessment Specialty Group and the Ecological Society of America Theoretical Ecology Section to provide independent scientific input into the formulation of methods and processes for risk assessment of invasive species. In breakout sessions on (1) the effects of invasive species on human health, (2) effects on plants and animals, (3) risk analysis issues and research needs related to entry and establishment of invasive species, and (4) risk analysis issues and research needs related to the spread and impacts of invasive species, workshop participants discussed an overall approach to risk assessment for invasive species. Workshop participants agreed on the need for empirical research on areas in which data are lacking, including potential invasive species, native species and habitats that may be impacted by invasive species, important biological processes and phenomena such as dispersal, and pathways of entry and spread for invasive species. Participants agreed that theoretical ecology can inform the process of risk assessment for invasive species by providing guidelines and conceptual models, and can contribute to improved decision making by providing a firm biological basis for risk assessments.     

Risk analysis of fast spreading species in a Kashmir Himalayan National Park (Dachigam) for better monitoring and management

Huge economic costs and ecological impacts of invasive alien species (IAS) in the protected areas (PAs) worldwide make their timely prediction and potential risk assessment of central importance for effective management. While the preborder weed risk assessment framework has been extensively evaluated and implemented, the postborder species risk assessment framework has not been subjected to the same degree of scrutiny. Here we used a rather more realistic modified version of the Australian Weed Risk framework (AWRM) for Dachigam National Park (DNP) in Kashmir Himalaya against 84 plant species, including 55 alien species and 29 fast spreading native species, for risk analysis. We found two very high-risk species, three high-risk species, 10 medium-risk species, 29 low-risk species, and 40 negligible-risk species in the DNP. The containment scores accordingly ranged from 14.4 to 293.5 comprising of 27 species that can be contained with very high feasibility, 23 species with high feasibility, 14 species with medium feasibility, and 12 species which cannot be contained easily thereby having low feasibility of containment (FOC) score. However, eight species which have a negligible FOC score are difficult to contain within their infestation sites. Our results demonstrate the merit of the AWRM with a caution that the necessary region-specific modifications may help in its better implementation. Overall, these results provide quite a promising tool in the hands of protected area managers to timely and effectively deal with the problem of plant invasions.     

Genetic Diversity of Invasive Species in the Great Lakes Versus Their Eurasian Source Populations: Insights for Risk Analysis

Combining DNA variation data and risk assessment procedures offers important diagnostic and monitoring tools for evaluating the relative success of exotic species invasions. Risk assessment may allow us to understand how the numbers of founding individuals, genetic variants, population sources, and introduction events affect successful establishment and spread. This is particularly important in habitats that are "hotbeds" for invasive species--such as the North American Great Lakes. This study compares genetic variability and its application to risk assessment within and among three Eurasian groups and five species that successfully invaded the Great Lakes during the mid 1980s through early 1990s; including zebra and quagga mussels, round and tubenose gobies, and the ruffe. DNA sequences are compared from exotic and native populations in order to evaluate the role of genetic diversity in invasions. Close relatives are also examined, since they often invade in concert and several are saline tolerant and are likely to spread to North American estuaries. Results show that very high genetic diversity characterizes the invasions of all five species, indicating that they were founded by very large numbers of propagules and underwent no founder effects. Genetic evidence points to multiple invasion sources for both dreissenid and goby species, which appears related to especially rapid spread and widespread colonization success in a variety of habitats. In contrast, results show that the ruffe population in the Great Lakes originated from a single founding population source from the Elbe River drainage. Both the Great Lakes and the Elbe River populations of ruffe have similar genetic diversity levels--showing no founder effect, as in the other invasive species. In conclusion, high genetic variability, large numbers of founders, and multiple founding sources likely significantly contribute to the risk of an exotic species introduction's success and persistence.     

Establishment Risks for Invasive Species

This article presents a quantitative methodology for evaluating the probability of invasive pest species establishing persistent populations. The estimation of pest establishment relies on data and information describing the biology and ecology of the pest and its interactions with potential host species and the regional environment. This information is developed using a model construct borrowed from theoretical population ecology. The methodology for estimating the probability of pest establishment is part of an overall framework that explores the implications of reductions in pest invasions on subsequent establishment. The risk reduction framework integrates the engineering aspects of different technologies for reducing pest entry, the biology and ecology of pest species, the suitability of potentially susceptible hosts, and the quality of available habitats. The methodology for estimating the risk of establishment is presented using an example pest, the Asian longhorned beetle ( Anoplophora glabripennis ), which has been introduced into the United States via solid wood packing materials (SWPM) used in international commerce. Uncertainties inherent to the estimation of model parameters that determine the risk of establishment are defined, quantified, and propagated through the population model. Advantages and limitations of the proposed methodology are discussed along with recommendations to make the approach more useful in the management of risks posed by the establishment of pest populations.     

Elusive Critical Elements of Transformative Risk Assessment Practice and Interpretation: Is Alternatives Analysis the Next Step?

This article argues that “game‐changing” approaches to risk analysis must focus on “democratizing” risk analysis in the same way that information technologies have democratized access to, and production of, knowledge. This argument is motivated by the author's reading of Goble and Bier's analysis, “Risk Assessment Can Be a Game‐Changing Information Technology—But Too Often It Isn't” (Risk Analysis, 2013; 33: 1942–1951), in which living risk assessments are shown to be “game changing” in probabilistic risk analysis. In this author's opinion, Goble and Bier's article focuses on living risk assessment's potential for transforming risk analysis from the perspective of risk professionals—yet, the game‐changing nature of information technologies has typically achieved a much broader reach. Specifically, information technologies change who has access to, and who can produce, information. From this perspective, the author argues that risk assessment is not a game‐changing technology in the same way as the printing press or the Internet because transformative information technologies reduce the cost of production of, and access to, privileged knowledge bases. The author argues that risk analysis does not reduce these costs. The author applies Goble and Bier's metaphor to the chemical risk analysis context, and in doing so proposes key features that transformative risk analysis technology should possess. The author also discusses the challenges and opportunities facing risk analysis in this context. These key features include: clarity in information structure and problem representation, economical information dissemination, increased transparency to nonspecialists, democratized manufacture and transmission of knowledge, and democratic ownership, control, and interpretation of knowledge. The chemical safety decision‐making context illustrates the impact of changing the way information is produced and accessed in the risk context. Ultimately, the author concludes that although new chemical safety regulations do transform access to risk information, they do not transform the costs of producing this information—rather, they change the bearer of these costs. The need for further risk assessment transformation continues to motivate new practical and theoretical developments in risk analysis and management.     

Risk Assessment for Invasive Species

Although estimates vary, there is a broad agreement that invasive species impose major costs on the U.S. economy, as well as posing risks to nonmarket environmental goods and services and to public health. The domestic effort to manage risks associated with invasive species is coordinated by the National Invasive Species Council (NISC), which is charged with developing a science-based process to evaluate risks associated with the introduction and spread of invasive species. Various international agreements have also elevated invasive species issues onto the international policy agenda. The World Trade Organization (WTO) Sanitary and Phytosanitary (SPS) Agreement establishes rights and obligations to adhere to the discipline of scientific risk assessment to ensure that SPS measures are applied only to the extent required to protect human, animal, and plant health, and do not constitute arbitrary or unjustifiable technical barriers to trade. Currently, however, the field of risk assessment for invasive species is in its infancy. Therefore, there is a pressing need to formulate scientifically sound methods and approaches in this emerging field, while acknowledging that the demand for situation-specific empirical evidence is likely to persistently outstrip supply. To begin addressing this need, the Society for Risk Analysis Ecological Risk Assessment Specialty Group and the Ecological Society of America Theoretical Ecology Section convened a joint workshop to provide independent scientific input into the formulation of methods and processes for risk assessment of invasive species to ensure that the analytic processes used domestically and internationally will be firmly rooted in sound scientific principles.     

Effectiveness of FISK,an Invasiveness Screening Tool for Non‐Native Freshwater Fishes,to Perform Risk Identification Assessments in the Iberian Peninsula

Risk assessments are crucial for identifying and mitigating impacts from biological invasions. The Fish Invasiveness Scoring Kit (FISK) is a risk identification (screening) tool for freshwater fishes consisting of two subject areas: biogeography/history and biology/ecology. According to the outcomes, species can be classified under particular risk categories. The aim of this study was to apply FISK to the Iberian Peninsula, a Mediterranean climate region highly important for freshwater fish conservation due to a high level of endemism. In total, 89 fish species were assessed by three independent assessors. Results from receiver operating characteristic analysis showed that FISK can discriminate reliably between noninvasive and invasive fishes for Iberia, with a threshold of 20.25, similar to those obtained in several regions around the world. Based on mean scores, no species was categorized as “low risk,” 50 species as “medium risk,” 17 as “moderately high risk,” 11 as “high risk,” and 11 as “very high risk.” The highest scoring species was goldfish Carassius auratus. Mean certainty in response was above the category “mostly certain,” ranging from tinfoil barb Barbonymus schwanenfeldii with the lowest certainty to eastern mosquitofish Gambusia holbrooki with the highest level. Pair‐wise comparison showed significant differences between one assessor and the other two on mean certainty, with these two assessors showing a high coincidence rate for the species categorization. Overall, the results suggest that FISK is a useful and viable tool for assessing risks posed by non‐native fish in the Iberian Peninsula and contributes to a “watch list” in this region.     

Evaluating Critical Uncertainty Thresholds in a Spatial Model of Forest Pest Invasion Risk

Pest risk maps can provide useful decision support in invasive species management, but most do not adequately consider the uncertainty associated with predicted risk values. This study explores how increased uncertainty in a risk model's numeric assumptions might affect the resultant risk map. We used a spatial stochastic model, integrating components for entry, establishment, and spread, to estimate the risks of invasion and their variation across a two-dimensional landscape for Sirex noctilio , a nonnative woodwasp recently detected in the United States and Canada. Here, we present a sensitivity analysis of the mapped risk estimates to variation in key model parameters. The tested parameter values were sampled from symmetric uniform distributions defined by a series of nested bounds (±5%, … , ±40%) around the parameters' initial values. The results suggest that the maximum annual spread distance, which governs long-distance dispersal, was by far the most sensitive parameter. At ±15% or larger variability bound increments for this parameter, there were noteworthy shifts in map risk values, but no other parameter had a major effect, even at wider bounds of variation. The methodology presented here is generic and can be used to assess the impact of uncertainties on the stability of pest risk maps as well as to identify geographic areas for which management decisions can be made confidently, regardless of uncertainty.     

Ensemble Habitat Mapping of Invasive Plant Species

Ensemble species distribution models combine the strengths of several species environmental matching models, while minimizing the weakness of any one model. Ensemble models may be particularly useful in risk analysis of recently arrived, harmful invasive species because species may not yet have spread to all suitable habitats, leaving species‐environment relationships difficult to determine. We tested five individual models (logistic regression, boosted regression trees, random forest, multivariate adaptive regression splines (MARS), and maximum entropy model or Maxent) and ensemble modeling for selected nonnative plant species in Yellowstone and Grand Teton National Parks, Wyoming; Sequoia and Kings Canyon National Parks, California, and areas of interior Alaska. The models are based on field data provided by the park staffs, combined with topographic, climatic, and vegetation predictors derived from satellite data. For the four invasive plant species tested, ensemble models were the only models that ranked in the top three models for both field validation and test data. Ensemble models may be more robust than individual species‐environment matching models for risk analysis.     

Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest

In pest risk assessment it is frequently necessary to make management decisions regarding emerging threats under severe uncertainty. Although risk maps provide useful decision support for invasive alien species, they rarely address knowledge gaps associated with the underlying risk model or how they may change the risk estimates. Failure to recognize uncertainty leads to risk‐ignorant decisions and miscalculation of expected impacts as well as the costs required to minimize these impacts. Here we use the information gap concept to evaluate the robustness of risk maps to uncertainties in key assumptions about an invading organism. We generate risk maps with a spatial model of invasion that simulates potential entries of an invasive pest via international marine shipments, their spread through a landscape, and establishment on a susceptible host. In particular, we focus on the question of how much uncertainty in risk model assumptions can be tolerated before the risk map loses its value. We outline this approach with an example of a forest pest recently detected in North America, Sirex noctilio Fabricius. The results provide a spatial representation of the robustness of predictions of S. noctilio invasion risk to uncertainty and show major geographic hotspots where the consideration of uncertainty in model parameters may change management decisions about a new invasive pest. We then illustrate how the dependency between the extent of uncertainties and the degree of robustness of a risk map can be used to select a surveillance network design that is most robust to knowledge gaps about the pest.     

Quantifying Potential Health Impacts of Cadmium in Cigarettes on Smoker Risk of Lung Cancer: A Portfolio-of-Mechanisms Approach

This article introduces an approach to estimating the uncertain potential effects on lung cancer risk of removing a particular constituent, cadmium (Cd), from cigarette smoke, given the useful but incomplete scientific information available about its modes of action. The approach considers normal cell proliferation; DNA repair inhibition in normal cells affected by initiating events; proliferation, promotion, and progression of initiated cells; and death or sparing of initiated and malignant cells as they are further transformed to become fully tumorigenic. Rather than estimating unmeasured model parameters by curve fitting to epidemiological or animal experimental tumor data, we attempt rough estimates of parameters based on their biological interpretations and comparison to corresponding genetic polymorphism data. The resulting parameter estimates are admittedly uncertain and approximate, but they suggest a portfolio approach to estimating impacts of removing Cd that gives usefully robust conclusions. This approach views Cd as creating a portfolio of uncertain health impacts that can be expressed as biologically independent relative risk factors having clear mechanistic interpretations. Because Cd can act through many distinct biological mechanisms, it appears likely (subjective probability greater than 40%) that removing Cd from cigarette smoke would reduce smoker risks of lung cancer by at least 10%, although it is possible (consistent with what is known) that the true effect could be much larger or smaller. Conservative estimates and assumptions made in this calculation suggest that the true impact could be greater for some smokers. This conclusion appears to be robust to many scientific uncertainties about Cd and smoking effects.     

The Application of Genetic Information for Regulatory Standard Setting Under the Clean Air Act: A Decision-Analytic Approach

In 2002, the U.S. Environmental Protection Agency (EPA) released an “Interim Policy on Genomics,” stating a commitment to developing guidance on the inclusion of genetic information in regulatory decision making. This statement was followed in 2004 by a document exploring the potential implications. Genetic information can play a key role in understanding and quantifying human susceptibility, an essential step in many of the risk assessments used to shape policy. For example, the federal Clean Air Act (CAA) requires EPA to set National Ambient Air Quality Standards (NAAQS) for criteria pollutants at levels to protect even sensitive populations from adverse health effects with an adequate margin of safety. Asthmatics are generally regarded as a sensitive population, yet substantial research gaps in understanding genetic susceptibility and disease have hindered quantitative risk analysis. This case study assesses the potential role of genomic information regarding susceptible populations in the NAAQS process for fine particulate matter (PM_2.5) under the CAA. In this initial assessment, we model the contribution of a single polymorphism to asthma risk and mortality risk; however, multiple polymorphisms and interactions (gene‐gene and gene‐environment) are known to play key roles in the disease process. We show that the impact of new information about susceptibility on estimates of population risk or average risk derived from large epidemiological studies depends on the circumstances. We also suggest that analysis of a single polymorphism, or other risk factor such as health status, may or may not change estimates of individual risk enough to alter a particular regulatory decision, but this depends on specific characteristics of the decision and risk information. We also show how new information about susceptibility in the context of the NAAQS for PM_2.5 could have a large impact on the estimated distribution of individual risk. This would occur if a group were consequently identified (based on genetic and/or disease status), that accounted for a disproportionate share of observed effects. Our results highlight certain conditions under which genetic information is likely to have an impact on risk estimates and the balance of costs and benefits within groups, and highlight critical research needs. As future studies explore more fully the relationship between exposure, genetic makeup, and disease status, the opportunity for genetic information and disease status to play pivotal roles in regulation can only increase.     

Application of data mining to minimize fire-induced domino effect risks

Data mining (DM) has been applied in many advanced science and technology fields, but it has still not been used for domino effect risk management to explore minimum risk scenarios. This work investigates the feasibility of DM in minimizing the risk of fire-induced domino effects in chemical processing facilities. Based on DM, an evidential failure mode and effects analysis (E-FMEA), which could bridge chemical facilities’ operational reliability and domino effect risk, is combined with fault tree analysis (FTA) for the occurrence risk modeling of loss of containment (LOC) event of chemical facilities, which is often the triggering point of fire-induced domino effects. Industry specific data such as reliability data, inspection records, and maintenance records are of great value to model the potential occurrence criticality of LOC. The data are used to characterize the LOC risk priority number (RPN) of chemical facilities through FTA and E-FMEA, search and statistics rules are proposed to mine inspection records to assess LOC risk factors. According to the RPN scores of facilities, inherent safety strategies to minimize risk via inventory control are proposed, and their effectiveness is tested using a well-known probit model. In this way, the approach proposes a unit-specific evidence-based risk minimization strategy for fire-induced domino effects. A case study demonstrates the capability of DM in the risk minimization of fire-induced domino effects.     

Interpersonal Amplification of Risk? Citizen Discussions and Their Impact on Perceptions of Risks and Benefits of a Biological Research Facility

Much risk communication research has demonstrated how mass media can influence individual risk perceptions, but lacks a comprehensive conceptual understanding of another key channel of communication: interpersonal discussion. Using the social amplification of risk as a theoretical framework, we consider the potential for discussions to function as amplification stations. We explore this possibility using data from a public opinion survey of residents living in potential locations for a new biological research facility in the United States. Controlling for a variety of key information variables, our results show that two dimensions of discussion—frequency and valence—have impacts on residents’ perceptions of the facility's benefits and its risks. We also explore the possibility that an individual's overall attitude moderates the effect of discussion on their perceptions of risks and benefits. Our results demonstrate the potential for discussions to operate as amplifiers or attenuators of perceptions of both risks and benefits.     

Revisions of the Fish Invasiveness Screening Kit (FISK) for its Application in Warmer Climatic Zones,with Particular Reference to Peninsular Florida

The initial version (v1) of the Fish Invasiveness Scoring Kit (FISK) was adapted from the Weed Risk Assessment of Pheloung, Williams, and Halloy to assess the potential invasiveness of nonnative freshwater fishes in the United Kingdom. Published applications of FISK v1 have been primarily in temperate‐zone countries (Belgium, Belarus, and Japan), so the specificity of this screening tool to that climatic zone was not noted until attempts were made to apply it in peninsular Florida. To remedy this shortcoming, the questions and guidance notes of FISK v1 were reviewed and revised to improve clarity and extend its applicability to broader climatic regions, resulting in changes to 36 of the 49 questions. In addition, upgrades were made to the software architecture of FISK to improve overall computational speed as well as graphical user interface flexibility and friendliness. We demonstrate the process of screening a fish species using FISK v2 in a realistic management scenario by assessing the Barcoo grunter Scortum barcoo (Terapontidae), a species whose management concerns are related to its potential use for aquaponics in Florida. The FISK v2 screening of Barcoo grunter placed the species into the lower range of medium risk (score = 5), suggesting it is a permissible species for use in Florida under current nonnative species regulations. Screening of the Barcoo grunter illustrates the usefulness of FISK v2 as a proactive tool serving to inform risk management decisions, but the low level of confidence associated with the assessment highlighted a dearth of critical information on this species.     

Tiered Chemical Testing: A Value of Information Approach

In December 2000 the EPA initiated the Voluntary Children's Chemical Evaluation Program (VCCEP) by asking manufacturers to voluntarily sponsor toxicological testing in a tiered process for 23 chemicals selected for the pilot phase. The tiered nature of the VCCEP pilot program creates the need for clearly defined criteria for determining when information is sufficient to assess the potential risks to children. This raises questions about how to determine the "adequacy" of the existing information and assess the need to undertake efforts to reduce uncertainty (through further testing). This article applies a value of information analysis approach to determine adequacy by modeling how toxicological and exposure data collected through the VCCEP may be used to inform risk management decisions. The analysis demonstrates the importance of information about the exposure level and control costs in making decisions regarding further toxicological testing. This article accounts for the cost of delaying control action and identifies the optimal testing strategy for a constrained decisionmaker who, absent applicable human data, cannot regulate without bioassay data on a specific chemical. It also quantifies the differences in optimal testing strategy for three decision criteria: maximizing societal net benefits, ensuring maximum exposure control while net benefits are positive (i.e., benefits outweigh costs), and controlling to the maximum extent technologically feasible while the lifetime risk of cancer exceeds a specific level of risk. Finally, this article shows the large differences that exist in net benefits between the three criteria for the range of exposure levels where the optimal actions differ.     

First Application of FISK,the Freshwater Fish Invasiveness Screening Kit,in Northern Europe: Example of Southern Finland

The climatic conditions of north temperate countries pose unique influences on the rates of invasion and the potential adverse impacts of non‐native species. Methods are needed to evaluate these risks, beginning with the pre‐screening of non‐native species for potential invasives. Recent improvements to the Fish Invasiveness Scoring Kit (FISK) have provided a means (i.e., FISK v2) of identifying potentially invasive non‐native freshwater fishes in virtually all climate zones. In this study, FISK is applied for the first time in a north temperate country, southern Finland, and calibrated to determine the appropriate threshold score for fish species that are likely to pose a high risk of being invasive in this risk assessment area. The threshold between “medium” and “high” risk was determined to be 22.5, which is slightly higher than the original threshold for the United Kingdom (i.e., 19) and that determined for a FISK application in southern Japan (19.8). This underlines the need to calibrate such decision‐support tools for the different areas where they are employed. The results are evaluated in the context of current management strategies in Finland regarding non‐native fishes.     

Integrated Analysis: Combining Risk and Economic Assessments While Preserving the Separation of Powers

This article presents a process for an integrated policy analysis that combines risk assessment and benefit-cost analysis. This concept, which explicitly combines the two types of related analyses, seems to contradict the long-accepted risk analysis paradigm of separating risk assessment and risk management since benefit-cost analysis is generally considered to be a part of risk management. Yet that separation has become a problem because benefit-cost analysis uses risk assessment results as a starting point and considerable debate over the last several years focused on the incompatibility of the use of upper bounds or "safe" point estimates in many risk assessments with benefit-cost analysis. The problem with these risk assessments is that they ignore probabilistic information. As advanced probabilistic techniques for risk assessment emerge and economic analysts receive distributions of risks instead of point estimates, the artificial separation between risk analysts and the economic/decision analysts complicates the overall analysis. In addition, recent developments in countervailing risk theory suggest that combining the risk and benefit-cost analyses is required to fully understand the complexity of choices and tradeoffs faced by the decisionmaker. This article also argues that the separation of analysis and management is important, but that benefit-cost analysis has been wrongly classified into the risk management category and that the analytical effort associated with understanding the economic impacts of risk reduction actions need to be part of a broader risk assessment process.     

Modeling the Risks of Nonindigenous Species Introductions Using a Patch-Dynamics Approach Incorporating Contaminant Effects as a Disturbance

The establishment and spread of invasive or nonindigenous species has caused concern from stakeholders in affected areas, and has prompted many field and modeling studies. We used stochastic two species, circular three patch dynamic models to investigate the patterns of invasion and impacts upon the affected species. Both persistent and degradable toxicants were incorporated as parts of the model system to act as disturbance regimens. There is a clear series of patterns that result from these simulations. Competition increases population variability, but decreases the number of distinct outcomes possible from the same initial conditions. Isolation of the patch of the introduction was the main determinant of successful establishment through a process we call the beachhead effect. Coexistence of species was often possible in local patches, contrary to the analytical solutions of Lotka-Volterra equations and numerous modeling studies. Contaminants and their resultant disturbances are important as contributors to the stochastic nature of models. The stochasticity leads to a variety of outcomes from some sets of initial conditions. Different outcomes have different probabilities of occurrence and are dependent upon the specific initial conditions of the simulation. A clear pattern that is apparent is the "beachhead effect," where the invasive establishes a population within a relatively remote patch before migrating to the remainder of the landscape. We make predictions and provide specific research hypotheses as to the causes and effects of invasive species establishment, spread, and impacts.     

Assessing the Benefits and Costs of Homeland Security Research: A Risk-Informed Methodology with Applications for the U.S. Coast Guard

This article describes a methodology for risk-informed benefit–cost analyses of homeland security research products. The methodology is field-tested with 10 research products developed for the U.S. Coast Guard. Risk-informed benefit–cost analysis is a tool for risk management that integrates elements of risk analysis, decision analysis, and benefit–cost analysis. The cost analysis methodology includes a full-cost accounting of research projects, starting with initial fundamental research costs and extending to the costs of implementation of the research products and, where applicable, training, maintenance, and upgrade costs. The benefits analysis methodology is driven by changes in costs and risks leading to five alternative models: cost savings at the same level of security, increased security at the same cost, signal detection improvements, risk reduction by deterrence, and value of information. The U.S. Coast Guard staff selected 10 research projects to test and generalize the methodology. Examples include tools to improve the detection of explosives, reduce the costs of harbor patrols, and provide better predictions of hurricane wind speeds and floods. Benefits models and estimates varied by research project and many input parameters of the benefit estimates were highly uncertain, so risk analysis for sensitivity testing and simulation was important. Aggregating across the 10 research products, we found an overall median net present value of about $385 million, with a range from $54 million (5th percentile) to $877 million (95th percentile). Lessons learned are provided for future applications.