Model Selection and Estimation with Quantal‐Response Data in Benchmark Risk Assessment

This article describes several approaches for estimating the benchmark dose (BMD) in a risk assessment study with quantal dose‐response data and when there are competing model classes for the dose‐response function. Strategies involving a two‐step approach, a model‐averaging approach, a focused‐inference approach, and a nonparametric approach based on a PAVA‐based estimator of the dose‐response function are described and compared. Attention is raised to the perils involved in data “double‐dipping” and the need to adjust for the model‐selection stage in the estimation procedure. Simulation results are presented comparing the performance of five model selectors and eight BMD estimators. An illustration using a real quantal‐response data set from a carcinogenecity study is provided.     

Model Uncertainty and Risk Estimation for Experimental Studies of Quantal Responses

Experimental animal studies often serve as the basis for predicting risk of adverse responses in humans exposed to occupational hazards. A statistical model is applied to exposure-response data and this fitted model may be used to obtain estimates of the exposure associated with a specified level of adverse response. Unfortunately, a number of different statistical models are candidates for fitting the data and may result in wide ranging estimates of risk. Bayesian model averaging (BMA) offers a strategy for addressing uncertainty in the selection of statistical models when generating risk estimates. This strategy is illustrated with two examples: applying the multistage model to cancer responses and a second example where different quantal models are fit to kidney lesion data. BMA provides excess risk estimates or benchmark dose estimates that reflects model uncertainty.     

A Simulation Study of Quantitative Risk Assessment for Bivariate Continuous Outcomes

The neurotoxic effects of chemical agents are often investigated in controlled studies on rodents, with binary and continuous multiple endpoints routinely collected. One goal is to conduct quantitative risk assessment to determine safe dose levels. Yu and Catalano (2005) describe a method for quantitative risk assessment for bivariate continuous outcomes by extending a univariate method of percentile regression. The model is likelihood based and allows for separate dose‐response models for each outcome while accounting for the bivariate correlation. The approach to benchmark dose (BMD) estimation is analogous to that for quantal data without having to specify arbitrary cutoff values. In this article, we evaluate the behavior of the BMD relative to background rates, sample size, level of bivariate correlation, dose‐response trend, and distributional assumptions. Using simulations, we explore the effects of these factors on the resulting BMD and BMDL distributions. In addition, we illustrate our method with data from a neurotoxicity study of parathion exposure in rats.     

A Bayesian Semiparametric Model for Radiation Dose‐Response Estimation

In evaluating the risk of exposure to health hazards, characterizing the dose‐response relationship and estimating acceptable exposure levels are the primary goals. In analyses of health risks associated with exposure to ionizing radiation, while there is a clear agreement that moderate to high radiation doses cause harmful effects in humans, little has been known about the possible biological effects at low doses, for example, below 0.1 Gy, which is the dose range relevant to most radiation exposures of concern today. A conventional approach to radiation dose‐response estimation based on simple parametric forms, such as the linear nonthreshold model, can be misleading in evaluating the risk and, in particular, its uncertainty at low doses. As an alternative approach, we consider a Bayesian semiparametric model that has a connected piece‐wise‐linear dose‐response function with prior distributions having an autoregressive structure among the random slope coefficients defined over closely spaced dose categories. With a simulation study and application to analysis of cancer incidence data among Japanese atomic bomb survivors, we show that this approach can produce smooth and flexible dose‐response estimation while reasonably handling the risk uncertainty at low doses and elsewhere. With relatively few assumptions and modeling options to be made by the analyst, the method can be particularly useful in assessing risks associated with low‐dose radiation exposures.     

Impact of Acquired Immunity and Dose‐Dependent Probability of Illness on Quantitative Microbial Risk Assessment

Dose‐response models in microbial risk assessment consider two steps in the process ultimately leading to illness: from exposure to (asymptomatic) infection, and from infection to (symptomatic) illness. Most data and theoretical approaches are available for the exposure‐infection step; the infection‐illness step has received less attention. Furthermore, current microbial risk assessment models do not account for acquired immunity. These limitations may lead to biased risk estimates. We consider effects of both dose dependency of the conditional probability of illness given infection, and acquired immunity to risk estimates, and demonstrate their effects in a case study on exposure to Campylobacter jejuni. To account for acquired immunity in risk estimates, an inflation factor is proposed. The inflation factor depends on the relative rates of loss of protection over exposure. The conditional probability of illness given infection is based on a previously published model, accounting for the within‐host dynamics of illness. We find that at low (average) doses, the infection‐illness model has the greatest impact on risk estimates, whereas at higher (average) doses and/or increased exposure frequencies, the acquired immunity model has the greatest impact. The proposed models are strongly nonlinear, and reducing exposure is not expected to lead to a proportional decrease in risk and, under certain conditions, may even lead to an increase in risk. The impact of different dose‐response models on risk estimates is particularly pronounced when introducing heterogeneity in the population exposure distribution.     

Dose‐Response Modeling with Summary Data from Developmental Toxicity Studies

Dose‐response analysis of binary developmental data (e.g., implant loss, fetal abnormalities) is best done using individual fetus data (identified to litter) or litter‐specific statistics such as number of offspring per litter and proportion abnormal. However, such data are not often available to risk assessors. Scientific articles usually present only dose‐group summaries for the number or average proportion abnormal and the total number of fetuses. Without litter‐specific data, it is not possible to estimate variances correctly (often characterized as a problem of overdispersion, intralitter correlation, or “litter effect”). However, it is possible to use group summary data when the design effect has been estimated for each dose group. Previous studies have demonstrated useful dose‐response and trend test analyses based on design effect estimates using litter‐specific data from the same study. This simplifies the analysis but does not help when litter‐specific data are unavailable. In the present study, we show that summary data on fetal malformations can be adjusted satisfactorily using estimates of the design effect based on historical data. When adjusted data are then analyzed with models designed for binomial responses, the resulting benchmark doses are similar to those obtained from analyzing litter‐level data with nested dichotomous models.     

Peak Exposures in Epidemiologic Studies and Cancer Risks: Considerations for Regulatory Risk Assessment

We review approaches for characterizing “peak” exposures in epidemiologic studies and methods for incorporating peak exposure metrics in dose–response assessments that contribute to risk assessment. The focus was on potential etiologic relations between environmental chemical exposures and cancer risks. We searched the epidemiologic literature on environmental chemicals classified as carcinogens in which cancer risks were described in relation to “peak” exposures. These articles were evaluated to identify some of the challenges associated with defining and describing cancer risks in relation to peak exposures. We found that definitions of peak exposure varied considerably across studies. Of nine chemical agents included in our review of peak exposure, six had epidemiologic data used by the U.S. Environmental Protection Agency (US EPA) in dose–response assessments to derive inhalation unit risk values. These were benzene, formaldehyde, styrene, trichloroethylene, acrylonitrile, and ethylene oxide. All derived unit risks relied on cumulative exposure for dose–response estimation and none, to our knowledge, considered peak exposure metrics. This is not surprising, given the historical linear no‐threshold default model (generally based on cumulative exposure) used in regulatory risk assessments. With newly proposed US EPA rule language, fuller consideration of alternative exposure and dose–response metrics will be supported. “Peak” exposure has not been consistently defined and rarely has been evaluated in epidemiologic studies of cancer risks. We recommend developing uniform definitions of “peak” exposure to facilitate fuller evaluation of dose response for environmental chemicals and cancer risks, especially where mechanistic understanding indicates that the dose response is unlikely linear and that short‐term high‐intensity exposures increase risk.     

Harnessing the Theoretical Foundations of the Exponential and Beta‐Poisson Dose‐Response Models to Quantify Parameter Uncertainty Using Markov Chain Monte Carlo

Dose‐response models are the essential link between exposure assessment and computed risk values in quantitative microbial risk assessment, yet the uncertainty that is inherent to computed risks because the dose‐response model parameters are estimated using limited epidemiological data is rarely quantified. Second‐order risk characterization approaches incorporating uncertainty in dose‐response model parameters can provide more complete information to decisionmakers by separating variability and uncertainty to quantify the uncertainty in computed risks. Therefore, the objective of this work is to develop procedures to sample from posterior distributions describing uncertainty in the parameters of exponential and beta‐Poisson dose‐response models using Bayes's theorem and Markov Chain Monte Carlo (in OpenBUGS). The theoretical origins of the beta‐Poisson dose‐response model are used to identify a decomposed version of the model that enables Bayesian analysis without the need to evaluate Kummer confluent hypergeometric functions. Herein, it is also established that the beta distribution in the beta‐Poisson dose‐response model cannot address variation among individual pathogens, criteria to validate use of the conventional approximation to the beta‐Poisson model are proposed, and simple algorithms to evaluate actual beta‐Poisson probabilities of infection are investigated. The developed MCMC procedures are applied to analysis of a case study data set, and it is demonstrated that an important region of the posterior distribution of the beta‐Poisson dose‐response model parameters is attributable to the absence of low‐dose data. This region includes beta‐Poisson models for which the conventional approximation is especially invalid and in which many beta distributions have an extreme shape with questionable plausibility.     

Fractional Poisson—A Simple Dose‐Response Model for Human Norovirus

This study utilizes old and new Norovirus (NoV) human challenge data to model the dose‐response relationship for human NoV infection. The combined data set is used to update estimates from a previously published beta‐Poisson dose‐response model that includes parameters for virus aggregation and for a beta‐distribution that describes variable susceptibility among hosts. The quality of the beta‐Poisson model is examined and a simpler model is proposed. The new model (fractional Poisson) characterizes hosts as either perfectly susceptible or perfectly immune, requiring a single parameter (the fraction of perfectly susceptible hosts) in place of the two‐parameter beta‐distribution. A second parameter is included to account for virus aggregation in the same fashion as it is added to the beta‐Poisson model. Infection probability is simply the product of the probability of nonzero exposure (at least one virus or aggregate is ingested) and the fraction of susceptible hosts. The model is computationally simple and appears to be well suited to the data from the NoV human challenge studies. The model's deviance is similar to that of the beta‐Poisson, but with one parameter, rather than two. As a result, the Akaike information criterion favors the fractional Poisson over the beta‐Poisson model. At low, environmentally relevant exposure levels (<100), estimation error is small for the fractional Poisson model; however, caution is advised because no subjects were challenged at such a low dose. New low‐dose data would be of great value to further clarify the NoV dose‐response relationship and to support improved risk assessment for environmentally relevant exposures.     

Comparison of Risk Predicted by Multiple Norovirus Dose–Response Models and Implications for Quantitative Microbial Risk Assessment

The application of quantitative microbial risk assessments (QMRAs) to understand and mitigate risks associated with norovirus is increasingly common as there is a high frequency of outbreaks worldwide. A key component of QMRA is the dose–response analysis, which is the mathematical characterization of the association between dose and outcome. For Norovirus, multiple dose–response models are available that assume either a disaggregated or an aggregated intake dose. This work reviewed the dose–response models currently used in QMRA, and compared predicted risks from waterborne exposures (recreational and drinking) using all available dose–response models. The results found that the majority of published QMRAs of norovirus use the ₁F₁ hypergeometric dose–response model with α = 0.04, β = 0.055. This dose–response model predicted relatively high risk estimates compared to other dose–response models for doses in the range of 1–1,000 genomic equivalent copies. The difference in predicted risk among dose–response models was largest for small doses, which has implications for drinking water QMRAs where the concentration of norovirus is low. Based on the review, a set of best practices was proposed to encourage the careful consideration and reporting of important assumptions in the selection and use of dose–response models in QMRA of norovirus. Finally, in the absence of one best norovirus dose–response model, multiple models should be used to provide a range of predicted outcomes for probability of infection.     

Animal and Human Dose‐Response Models for Brucella Species

Human Brucellosis is one of the most common zoonotic diseases worldwide. Disease transmission often occurs through the handling of domestic livestock, as well as ingestion of unpasteurized milk and cheese, but can have enhanced infectivity if aerosolized. Because there is no human vaccine available, rising concerns about the threat of Brucellosis to human health and its inclusion in the Center for Disease Control's Category B Bioterrorism/Select Agent List make a better understanding of the dose‐response relationship of this microbe necessary. Through an extensive peer‐reviewed literature search, candidate dose‐response data were appraised so as to surpass certain standards for quality. The statistical programming language, “R,” was used to compute the maximum likelihood estimation to fit two models, the exponential and the approximate beta‐Poisson (widely used for quantitative risk assessment) to dose‐response data. Dose‐response models were generated for prevalent species of Brucella: Br. suis, Br. melitensis, and Br. abortus. Dose‐response models were created for aerosolized Br. suis exposure to guinea pigs from pooled studies. A parallel model for guinea pigs inoculated through both aerosol and subcutaneous routes with Br. melitensis showed that the median infectious dose corresponded to a 30 colony‐forming units (CFU) dose of Br. suis, much less than the N₅₀ dose of about 94 CFU for Br. melitensis organisms. When Br. melitensis was tested subcutaneously on mice, the N₅₀ dose was higher, 1,840 CFU. A dose‐response model was constructed from pooled data for mice, rhesus macaques, and humans inoculated through three routes (subcutaneously/aerosol/intradermally) with Br. melitensis.     

Dose‐Response Models Incorporating Aerosol Size Dependency for Francisella tularensis

The effect of bioaerosol size was incorporated into predictive dose‐response models for the effects of inhaled aerosols of Francisella tularensis (the causative agent of tularemia) on rhesus monkeys and guinea pigs with bioaerosol diameters ranging between 1.0 and 24 μm. Aerosol‐size‐dependent models were formulated as modification of the exponential and β‐Poisson dose‐response models and model parameters were estimated using maximum likelihood methods and multiple data sets of quantal dose‐response data for which aerosol sizes of inhaled doses were known. Analysis of F. tularensis dose‐response data was best fit by an exponential dose‐response model with a power function including the particle diameter size substituting for the rate parameter k scaling the applied dose. There were differences in the pathogen's aerosol‐size‐dependence equation and models that better represent the observed dose‐response results than the estimate derived from applying the model developed by the International Commission on Radiological Protection (ICRP, 1994) that relies on differential regional lung deposition for human particle exposure.     

Modeling for Risk Assessment of Neurotoxic Effects

The regulation of noncancer toxicants, including neurotoxicants, has usually been based upon a reference dose (allowable daily intake). A reference dose is obtained by dividing a no-observed-effect level by uncertainty (safety) factors to account for intraspecies and interspecies sensitivities to a chemical. It is assumed that the risk at the reference dose is negligible, but no attempt generally is made to estimate the risk at the reference dose. A procedure is outlined that provides estimates of risk as a function of dose. The first step is to establish a mathematical relationship between a biological effect and the dose of a chemical. Knowledge of biological mechanisms and/or pharmacokinetics can assist in the choice of plausible mathematical models. The mathematical model provides estimates of average responses as a function of dose. Secondly, estimates of risk require selection of a distribution of individual responses about the average response given by the mathematical model. In the case of a normal or lognormal distribution, only an estimate of the standard deviation is needed. The third step is to define an adverse level for a response so that the probability (risk) of exceeding that level can be estimated as a function of dose. Because a firm response level often cannot be established at which adverse biological effects occur, it may be necessary to at least establish an abnormal response level that only a small proportion of individuals would exceed in an unexposed group. That is, if a normal range of responses can be established, then the probability (risk) of abnormal responses can be estimated. In order to illustrate this process, measures of the neurotransmitter serotonin and its metabolite 5-hydroxyindoleacetic acid in specific areas of the brain of rats and monkeys are analyzed after exposure to the neurotoxicant methylene-dioxymethamphetamine. These risk estimates are compared with risk estimates from the quantal approach in which animals are classified as either abnormal or not depending upon abnormal serotonin levels.     

Development of an Interspecies Nested Dose‐Response Model for Mycobacterium avium subspecies paratuberculosis

Mycobacterium avium subspecies paratuberculosis (MAP) causes chronic inflammation of the intestines in humans, ruminants, and other species. It is the causative agent of Johne's disease in cattle, and has been implicated as the causative agent of Crohn's disease in humans. To date, no quantitative microbial risk assessment (QMRA) for MAP utilizing a dose‐response function exists. The objective of this study is to develop a nested dose‐response model for infection from oral exposure to MAP utilizing data from the peer‐reviewed literature. Four studies amenable to dose‐response modeling were identified in the literature search and optimized to the one‐parameter exponential or two‐parameter beta‐Poisson dose‐response models. A nesting analysis was performed on all permutations of the candidate data sets to determine the acceptability of pooling data sets across host species. Three of four data sets exhibited goodness of fit to at least one model. All three data sets exhibited good fit to the beta‐Poisson model, and one data set exhibited goodness of fit, and best fit, to the exponential model. Two data sets were successfully nested using the beta‐Poisson model with parameters α = 0.0978 and N₅₀ = 2.70 × 10² CFU. These data sets were derived from sheep and red deer host species, indicating successful interspecies nesting, and demonstrate the highly infective nature of MAP. The nested dose‐response model described should be used for future QMRA research regarding oral exposure to MAP.     

Classic Dose‐Response and Time Postinoculation Models for Leptospira

Leptospirosis is a preeminent zoonotic disease concentrated in tropical areas, and prevalent in both industrialized and rural settings. Dose‐response models were generated from 22 data sets reported in 10 different studies. All of the selected studies used rodent subjects, primarily hamsters, with the predominant endpoint as mortality with the challenge strain administered intraperitoneally. Dose‐response models based on a single evaluation postinfection displayed median lethal dose (LD₅₀) estimates that ranged between 1 and 10⁷ leptospirae depending upon the strain's virulence and the period elapsed since the initial exposure inoculation. Twelve of the 22 data sets measured the number of affected subjects daily over an extended period, so dose‐response models with time‐dependent parameters were estimated. Pooling between data sets produced seven common dose‐response models and one time‐dependent model. These pooled common models had data sets with different test subject hosts, and between disparate leptospiral strains tested on identical hosts. Comparative modeling was done with parallel tests to test the effects of a single different variable of either strain or test host and quantify the difference by calculating a dose multiplication factor. Statistical pooling implies that the mechanistic processes of leptospirosis can be represented by the same dose‐response model for different experimental infection tests even though they may involve different host species, routes, and leptospiral strains, although the cause of this pathophysiological phenomenon has not yet been identified.     

QMRA for Drinking Water: 2. The Effect of Pathogen Clustering in Single‐Hit Dose‐Response Models

Spatial and/or temporal clustering of pathogens will invalidate the commonly used assumption of Poisson‐distributed pathogen counts (doses) in quantitative microbial risk assessment. In this work, the theoretically predicted effect of spatial clustering in conventional “single‐hit” dose‐response models is investigated by employing the stuttering Poisson distribution, a very general family of count distributions that naturally models pathogen clustering and contains the Poisson and negative binomial distributions as special cases. The analysis is facilitated by formulating the dose‐response models in terms of probability generating functions. It is shown formally that the theoretical single‐hit risk obtained with a stuttering Poisson distribution is lower than that obtained with a Poisson distribution, assuming identical mean doses. A similar result holds for mixed Poisson distributions. Numerical examples indicate that the theoretical single‐hit risk is fairly insensitive to moderate clustering, though the effect tends to be more pronounced for low mean doses. Furthermore, using Jensen's inequality, an upper bound on risk is derived that tends to better approximate the exact theoretical single‐hit risk for highly overdispersed dose distributions. The bound holds with any dose distribution (characterized by its mean and zero inflation index) and any conditional dose‐response model that is concave in the dose variable. Its application is exemplified with published data from Norovirus feeding trials, for which some of the administered doses were prepared from an inoculum of aggregated viruses. The potential implications of clustering for dose‐response assessment as well as practical risk characterization are discussed.     

Aldrin and Dieldrin: A Reevaluation of the Cancer and Noncancer Dose‐Response Assessments

The dose‐response analyses of cancer and noncancer health effects of aldrin and dieldrin were evaluated using current methodology, including benchmark dose analysis and the current U.S. Environmental Protection Agency (U.S. EPA) guidance on body weight scaling and uncertainty factors. A literature review was performed to determine the most appropriate adverse effect endpoints. Using current methodology and information, the estimated reference dose values were 0.0001 and 0.00008 mg/kg‐day for aldrin and dieldrin, respectively. The estimated cancer slope factors for aldrin and dieldrin were 3.4 and 7.0 (mg/kg‐day)^?1, respectively (i.e., about 5‐ and 2.3‐fold lower risk than the 1987 U.S. EPA assessments). Because aldrin and dieldrin are no longer used as pesticides in the United States, they are presumed to be a low priority for additional review by the U.S. EPA. However, because they are persistent and still detected in environmental samples, quantitative risk assessments based on the best available methods are required. Recent epidemiologic studies do not demonstrate a causal association between aldrin and dieldrin and human cancer risk. The proposed reevaluations suggest that these two compounds pose a lower human health risk than currently reported by the U.S. EPA.     

Development of Dose‐Response Models to Predict the Relationship for Human Toxoplasma gondii Infection Associated with Meat Consumption

Toxoplasma gondii is a protozoan parasite that is responsible for approximately 24% of deaths attributed to foodborne pathogens in the United States. It is thought that a substantial portion of human T. gondii infections is acquired through the consumption of meats. The dose‐response relationship for human exposures to T. gondii‐infected meat is unknown because no human data are available. The goal of this study was to develop and validate dose‐response models based on animal studies, and to compute scaling factors so that animal‐derived models can predict T. gondii infection in humans. Relevant studies in literature were collected and appropriate studies were selected based on animal species, stage, genotype of T. gondii, and route of infection. Data were pooled and fitted to four sigmoidal‐shaped mathematical models, and model parameters were estimated using maximum likelihood estimation. Data from a mouse study were selected to develop the dose‐response relationship. Exponential and beta‐Poisson models, which predicted similar responses, were selected as reasonable dose‐response models based on their simplicity, biological plausibility, and goodness fit. A confidence interval of the parameter was determined by constructing 10,000 bootstrap samples. Scaling factors were computed by matching the predicted infection cases with the epidemiological data. Mouse‐derived models were validated against data for the dose‐infection relationship in rats. A human dose‐response model was developed as P (d) = 1–exp (–0.0015 × 0.005 × d) or P (d) = 1–(1 + d × 0.003 / 582.414)^?1.479. Both models predict the human response after consuming T. gondii‐infected meats, and provide an enhanced risk characterization in a quantitative microbial risk assessment model for this pathogen.     

Bayesian Development of a Dose‐Response Model for Aspergillus fumigatus and Invasive Aspergillosis

Invasive aspergillosis (IA) is a major cause of mortality in immunocompromized hosts, most often consecutive to the inhalation of spores of Aspergillus. However, the relationship between Aspergillus concentration in the air and probability of IA is not quantitatively known. In this study, this relationship was examined in a murine model of IA. Immunosuppressed Balb/c mice were exposed for 60 minutes at day 0 to an aerosol of A. fumigatus spores (Af293 strain). At day 10, IA was assessed in mice by quantitative culture of the lungs and galactomannan dosage. Fifteen separate nebulizations with varying spore concentrations were performed. Rates of IA ranged from 0% to 100% according to spore concentrations. The dose‐response relationship between probability of infection and spore exposure was approximated using the exponential model and the more flexible beta‐Poisson model. Prior distributions of the parameters of the models were proposed then updated with data in a Bayesian framework. Both models yielded close median dose‐responses of the posterior distributions for the main parameter of the model, but with different dispersions, either when the exposure dose was the concentration in the nebulized suspension or was the estimated quantity of spores inhaled by a mouse during the experiment. The median quantity of inhaled spores that infected 50% of mice was estimated at 1.8 × 10⁴ and 3.2 × 10⁴ viable spores in the exponential and beta‐Poisson models, respectively. This study provides dose‐response parameters for quantitative assessment of the relationship between airborne exposure to the reference A. fumigatus strain and probability of IA in immunocompromized hosts.     

Framework for Microbial Food‐Safety Risk Assessments Amenable to Bayesian Modeling

Regulatory agencies often perform microbial risk assessments to evaluate the change in the number of human illnesses as the result of a new policy that reduces the level of contamination in the food supply. These agencies generally have regulatory authority over the production and retail sectors of the farm‐to‐table continuum. Any predicted change in contamination that results from new policy that regulates production practices occurs many steps prior to consumption of the product. This study proposes a framework for conducting microbial food‐safety risk assessments; this framework can be used to quantitatively assess the annual effects of national regulatory policies. Advantages of the framework are that estimates of human illnesses are consistent with national disease surveillance data (which are usually summarized on an annual basis) and some of the modeling steps that occur between production and consumption can be collapsed or eliminated. The framework leads to probabilistic models that include uncertainty and variability in critical input parameters; these models can be solved using a number of different Bayesian methods. The Bayesian synthesis method performs well for this application and generates posterior distributions of parameters that are relevant to assessing the effect of implementing a new policy. An example, based on Campylobacter and chicken, estimates the annual number of illnesses avoided by a hypothetical policy; this output could be used to assess the economic benefits of a new policy. Empirical validation of the policy effect is also examined by estimating the annual change in the numbers of illnesses observed via disease surveillance systems.