A Comparison of Methods for Estimating the Benchmark Dose Based on Overdispersed Data from Developmental Toxicity Studies

Developmental anomalies resulting from prenatal toxicity can be manifested in terms of both malformations among surviving offspring and prenatal death. Although these two endpoints have traditionally been analyzed separately in the assessment of risk, multivariate methods of risk characterization have recently been proposed. We examined this and other issues in developmental toxicity risk assessment by evaluating the accuracy and precision of estimates of the effective dose ( ED ₀₅) and the benchmark dose ( BMD ₀₅) using computer simulation. Our results indicated that different variance structures (Dirichlet-trinomial and generalized linear model) used to characterize overdispersion yielded comparable results when fitting joint dose response models based on generalized estimating equations. (The choice of variance structure in separate modeling was also not critical.) However, using the Rao-Scott transformation to eliminate overdispersion tended to produce estimates of the ED ₀₅ with reduced bias and mean squared error. Because joint modeling ensures that the ED ₀₅ for overall toxicity (based on both malformations and prenatal death) is always less than the ED ₀₅ for either malformations or prenatal death, joint modeling is preferred to separate modeling for risk assessment purposes.     

A Simulation Study of the Influence of Study Design on the Estimation of Benchmark Doses for Developmental Toxicity1

The benchmark dose (BMD)⁴ approach is emerging as replacement to determination of the No Observed Adverse Effect Level (NOAEL) in noncancer risk assessment. This possibility raises the issue as to whether current study designs for endpoints such as developmental toxicity, optimized for detecting pair wise comparisons, could be improved for the purpose of calculating BMDs. In this paper, we examine various aspects of study design (number of dose groups, dose spacing, dose placement, and sample size per dose group) on BMDs for two endpoints of developmental toxicity (the incidence of abnormalities and of reduced fetal weight). Design performance was judged by the mean-squared error (reflective of the variance and bias) of the maximum likelihood estimate (MLE) from the log-logistic model of the 5% added risk level (the likely target risk for a benchmark calculation), as well as by the length of its 95% confidence interval (the lower value of which is the BMD). We found that of the designs evaluated, the best results were obtained when two dose levels had response rates above the background level, one of which was near the ED₀₅, were present. This situation is more likely to occur with more, rather than fewer dose levels per experiment. In this instance, there was virtually no advantage in increasing the sample size from 10 to 20 litters per dose group. If neither of the two dose groups with response rates above the background level was near the ED₀₅, satisfactory results were also obtained, but the BMDs tended to be more conservative (i.e., lower). If only one dose level with a response rate above the background level was present, and it was near the ED₀₅, reasonable results for the MLE and BMD were obtained, but here we observed benefits of larger dose group sizes. The poorest results were obtained when only a single group with an elevated response rate was present, and the response rate was much greater than the ED₀₅. The results indicate that while the benchmark dose approach is readily applicable to the standard study designs and generally observed dose-responses in developmental assays, some minor design modifications would increase the accuracy and precision of the BMD.     

A Simulation Study of Quantitative Risk Assessment for Bivariate Continuous Outcomes

The neurotoxic effects of chemical agents are often investigated in controlled studies on rodents, with binary and continuous multiple endpoints routinely collected. One goal is to conduct quantitative risk assessment to determine safe dose levels. Yu and Catalano (2005) describe a method for quantitative risk assessment for bivariate continuous outcomes by extending a univariate method of percentile regression. The model is likelihood based and allows for separate dose‐response models for each outcome while accounting for the bivariate correlation. The approach to benchmark dose (BMD) estimation is analogous to that for quantal data without having to specify arbitrary cutoff values. In this article, we evaluate the behavior of the BMD relative to background rates, sample size, level of bivariate correlation, dose‐response trend, and distributional assumptions. Using simulations, we explore the effects of these factors on the resulting BMD and BMDL distributions. In addition, we illustrate our method with data from a neurotoxicity study of parathion exposure in rats.     

Optimal Designs for Estimating the Effective Dose in Developmental Toxicity Experiments

Recent advances in risk assessment have led to the development of joint dose-response models to describe prenatal death and fetal malformation rates in developmental toxicity experiments. These models can be used to estimate the effective dose corresponding to a 5% excess risk for both these toxicological endpoints, as well as for overall toxicity. In this article, we develop optimal experimental designs for the estimation of the effective dose for developmental toxicity using joint Weibull dose-response models for prenatal death and fetal malformation. Based on an extended series of developmental studies, near-optimal designs for prenatal death, malformation, and overall toxicity were found to involve three dose groups: an unexposed control group, a high dose equal to the maximum tolerated dose, and a low dose above or comparable to the effective dose. The effect on the optimal designs of changing the number of implants and the degree of intra-litter correlation is also investigated. Although the optimal design has only three dose groups in most cases, practical considerations involving model lack of fit and estimation of the shape of the dose-response curve suggest that, in practice, suboptimal designs with more than three doses will often be preferred.     

Bootstrap Estimation of Benchmark Doses and Confidence Limits with Clustered Quantal Data

The benchmark dose (BMD) is an exposure level that would induce a small risk increase (BMR level) above the background. The BMD approach to deriving a reference dose for risk assessment of noncancer effects is advantageous in that the estimate of BMD is not restricted to experimental doses and utilizes most available dose-response information. To quantify statistical uncertainty of a BMD estimate, we often calculate and report its lower confidence limit (i.e., BMDL), and may even consider it as a more conservative alternative to BMD itself. Computation of BMDL may involve normal confidence limits to BMD in conjunction with the delta method. Therefore, factors, such as small sample size and nonlinearity in model parameters, can affect the performance of the delta method BMDL, and alternative methods are useful. In this article, we propose a bootstrap method to estimate BMDL utilizing a scheme that consists of a resampling of residuals after model fitting and a one-step formula for parameter estimation. We illustrate the method with clustered binary data from developmental toxicity experiments. Our analysis shows that with moderately elevated dose-response data, the distribution of BMD estimator tends to be left-skewed and bootstrap BMDL s are smaller than the delta method BMDL s on average, hence quantifying risk more conservatively. Statistically, the bootstrap BMDL quantifies the uncertainty of the true BMD more honestly than the delta method BMDL as its coverage probability is closer to the nominal level than that of delta method BMDL. We find that BMD and BMDL estimates are generally insensitive to model choices provided that the models fit the data comparably well near the region of BMD. Our analysis also suggests that, in the presence of a significant and moderately strong dose-response relationship, the developmental toxicity experiments under the standard protocol support dose-response assessment at 5% BMR for BMD and 95% confidence level for BMDL.     

Model Uncertainty and Bayesian Model Averaged Benchmark Dose Estimation for Continuous Data

The benchmark dose (BMD) approach has gained acceptance as a valuable risk assessment tool, but risk assessors still face significant challenges associated with selecting an appropriate BMD/BMDL estimate from the results of a set of acceptable dose‐response models. Current approaches do not explicitly address model uncertainty, and there is an existing need to more fully inform health risk assessors in this regard. In this study, a Bayesian model averaging (BMA) BMD estimation method taking model uncertainty into account is proposed as an alternative to current BMD estimation approaches for continuous data. Using the “hybrid” method proposed by Crump, two strategies of BMA, including both “maximum likelihood estimation based” and “Markov Chain Monte Carlo based” methods, are first applied as a demonstration to calculate model averaged BMD estimates from real continuous dose‐response data. The outcomes from the example data sets examined suggest that the BMA BMD estimates have higher reliability than the estimates from the individual models with highest posterior weight in terms of higher BMDL and smaller 90th percentile intervals. In addition, a simulation study is performed to evaluate the accuracy of the BMA BMD estimator. The results from the simulation study recommend that the BMA BMD estimates have smaller bias than the BMDs selected using other criteria. To further validate the BMA method, some technical issues, including the selection of models and the use of bootstrap methods for BMDL derivation, need further investigation over a more extensive, representative set of dose‐response data.     

Influence of Distribution of Animals between Dose Groups on Estimated Benchmark Dose and Animal Welfare for Continuous Effects

The benchmark dose (BMD) approach is increasingly used as a preferred approach for dose–effect analysis, but standard experimental designs are generally not optimized for BMD analysis. The aim of this study was to evaluate how the use of unequally sized dose groups affects the quality of BMD estimates in toxicity testing, with special consideration of the total burden of animal distress. We generated continuous dose–effect data by Monte Carlo simulation using two dose–effect curves based on endpoints with different shape parameters. Eighty‐five designs, each with four dose groups of unequal size, were examined in scenarios ranging from low‐ to high‐dose placements and with a total number of animals set to 40, 80, or 200. For each simulation, a BMD value was estimated and compared with the “true” BMD. In general, redistribution of animals from higher to lower dose groups resulted in an improved precision of the calculated BMD value as long as dose placements were high enough to detect a significant trend in the dose–effect data with sufficient power. The improved BMD precision and the associated reduction of the number of animals exposed to the highest dose, where chemically induced distress is most likely to occur, are favorable for the reduction and refinement principles. The result thereby strengthen BMD‐aligned design of experiments as a means for more accurate hazard characterization along with animal welfare improvements.     

Potential Uncertainty Reduction in Model‐Averaged Benchmark Dose Estimates Informed by an Additional Dose Study

A methodology is presented for assessing the information value of an additional dosage experiment in existing bioassay studies. The analysis demonstrates the potential reduction in the uncertainty of toxicity metrics derived from expanded studies, providing insights for future studies. Bayesian methods are used to fit alternative dose‐response models using Markov chain Monte Carlo (MCMC) simulation for parameter estimation and Bayesian model averaging (BMA) is used to compare and combine the alternative models. BMA predictions for benchmark dose (BMD) are developed, with uncertainty in these predictions used to derive the lower bound BMDL. The MCMC and BMA results provide a basis for a subsequent Monte Carlo analysis that backcasts the dosage where an additional test group would have been most beneficial in reducing the uncertainty in the BMD prediction, along with the magnitude of the expected uncertainty reduction. Uncertainty reductions are measured in terms of reduced interval widths of predicted BMD values and increases in BMDL values that occur as a result of this reduced uncertainty. The methodology is illustrated using two existing data sets for TCDD carcinogenicity, fitted with two alternative dose‐response models (logistic and quantal‐linear). The example shows that an additional dose at a relatively high value would have been most effective for reducing the uncertainty in BMA BMD estimates, with predicted reductions in the widths of uncertainty intervals of approximately 30%, and expected increases in BMDL values of 5–10%. The results demonstrate that dose selection for studies that subsequently inform dose‐response models can benefit from consideration of how these models will be fit, combined, and interpreted.     

A Parametric Model for Detecting Hormetic Effects in Developmental Toxicity Studies

Hormetic effects have been observed at low exposure levels based on the dose-response pattern of data from developmental toxicity studies. This indicates that there might actually be a reduced risk of exhibiting toxic effects at low exposure levels. Hormesis implies the existence of a threshold dose level and there are dose-response models that include parameters that account for the threshold. We propose a function that introduces a parameter to account for hormesis. This function is a subset of the set of all functions that could represent a hormetic dose-response relationship at low exposure levels to toxic agents. We characterize the overall dose-response relationship with a piecewise function that consists of a hormetic u-shape curve at low dose levels and a logistic curve at high dose levels. We apply our model to a data set from an experiment conducted at the National Toxicology Program (NTP). We also use the beta-binomial distribution to model the litter response data. It can be seen by observing the structure of these data that current experimental designs for developmental studies employ a limited number of dose groups. These designs may not be satisfactory when the goal is to illustrate the existence of hormesis. In particular, increasing the number of low-level doses improves the power for detecting hormetic effects. Therefore, we also provide the results of simulations that were done to characterize the power of current designs in detecting hormesis and to demonstrate how this power can be improved upon by altering these designs with the addition of only a few low exposure levels.     

Comparing Experimental Designs for Benchmark Dose Calculations for Continuous Endpoints

The BMD (benchmark dose) method that is used in risk assessment of chemical compounds was introduced by Crump (1984) and is based on dose-response modeling. To take uncertainty in the data and model fitting into account, the lower confidence bound of the BMD estimate (BMDL) is suggested to be used as a point of departure in health risk assessments. In this article, we study how to design optimum experiments for applying the BMD method for continuous data. We exemplify our approach by considering the class of Hill models. The main aim is to study whether an increased number of dose groups and at the same time a decreased number of animals in each dose group improves conditions for estimating the benchmark dose. Since Hill models are nonlinear, the optimum design depends on the values of the unknown parameters. That is why we consider Bayesian designs and assume that the parameter vector has a prior distribution. A natural design criterion is to minimize the expected variance of the BMD estimator. We present an example where we calculate the value of the design criterion for several designs and try to find out how the number of dose groups, the number of animals in the dose groups, and the choice of doses affects this value for different Hill curves. It follows from our calculations that to avoid the risk of unfavorable dose placements, it is good to use designs with more than four dose groups. We can also conclude that any additional information about the expected dose-response curve, e.g., information obtained from studies made in the past, should be taken into account when planning a study because it can improve the design.     

Implications of Developmental Toxicity Study Design for Quantitative Risk Assessment

Standard experimental designs for conducting developmental toxicity studies typically include three- or four-dose levels in addition to a control group. Some researchers have suggested that designs with more exposure groups would improve dose-response characterization and risk estimation. Such proposals have not, however, been supported by the results of simulation studies, which instead back the use of fewer dose levels. This discrepancy is partly due to using a known dose–response pattern to generate data, making model choice obvious. While the carcinogenicity literature has explored implications of different study designs, little attention has been given to the role of design in developmental toxicity risk assessment (or noncancer toxicology in general). In this research, we explore the implications of various experimental designs for developmental toxicity by resampling data from a large study of 2,4,5-trichlorophenoxyacetic acid in mice. We compare the properties of benchmark dose (BMD) estimation for different design strategies by randomly selecting animals within particular dose groups from the entire 2,4,5-T database of over 77,000 birth outcomes to create smaller "pseudo-studies" that are representative of standard bioassay sample sizes. Our results show that experimental designs which include more dose levels have advantages in terms of risk characterization and estimation.     

Characterizing Dose-Response I: Critical Assessment of the Benchmark Dose Concept

We present a critical assessment of the benchmark dose (BMD) method introduced by Crump⁽¹⁾ as an alternative method for setting a characteristic dose level for toxicant risk assessment. The no-observed-adverse-effect-level (NOAEL) method has been criticized because it does not use all of the data and because the characteristic dose level obtained depends on the dose levels and the statistical precision (sample sizes) of the study design. Defining the BMD in terms of a confidence bound on a point estimate results in a characteristic dose that also varies with the statistical precision and still depends on the study dose levels.⁽²⁾ Indiscriminate choice of benchmark response level may result in a BMD that reflects little about the dose-response behavior available from using all of the data. Another concern is that the definition of the BMD for the quantal response case is different for the continuous response case. Specifically, defining the BMD for continuous data using a ratio of increased effect divided by the background response results in an arbitrary dependence on the natural background for the endpoint being studied, making comparison among endpoints less meaningful and standards more arbitrary. We define a modified benchmark dose as a point estimate using the ratio of increased effect divided by the full adverse response range which enables consistent placement of the benchmark response level and provides a BMD with a more consistent relationship to the dose-response curve shape.     

Quantitative Risk Assessment for Developmental Neurotoxic Effects

Developmental neurotoxicity concerns the adverse health effects of exogenous agents acting on neurodevelopment. Because human brain development is a delicate process involving many cellular events, the developing fetus is rather susceptible to compounds that can alter the structure and function of the brain. Today, there is clear evidence that early exposure to many neurotoxicants can severely damage the developing nervous system. Although in recent years, there has been much attention given to model development and risk assessment procedures for developmental toxicants, the area of developmental neurotoxicity has been largely ignored. Here, we consider the problem of risk estimation for developmental neurotoxicants from animal bioassay data. Since most responses from developmental neurotoxicity experiments are nonquantal in nature, an adverse health effect will be defined as a response that occurs with very small probability in unexposed animals. Using a two-stage hierarchical normal dose-response model, upper confidence limits on the excess risk due to a given level of added exposure are derived. Equivalently, the model is used to obtain lower confidence limits on dose for a small negligible level of risk. Our method is based on the asymptotic distribution of the likelihood ratio statistic (cf. Crump, 1995). An example is used to provide further illustration.     

Regression Models and Risk Estimation for Mixed Discrete and Continuous Outcomes in Developmental Toxicology

Multivariate dose-response models have recently been proposed for developmental toxicity data to simultaneously model malformation incidence (a binary outcome), and reductions in fetal weight (a continuous outcome). In this and other applications, the binary outcome often represents a dichotomization of another outcome or a composite of outcomes, which facilitates analysis. For example, in Segment II developmental toxicology studies, multiple malformation types (i.e., external, visceral, skeletal) are evaluated on each fetus; malformation status may also be ordinally measured (e.g., normal, signs of variation, full malformation). A model is proposed is for fetal weight and multiple malformation variables measured on an ordinal scale, where the correlations between the outcomes and between the offspring within a litter are taken into account. Fully specifying the joint distribution of outcomes within a litter is avoided by specifying only the distribution of the multivariate outcome for each fetus and using generalized estimating equation methodology to account for correlations due to litter clustering. The correlations between the outcomes are required to characterize joint risk to the fetus, and are therefore a focus of inference. Dose-response models and their application to quantitative risk assessment are illustrated using data from a recent developmental toxicology experiment of ethylene oxide in mice.     

Re-evaluation of the Reference Dose for Methylmercury and Assessment of Current Exposure Levels

Methylmercury (Me-Hg) is widely distributed through freshwater and saltwater food chains and human consumption of fish and shellfish has lead to widespread exposure. Both the U.S. EPA Reference Dose (0.3 μg/kg/day) and the FAO/WHO Permissible Tolerable Weekly Intake (3.3 μg/kg/week) are currently based on the prevention of paraesthesia in adult and older children. However, Me-Hg exposure in utero is known to result in a range of developmental neurologic effects including clinical CNS symptoms and delayed onset of walking. Based on a critical review of developmental toxicity data from human and animal studies, it is concluded that current guidelines for the prevention of paraesthesia are not adequate to address developmental effects. A dose of 0.07 μ/kg/day is suggested as the best estimate of a potential reference dose for developmental effects. Data on nationwide fish consumption rates and Me-Hg levels in fish/seafood weighted by proportion of the catch intended for human consumption are analyzed in a Monte Carlo simulation to derive a probability distribution of background Me-Hg exposure. While various uncertainties in the toxicologic and exposure data limit the precision with which health risk can be estimated, this analysis suggests that at current levels of Me-Hg exposure, a significant fraction of women of childbearing age have exposures above this suggested reference dose.     

Model Selection and Estimation with Quantal‐Response Data in Benchmark Risk Assessment

This article describes several approaches for estimating the benchmark dose (BMD) in a risk assessment study with quantal dose‐response data and when there are competing model classes for the dose‐response function. Strategies involving a two‐step approach, a model‐averaging approach, a focused‐inference approach, and a nonparametric approach based on a PAVA‐based estimator of the dose‐response function are described and compared. Attention is raised to the perils involved in data “double‐dipping” and the need to adjust for the model‐selection stage in the estimation procedure. Simulation results are presented comparing the performance of five model selectors and eight BMD estimators. An illustration using a real quantal‐response data set from a carcinogenecity study is provided.     

Risk Perception, Federal Spending, and the Savannah River Site: Attitudes of Hunters and Fishermen

This paper examines the attitudes of 285 hunters and fishermen from South Carolina about hunting and fishing, risk, environmental issues, and future land use of the Savannah River Site. We test the null hypothesis that there is no difference in hunting and fishing rates, attitudes toward the safety of fish and deer obtained from SRS, attitudes toward future land use at SRS, and perceptions of the severity of environmental problems as a function of how far respondents lived from the site. Respondents hunted or fished an average of over 40 days a year, and only half felt that the fish and deer from SRS were safe to eat. Willingness to expend federal funds was correlated with perceptions of the severity of the problem. Preferences for future land use at SRS fell into three categories: high (environmental research park, hunting, fishing, camping), medium (nuclear production, factories, preserve only), and low (nuclear waste storage, residential). There were no differences in hunting and fishing rates, ranking of the severity of environmental problems, and willingness to expend federal funds as a function of distance of residence from SRS, but attitudes toward future land use differed significantly as a function of location of residence. Those living close to SRS were more willing to have the site used for factories, residential, nuclear material production and to store nuclear wastes than those living farther from the site. Our data on recreational rates, attitudes toward future land use, and willingness to expend federal funds to solve environmental problems reiterate the importance of assessing stakeholder attitudes toward decisions regarding future land use at DOE sites     

Influence of Distribution of Animals between Dose Groups on Estimated Benchmark Dose and Animal Distress for Quantal Responses

Increasingly, dose‐response data are being evaluated with the benchmark dose (BMD) approach rather than by the less precise no‐observed‐adverse‐effect‐level (NOAEL) approach. However, the basis for designing animal experiments, using equally sized dose groups, is still primed for the NOAEL approach. The major objective here was to assess the impact of using dose groups of unequal size on both the quality of the BMD and overall animal distress. We examined study designs with a total number of 200 animals distributed in four dose groups employing quantal data generated by Monte Carlo simulations. Placing more animals at doses close to the targeted BMD provided an estimate of BMD that was slightly better than the standard design with equally sized dose groups. In situations involving a clear dose‐response, this translates into fewer animals receiving high doses and thus less overall animal distress. Accordingly, in connection with risk and safety assessment, animal distress can potentially be reduced by distributing the animals appropriately between dose groups without decreasing the quality of the information obtained.     

Dose‐Response Modeling with Summary Data from Developmental Toxicity Studies

Dose‐response analysis of binary developmental data (e.g., implant loss, fetal abnormalities) is best done using individual fetus data (identified to litter) or litter‐specific statistics such as number of offspring per litter and proportion abnormal. However, such data are not often available to risk assessors. Scientific articles usually present only dose‐group summaries for the number or average proportion abnormal and the total number of fetuses. Without litter‐specific data, it is not possible to estimate variances correctly (often characterized as a problem of overdispersion, intralitter correlation, or “litter effect”). However, it is possible to use group summary data when the design effect has been estimated for each dose group. Previous studies have demonstrated useful dose‐response and trend test analyses based on design effect estimates using litter‐specific data from the same study. This simplifies the analysis but does not help when litter‐specific data are unavailable. In the present study, we show that summary data on fetal malformations can be adjusted satisfactorily using estimates of the design effect based on historical data. When adjusted data are then analyzed with models designed for binomial responses, the resulting benchmark doses are similar to those obtained from analyzing litter‐level data with nested dichotomous models.     

Model Averaging Using Fractional Polynomials to Estimate a Safe Level of Exposure

Quantitative risk assessment involves the determination of a safe level of exposure. Recent techniques use the estimated dose-response curve to estimate such a safe dose level. Although such methods have attractive features, a low-dose extrapolation is highly dependent on the model choice. Fractional polynomials, basically being a set of (generalized) linear models, are a nice extension of classical polynomials, providing the necessary flexibility to estimate the dose-response curve. Typically, one selects the best-fitting model in this set of polynomials and proceeds as if no model selection were carried out. We show that model averaging using a set of fractional polynomials reduces bias and has better precision in estimating a safe level of exposure (say, the benchmark dose), as compared to an estimator from the selected best model. To estimate a lower limit of this benchmark dose, an approximation of the variance of the model-averaged estimator, as proposed by Burnham and Anderson, can be used. However, this is a conservative method, often resulting in unrealistically low safe doses. Therefore, a bootstrap-based method to more accurately estimate the variance of the model averaged parameter is proposed.