Characterizing Dose-Response I: Critical Assessment of the Benchmark Dose Concept

We present a critical assessment of the benchmark dose (BMD) method introduced by Crump⁽¹⁾ as an alternative method for setting a characteristic dose level for toxicant risk assessment. The no-observed-adverse-effect-level (NOAEL) method has been criticized because it does not use all of the data and because the characteristic dose level obtained depends on the dose levels and the statistical precision (sample sizes) of the study design. Defining the BMD in terms of a confidence bound on a point estimate results in a characteristic dose that also varies with the statistical precision and still depends on the study dose levels.⁽²⁾ Indiscriminate choice of benchmark response level may result in a BMD that reflects little about the dose-response behavior available from using all of the data. Another concern is that the definition of the BMD for the quantal response case is different for the continuous response case. Specifically, defining the BMD for continuous data using a ratio of increased effect divided by the background response results in an arbitrary dependence on the natural background for the endpoint being studied, making comparison among endpoints less meaningful and standards more arbitrary. We define a modified benchmark dose as a point estimate using the ratio of increased effect divided by the full adverse response range which enables consistent placement of the benchmark response level and provides a BMD with a more consistent relationship to the dose-response curve shape.     

A Simulation Study of Quantitative Risk Assessment for Bivariate Continuous Outcomes

The neurotoxic effects of chemical agents are often investigated in controlled studies on rodents, with binary and continuous multiple endpoints routinely collected. One goal is to conduct quantitative risk assessment to determine safe dose levels. Yu and Catalano (2005) describe a method for quantitative risk assessment for bivariate continuous outcomes by extending a univariate method of percentile regression. The model is likelihood based and allows for separate dose‐response models for each outcome while accounting for the bivariate correlation. The approach to benchmark dose (BMD) estimation is analogous to that for quantal data without having to specify arbitrary cutoff values. In this article, we evaluate the behavior of the BMD relative to background rates, sample size, level of bivariate correlation, dose‐response trend, and distributional assumptions. Using simulations, we explore the effects of these factors on the resulting BMD and BMDL distributions. In addition, we illustrate our method with data from a neurotoxicity study of parathion exposure in rats.     

Influence of Distribution of Animals between Dose Groups on Estimated Benchmark Dose and Animal Distress for Quantal Responses

Increasingly, dose‐response data are being evaluated with the benchmark dose (BMD) approach rather than by the less precise no‐observed‐adverse‐effect‐level (NOAEL) approach. However, the basis for designing animal experiments, using equally sized dose groups, is still primed for the NOAEL approach. The major objective here was to assess the impact of using dose groups of unequal size on both the quality of the BMD and overall animal distress. We examined study designs with a total number of 200 animals distributed in four dose groups employing quantal data generated by Monte Carlo simulations. Placing more animals at doses close to the targeted BMD provided an estimate of BMD that was slightly better than the standard design with equally sized dose groups. In situations involving a clear dose‐response, this translates into fewer animals receiving high doses and thus less overall animal distress. Accordingly, in connection with risk and safety assessment, animal distress can potentially be reduced by distributing the animals appropriately between dose groups without decreasing the quality of the information obtained.     

Evaluation of the benchmark dose for point of departure determination for a variety of chemical classes in applied regulatory settings

Repeated-dose studies received by the New Substances Assessment and Control Bureau (NSACB) of Health Canada are used to provide hazard information toward risk calculation. These studies provide a point of departure (POD), traditionally the NOAEL or LOAEL, which is used to extrapolate the quantity of substance above which adverse effects can be expected in humans. This project explored the use of benchmark dose (BMD) modeling as an alternative to this approach for studies with few dose groups. Continuous data from oral repeated-dose studies for chemicals previously assessed by NSACB were reanalyzed using U.S. EPA benchmark dose software (BMDS) to determine the BMD and BMD 95% lower confidence limit (BMDL(05) ) for each endpoint critical to NOAEL or LOAEL determination for each chemical. Endpoint-specific benchmark dose-response levels , indicative of adversity, were consistently applied. An overall BMD and BMDL(05) were calculated for each chemical using the geometric mean. The POD obtained from benchmark analysis was then compared with the traditional toxicity thresholds originally used for risk assessment. The BMD and BMDL(05) generally were higher than the NOAEL, but lower than the LOAEL. BMDL(05) was generally constant at 57% of the BMD. Benchmark provided a clear advantage in health risk assessment when a LOAEL was the only POD identified, or when dose groups were widely distributed. Although the benchmark method cannot always be applied, in the selected studies with few dose groups it provided a more accurate estimate of the real no-adverse-effect level of a substance.     

Properties of Model-Averaged BMDLs: A Study of Model Averaging in Dichotomous Response Risk Estimation

Model averaging (MA) has been proposed as a method of accounting for model uncertainty in benchmark dose (BMD) estimation. The technique has been used to average BMD dose estimates derived from dichotomous dose-response experiments, microbial dose-response experiments, as well as observational epidemiological studies. While MA is a promising tool for the risk assessor, a previous study suggested that the simple strategy of averaging individual models' BMD lower limits did not yield interval estimators that met nominal coverage levels in certain situations, and this performance was very sensitive to the underlying model space chosen. We present a different, more computationally intensive, approach in which the BMD is estimated using the average dose-response model and the corresponding benchmark dose lower bound (BMDL) is computed by bootstrapping. This method is illustrated with TiO(2) dose-response rat lung cancer data, and then systematically studied through an extensive Monte Carlo simulation. The results of this study suggest that the MA-BMD, estimated using this technique, performs better, in terms of bias and coverage, than the previous MA methodology. Further, the MA-BMDL achieves nominal coverage in most cases, and is superior to picking the "best fitting model" when estimating the benchmark dose. Although these results show utility of MA for benchmark dose risk estimation, they continue to highlight the importance of choosing an adequate model space as well as proper model fit diagnostics.     

Model Selection and Estimation with Quantal‐Response Data in Benchmark Risk Assessment

This article describes several approaches for estimating the benchmark dose (BMD) in a risk assessment study with quantal dose‐response data and when there are competing model classes for the dose‐response function. Strategies involving a two‐step approach, a model‐averaging approach, a focused‐inference approach, and a nonparametric approach based on a PAVA‐based estimator of the dose‐response function are described and compared. Attention is raised to the perils involved in data “double‐dipping” and the need to adjust for the model‐selection stage in the estimation procedure. Simulation results are presented comparing the performance of five model selectors and eight BMD estimators. An illustration using a real quantal‐response data set from a carcinogenecity study is provided.     

Estimation of a Benchmark Dose in the Presence or Absence of Hormesis Using Posterior Averaging

U.S. Environment Protection Agency benchmark doses for dichotomous cancer responses are often estimated using a multistage model based on a monotonic dose‐response assumption. To account for model uncertainty in the estimation process, several model averaging methods have been proposed for risk assessment. In this article, we extend the usual parameter space in the multistage model for monotonicity to allow for the possibility of a hormetic dose‐response relationship. Bayesian model averaging is used to estimate the benchmark dose and to provide posterior probabilities for monotonicity versus hormesis. Simulation studies show that the newly proposed method provides robust point and interval estimation of a benchmark dose in the presence or absence of hormesis. We also apply the method to two data sets on carcinogenic response of rats to 2,3,7,8‐tetrachlorodibenzo‐p‐dioxin.     

Quantitative Risk Assessment: Developing a Bayesian Approach to Dichotomous Dose–Response Uncertainty

Model averaging for dichotomous dose–response estimation is preferred to estimate the benchmark dose (BMD) from a single model, but challenges remain regarding implementing these methods for general analyses before model averaging is feasible to use in many risk assessment applications, and there is little work on Bayesian methods that include informative prior information for both the models and the parameters of the constituent models. This article introduces a novel approach that addresses many of the challenges seen while providing a fully Bayesian framework. Furthermore, in contrast to methods that use Monte Carlo Markov Chain, we approximate the posterior density using maximum a posteriori estimation. The approximation allows for an accurate and reproducible estimate while maintaining the speed of maximum likelihood, which is crucial in many applications such as processing massive high throughput data sets. We assess this method by applying it to empirical laboratory dose–response data and measuring the coverage of confidence limits for the BMD. We compare the coverage of this method to that of other approaches using the same set of models. Through the simulation study, the method is shown to be markedly superior to the traditional approach of selecting a single preferred model (e.g., from the U.S. EPA BMD software) for the analysis of dichotomous data and is comparable or superior to the other approaches.     

On Determining the BMD from Multiple Outcomes in Developmental Toxicity Studies when One Outcome is Intentionally Missing

Public health concerns over the occurrence of developmental abnormalities that can occur as a result of prenatal exposure to drugs, chemicals, and other environmental factors has led to a number of developmental toxicity studies and the use of the benchmark dose (BMD) for risk assessment. To characterize risk from multiple sources, more recent analytic methods involve a joint modeling approach, accounting for multiple dichotomous and continuous outcomes. For some continuous outcomes, evaluating all subjects may not be feasible, and only a subset may be evaluated due to limited resources. The subset can be selected according to a prespecified probability model and the unobserved data can be viewed as intentionally missing in the sense that subset selection results in missingness that is experimentally planned. We describe a subset selection model that allows for sampling pups with malformations and healthy pups at different rates, and includes the well‐known simple random sample (SRS) as a special case. We were interested in understanding how sampling rates that are selected beforehand influence the precision of the BMD. Using simulations we show how improvements over the SRS can be obtained by oversampling malformations, and how some sampling rates can yield precision that is substantially worse than the SRS. We also illustrate the potential for cost saving with oversampling. Simulations are based on a joint mixed effects model, and to account for subset selection, use of case weights to obtain valid dose‐response estimates.     

Model Uncertainty and Bayesian Model Averaged Benchmark Dose Estimation for Continuous Data

The benchmark dose (BMD) approach has gained acceptance as a valuable risk assessment tool, but risk assessors still face significant challenges associated with selecting an appropriate BMD/BMDL estimate from the results of a set of acceptable dose‐response models. Current approaches do not explicitly address model uncertainty, and there is an existing need to more fully inform health risk assessors in this regard. In this study, a Bayesian model averaging (BMA) BMD estimation method taking model uncertainty into account is proposed as an alternative to current BMD estimation approaches for continuous data. Using the “hybrid” method proposed by Crump, two strategies of BMA, including both “maximum likelihood estimation based” and “Markov Chain Monte Carlo based” methods, are first applied as a demonstration to calculate model averaged BMD estimates from real continuous dose‐response data. The outcomes from the example data sets examined suggest that the BMA BMD estimates have higher reliability than the estimates from the individual models with highest posterior weight in terms of higher BMDL and smaller 90th percentile intervals. In addition, a simulation study is performed to evaluate the accuracy of the BMA BMD estimator. The results from the simulation study recommend that the BMA BMD estimates have smaller bias than the BMDs selected using other criteria. To further validate the BMA method, some technical issues, including the selection of models and the use of bootstrap methods for BMDL derivation, need further investigation over a more extensive, representative set of dose‐response data.     

Correlation of Noncancer Benchmark Doses in Short‐ and Long‐Term Rodent Bioassays

This study investigated whether, in the absence of chronic noncancer toxicity data, short‐term noncancer toxicity data can be used to predict chronic toxicity effect levels by focusing on the dose–response relationship instead of a critical effect. Data from National Toxicology Program (NTP) technical reports have been extracted and modeled using the Environmental Protection Agency's Benchmark Dose Software. Best‐fit, minimum benchmark dose (BMD), and benchmark dose lower limits (BMDLs) have been modeled for all NTP pathologist identified significant nonneoplastic lesions, final mean body weight, and mean organ weight of 41 chemicals tested by NTP between 2000 and 2012. Models were then developed at the chemical level using orthogonal regression techniques to predict chronic (two years) noncancer health effect levels using the results of the short‐term (three months) toxicity data. The findings indicate that short‐term animal studies may reasonably provide a quantitative estimate of a chronic BMD or BMDL. This can allow for faster development of human health toxicity values for risk assessment for chemicals that lack chronic toxicity data.     

Relation Between Benchmark Dose and No-Observed-Adverse-Effect Level in Clinical Research: Effects of Daily Alcohol Intake on Blood Pressure in Japanese Salesmen

The benchmark dose (BMD) is defined as the dose that corresponds to a specific change in an adverse response compared to the response in unexposed subjects, and the lower 95% confidence limit is termed the benchmark dose level (BMDL). In this study, the threshold of daily ethanol intake affecting blood pressure was calculated by both the BMD approach and multiple logistic regression analysis to clarify the relation between the BMDL and no-observed-adverse-effect level (NOAEL). Systolic and diastolic blood pressures (SBP and DBP) and daily ethanol intake were explored in 1,100 Japanese salesmen. The SBP and DBP were positively related to daily ethanol intake (p < 0.001) when adjusting for possible confounders such as age, body mass index, and smoking status. The adjusted risk for hypertension (SBP >or= 140 mmHg or DBP >or= 90 mmHg) increased significantly when daily ethanol intake exceeded 60 g/day, and the categorical dose of interest was 60.1-90 g/day. The BMDL and BMD of ethanol intake for increased SBP and DBP were estimated to be approximately 60 and 75 g/day, respectively. These findings suggest that the BMDL and BMD correspond to the NOAEL and lowest-observed-adverse-effect level, respectively, if the sample number of clinical data is large enough to confirm the dose-response association.     

Influence of Distribution of Animals between Dose Groups on Estimated Benchmark Dose and Animal Welfare for Continuous Effects

The benchmark dose (BMD) approach is increasingly used as a preferred approach for dose–effect analysis, but standard experimental designs are generally not optimized for BMD analysis. The aim of this study was to evaluate how the use of unequally sized dose groups affects the quality of BMD estimates in toxicity testing, with special consideration of the total burden of animal distress. We generated continuous dose–effect data by Monte Carlo simulation using two dose–effect curves based on endpoints with different shape parameters. Eighty‐five designs, each with four dose groups of unequal size, were examined in scenarios ranging from low‐ to high‐dose placements and with a total number of animals set to 40, 80, or 200. For each simulation, a BMD value was estimated and compared with the “true” BMD. In general, redistribution of animals from higher to lower dose groups resulted in an improved precision of the calculated BMD value as long as dose placements were high enough to detect a significant trend in the dose–effect data with sufficient power. The improved BMD precision and the associated reduction of the number of animals exposed to the highest dose, where chemically induced distress is most likely to occur, are favorable for the reduction and refinement principles. The result thereby strengthen BMD‐aligned design of experiments as a means for more accurate hazard characterization along with animal welfare improvements.     

A Health Risk Benchmark for the Neurologic Effects of Styrene: Comparison with NOAEL/LOAEL Approach

Benchmark dose (BMD) analysis was used to estimate an inhalation benchmark concentration for styrene neurotoxicity. Quantal data on neuropsychologic test results from styrene-exposed workers [Mutti et al. (1984). American Journal of Industrial Medicine, 5, 275-286] were used to quantify neurotoxicity, defined as the percent of tested workers who responded abnormally to > or = 1, > or = 2, or > or = 3 out of a battery of eight tests. Exposure was based on previously published results on mean urinary mandelic- and phenylglyoxylic acid levels in the workers, converted to air styrene levels (15, 44, 74, or 115 ppm). Nonstyrene-exposed workers from the same region served as a control group. Maximum-likelihood estimates (MLEs) and BMDs at 5 and 10% response levels of the exposed population were obtained from log-normal analysis of the quantal data. The highest MLE was 9 ppm (BMD = 4 ppm) styrene and represents abnormal responses to > or = 3 tests by 10% of the exposed population. The most health-protective MLE was 2 ppm styrene (BMD = 0.3 ppm) and represents abnormal responses to > or = 1 test by 5% of the exposed population. A no observed adverse effect level/lowest observed adverse effect level (NOAEL/LOAEL) analysis of the same quantal data showed workers in all styrene exposure groups responded abnormally to > or = 1, > or = 2, or > or = 3 tests, compared to controls, and the LOAEL was 15 ppm. A comparison of the BMD and NOAEL/LOAEL analyses suggests that at air styrene levels below the LOAEL, a segment of the worker population may be adversely affected. The benchmark approach will be useful for styrene noncancer risk assessment purposes by providing a more accurate estimate of potential risk that should, in turn, help to reduce the uncertainty that is a common problem in setting exposure levels.     

Benchmark Dose Analysis via Nonparametric Regression Modeling

Estimation of benchmark doses (BMDs) in quantitative risk assessment traditionally is based upon parametric dose‐response modeling. It is a well‐known concern, however, that if the chosen parametric model is uncertain and/or misspecified, inaccurate and possibly unsafe low‐dose inferences can result. We describe a nonparametric approach for estimating BMDs with quantal‐response data based on an isotonic regression method, and also study use of corresponding, nonparametric, bootstrap‐based confidence limits for the BMD. We explore the confidence limits’ small‐sample properties via a simulation study, and illustrate the calculations with an example from cancer risk assessment. It is seen that this nonparametric approach can provide a useful alternative for BMD estimation when faced with the problem of parametric model uncertainty.     

Calculation of Benchmark Doses from Continuous Data

A benchmark dose (BMD) is the dose of a substance that corresponds to a prescribed increase in the response (called the benchmark response or BMR) of a health effect. A statistical lower bound on the benchmark dose (BMDL) has been proposed as a replacement for the no-observed-adverse-effect-level (NOAEL) in setting acceptable human exposure levels. A method is developed in this paper for calculating BMDs and BMDLs from continuous data in a manner that is consistent with those calculated from quantal data. The method involves defining an abnormal response, either directly by specifying a cutoff x₀ that separates continuous responses into normal and abnormal categories, or indirectly by specifying the proportion P₀ of abnormal responses expected among unexposed subjects. The method does not involve actually dichotomizing individual continuous responses into quantal responses, and in certain cases can be applied to continuous data in summarized form (e.g., means and standard deviations of continuous responses among subjects in discrete dose groups). In addition to specifying the BMR and either x₀ or P₀ , the method requires specification of the distribution of continuous responses, including specification of the dose-response θ(d) for a measure of central tendency. A method is illustrated for selecting θ(d) to make the probability of an abnormal response any desired dose-response function. This enables the same dose-response model (Weibull, log-logistic, etc.) to be used for the probability of an abnormal response, regardless of whether the underlying data are continuous or quantal. Whenever the continuous responses are normally distributed with standard deviation σ (independent of dose), the method is equivalent to defining the BMD as the dose corresponding to a prescribed change in the mean response relative to σ.     

Benchmark Calculations in Risk Assessment Using Continuous Dose-Response Information: The Influence of Variance and the Determination of a Cut-Off Value

A benchmark dose (BMD) is the dose of a chemical that corresponds to a predetermined increase in the response (the benchmark response, BMR) of a health effect. In this article, a method (the hybrid approach) for benchmark calculations from continuous dose-response information is investigated. In the formulation of the methodology, a cut-off value for an adverse health effect has to be determined. It is shown that the influence of variance on the hybrid model depends on the choice of determination of the cut-off point. If the cut-off value is determined as corresponding to a specified tail proportion of the control distribution, P(0), the BMD becomes biased upward when the variance is biased upward. On the contrary, if the cut-off value is directly determined to some level of the continuous response variable, the BMD becomes biased upward when the variance is biased downward. A simulation study was also performed in which the accuracy and precision of the BMD was compared for the two ways of determining the cut-off value. In general, considering BMRs of 1, 5, and 10% (additional risk) the precision of the BMD became higher when the cut-off value was estimated by specifying P(0), relative to the case with a direct determination. Use of the square-root of the maximum-likelihood estimator of the variance in BMD estimation may provide a bias that is reflected by the cut-off formulation (downward bias if specifying P(0), and upward bias if specifying the cut-off, c, directly). This feature may be reduced if an unbiased estimator of the standard deviation is used in the calculations.     

Benchmark Dose Risk Assessment for Formaldehyde Using Airflow Modeling and a Single-Compartment, DNA-Protein Cross-Link Dosimetry Model to Estimate Human Equivalent Doses

Formaldehyde induced squamous-cell carcinomas in the nasal passages of F344 rats in two inhalation bioassays at exposure levels of 6 ppm and above. Increases in rates of cell proliferation were measured by T. M. Monticello and colleagues at exposure levels of 0.7 ppm and above in the same tissues from which tumors arose. A risk assessment for formaldehyde was conducted at the CIIT Centers for Health Research, in collaboration with investigators from Toxicological Excellence in Risk Assessment (TERA) and the U.S. Environmental Protection Agency (U.S. EPA) in 1999. Two methods for dose-response assessment were used: a full biologically based modeling approach and a statistically oriented analysis by benchmark dose (BMD) method. This article presents the later approach, the purpose of which is to combine BMD and pharmacokinetic modeling to estimate human cancer risks from formaldehyde exposure. BMD analysis was used to identify points of departure (exposure levels) for low-dose extrapolation in rats for both tumor and the cell proliferation endpoints. The benchmark concentrations for induced cell proliferation were lower than for tumors. These concentrations were extrapolated to humans using two mechanistic models. One model used computational fluid dynamics (CFD) alone to determine rates of delivery of inhaled formaldehyde to the nasal lining. The second model combined the CFD method with a pharmacokinetic model to predict tissue dose with formaldehyde-induced DNA-protein cross-links (DPX) as a dose metric. Both extrapolation methods gave similar results, and the predicted cancer risk in humans at low exposure levels was found to be similar to that from a risk assessment conducted by the U.S. EPA in 1991. Use of the mechanistically based extrapolation models lends greater certainty to these risk estimates than previous approaches and also identifies the uncertainty in the measured dose-response relationship for cell proliferation at low exposure levels, the dose-response relationship for DPX in monkeys, and the choice between linear and nonlinear methods of extrapolation as key remaining sources of uncertainty.     

Comparing Experimental Designs for Benchmark Dose Calculations for Continuous Endpoints

The BMD (benchmark dose) method that is used in risk assessment of chemical compounds was introduced by Crump (1984) and is based on dose-response modeling. To take uncertainty in the data and model fitting into account, the lower confidence bound of the BMD estimate (BMDL) is suggested to be used as a point of departure in health risk assessments. In this article, we study how to design optimum experiments for applying the BMD method for continuous data. We exemplify our approach by considering the class of Hill models. The main aim is to study whether an increased number of dose groups and at the same time a decreased number of animals in each dose group improves conditions for estimating the benchmark dose. Since Hill models are nonlinear, the optimum design depends on the values of the unknown parameters. That is why we consider Bayesian designs and assume that the parameter vector has a prior distribution. A natural design criterion is to minimize the expected variance of the BMD estimator. We present an example where we calculate the value of the design criterion for several designs and try to find out how the number of dose groups, the number of animals in the dose groups, and the choice of doses affects this value for different Hill curves. It follows from our calculations that to avoid the risk of unfavorable dose placements, it is good to use designs with more than four dose groups. We can also conclude that any additional information about the expected dose-response curve, e.g., information obtained from studies made in the past, should be taken into account when planning a study because it can improve the design.     

A Simulation Study of the Influence of Study Design on the Estimation of Benchmark Doses for Developmental Toxicity1

The benchmark dose (BMD)⁴ approach is emerging as replacement to determination of the No Observed Adverse Effect Level (NOAEL) in noncancer risk assessment. This possibility raises the issue as to whether current study designs for endpoints such as developmental toxicity, optimized for detecting pair wise comparisons, could be improved for the purpose of calculating BMDs. In this paper, we examine various aspects of study design (number of dose groups, dose spacing, dose placement, and sample size per dose group) on BMDs for two endpoints of developmental toxicity (the incidence of abnormalities and of reduced fetal weight). Design performance was judged by the mean-squared error (reflective of the variance and bias) of the maximum likelihood estimate (MLE) from the log-logistic model of the 5% added risk level (the likely target risk for a benchmark calculation), as well as by the length of its 95% confidence interval (the lower value of which is the BMD). We found that of the designs evaluated, the best results were obtained when two dose levels had response rates above the background level, one of which was near the ED₀₅, were present. This situation is more likely to occur with more, rather than fewer dose levels per experiment. In this instance, there was virtually no advantage in increasing the sample size from 10 to 20 litters per dose group. If neither of the two dose groups with response rates above the background level was near the ED₀₅, satisfactory results were also obtained, but the BMDs tended to be more conservative (i.e., lower). If only one dose level with a response rate above the background level was present, and it was near the ED₀₅, reasonable results for the MLE and BMD were obtained, but here we observed benefits of larger dose group sizes. The poorest results were obtained when only a single group with an elevated response rate was present, and the response rate was much greater than the ED₀₅. The results indicate that while the benchmark dose approach is readily applicable to the standard study designs and generally observed dose-responses in developmental assays, some minor design modifications would increase the accuracy and precision of the BMD.