Estimating and comparing adverse event probabilities in the presence of varying follow-up times and competing events

Safety analyses of adverse events (AEs) are important in assessing benefit–risk of therapies but are often rather simplistic compared to efficacy analyses. AE probabilities are typically estimated by incidence proportions, sometimes incidence densities or Kaplan–Meier estimation are proposed. These analyses either do not account for censoring, rely on a too restrictive parametric model, or ignore competing events. With the non-parametric Aalen-Johansen estimator as the “gold standard”, that is, reference estimator, potential sources of bias are investigated in an example from oncology and in simulations, for both one-sample and two-sample scenarios. The Aalen-Johansen estimator serves as a reference, because it is the proper non-parametric generalization of the Kaplan–Meier estimator to multiple outcomes. Because of potential large variances at the end of follow-up, comparisons also consider further quantiles of the observed times. To date, consequences for safety comparisons have hardly been investigated, the impact of using different estimators for group comparisons being unclear. For example, the ratio of two both underestimating or overestimating estimators may not be comparable to the ratio of the reference, and our investigation also considers the ratio of AE probabilities. We find that ignoring competing events is more of a problem than falsely assuming constant hazards by the use of the incidence density and that the choice of the AE probability estimator is crucial for group comparisons.