Quasi-experimental evaluation using confirmatory procedures: A case study of an economic and social empowerment intervention to reduce girls’ vulnerability to HIV in rural Mozambique

PurposeOur goal was to determine whether an economic and social empowerment intervention implemented in Zambézia Province, Mozambique reduced girls’ vulnerability to HIV. We use this experience to discuss challenges of evaluating real-world interventions.MethodsTwo rounds of data were collected from 885 girls, 13–19 years, for this clustered, non-equivalent (two-stage) cohort trial. We used multi-level exact matching and difference-in-differences estimation to estimate intervention effects on two outcomes: girls’ knowledge of gender-based violence and school attendance.ResultsEstimates of two outcomes analysed indicated no statistically significant intervention effects. Preliminary analysis of data from the intervention group revealed this study was unable to obtain accurate measures for five outcomes related to HIV vulnerability.ConclusionsAlthough our study did not find evidence of impact on the a priori selected outcomes, we report on our experience implementing this robust methodologic design and describe how the challenges encountered in this program setting affected our ability to attain results. We recommend prospective evaluation designs with random allocation be accommodated early during planning. When not possible, quasi-experimental studies should collect data from large samples. To reduce measurement bias, biological endpoints such sexually transmitted infections should serve as primary outcomes for programs intending to reduce sexual behaviors.