Addressing self-selection effects in evaluations of mutual help groups and professional mental health services: An introduction to two-stage sample selection models

Because random assignment to conditions is often neither possible nor desirable in longitudinal evaluations of mutual help organizations, the influence of self-selection effects must be assessed in order to accurately interpret outcome data. One approach to adjusting for self-selection effects is to control for covariates that predict outcome using statistical procedures such as analysis of covariance (ANCOVA), partial correlations, and hierarchical regression. This approach has considerable power, but is less useful when an evaluator is interested in directly modeling the process of entry into a program and incorporating information on the factors affecting self-selection into estimation of program effects. Two-stage sample selection models are designed to address such situations. These models rely on regression procedures in which program participation is modeled in an initial equation, which yields a sample selection correction factor. The correction factor is included with participation in a second equation that predicts outcome. This two-stage procedure allows the evaluator to interpret the observed effects of a professional service or a self-help group in the context of the magnitude and direction of selection effects. We compare and contrast the covariate control and sample selection models in a longitudinal study of the effects of participation in Alcoholics Anonymous on drinking behavior.