On participant-observation as a component of evaluation. Strategies, constraints and issues

Described here is a participant-observation strategy employed as one portion of the evaluation research at Family House, a residential treatment program for alcoholic mothers and their children. Major objectives of this leg of a two-legged research design combining this approach with a more standard pre- and post-test strategy were to document and analyze the processes of social development in the new "small society." The ethnographer acted as consultant, with "raw data" generated in the form of daily observations dictated on the rotating basis by staff members over a two-year period. The resulting narrative was coded and analyzed by the anthropologist, utilizing a phenomenological or interpretationist perspective which allowed the "native categories" of significance to emerge and guide the analysis. This novel research strategy is compared with standard ethnographic fieldwork, as well as with pre- and post-test design, in terms of its strengths and weaknesses. Strengths include access to an authentic "inside view" of Family House reality, as well as an unusually rich longitudinal record of daily life in the facility. Problems include danger of contaminating the data by means of the "feedback" provided to reward staff for their cooperation in the research effort. The author concludes, however, that since reactive effects are present in all designs, and since "feedback" is a normal feature of therapeutic milieux, validity is not unusually threatened. Finally, the purposes of evaluation research are scrutinized.