What we know about child maltreatment

This article presents the results of an informal survey of nationally recognized experts on child maltreatment. The poll asked, “What do we know about child maltreatment?” A process of collating and sequential validation produced an outline of current knowledge that demonstrates some progress in understanding child maltreatment.     

A protective service system

This article presents the results of a small scale study of the protective service caseload of a mid-western metropolitan area. The basic characteristics of the caseload match the national picture presented by the American Humane Association's National Study on Child Abuse and Neglect. The present study focused on the degree to which “systematic” vs. “random” forces appear to be at work in the progress of cases through the system. The overall picture is of some systematic efforts to conduct the process of referrals to community agencies but of a great deal of “randomness” in the bureaucratic closing of cases. This is a major area of concern.     

Cost/effectiveness analysis of alcoholism and drug abuse treatment programs: the relevance of recidivism and resource absorption.

Cost/effectiveness measures are increasingly being applied to alcoholism and drug abuse treatment programs. Program evaluators usually regard readmissions as inadequate for assessing outcome, and resource absorption is rarely examined; this paper presents the argument that increased attention to recidivism and resource absorption is integral to the program evaluation process. Readmissions formed 53% of admissions to U.S. drug abuse programs, and in alcoholism treatment centers, 4% of the patients accounted for 24% of the detoxications. Problems with assessing cost/effectiveness are associated with duration of treatment, average cost, time span of analyses, quasi-experimental designs, multiple episodes of treatment, "silting up", and extrapolation. Five approaches to dealing with these problems are suggested: analyzing existing program data bases for recidivism and resource absorption, considering recidivism when evaluating programs, tempering clinical enthusiasm with skepticism, matching patients with levels of care appropriate to previous treatment history, and assessing additional benefit derived from increasing amounts of care per patient as part of cost/effectiveness analysis.     

Putting schools as socio/economic institutions back into educational evaluation: A response to Kean and McNamara

A number of individuals have proposed the gradual refinement of existing programs, rather than their continual closing down, as a safeguard against educational fads and political pressures. The gradual refinement thesis is extended, refined, and criticized in this article. A different unit of analysis is proposed, one which takes into account the role of educational institutions in assisting in the reproduction of both cultural and economic inequality. It is argued that decisions about the gradual refinement of programs and institutions are not only technical but ideological and politico/economic in nature. Criteria for deciding on gradual refinement, based on a strategy of nonreformist reforms, are suggested.     

Client progress toward referral for discharge to the community: an illustration of an evaluation technique.

Data on referral and discharge from a short-term habilitative facility are used to illustrate the "survival analysis" of the rate at which institutionalized mentally retarded persons are released to the community and regression analysis of factors which help predict time to release. These techniques can be used to assess the degree to which habilitative facilities and community agencies achieve their stated goals, to help community agencies plan placement requirements, to compare the progress of various subgroups of clients, and to assess the overall benefit of deinstitutionalization efforts for the entire population at which it is aimed.     

Quality of life assessment and drug abuse treatment program evaluation

Quality of life assessment is used as an outcome measure technique to evaluate a central medical intake (CMI) unit for drug treatment programs. While significant improvements across time are evident in both perceived well-being and in the standard federal funding criteria, no differences are observed between those referred into treatment by the CMI and those entering directly. Moreover, the improvements observed are the same regardless of whether the patient stayed in treatment or not.     

On participant-observation as a component of evaluation. Strategies, constraints and issues

Described here is a participant-observation strategy employed as one portion of the evaluation research at Family House, a residential treatment program for alcoholic mothers and their children. Major objectives of this leg of a two-legged research design combining this approach with a more standard pre- and post-test strategy were to document and analyze the processes of social development in the new "small society." The ethnographer acted as consultant, with "raw data" generated in the form of daily observations dictated on the rotating basis by staff members over a two-year period. The resulting narrative was coded and analyzed by the anthropologist, utilizing a phenomenological or interpretationist perspective which allowed the "native categories" of significance to emerge and guide the analysis. This novel research strategy is compared with standard ethnographic fieldwork, as well as with pre- and post-test design, in terms of its strengths and weaknesses. Strengths include access to an authentic "inside view" of Family House reality, as well as an unusually rich longitudinal record of daily life in the facility. Problems include danger of contaminating the data by means of the "feedback" provided to reward staff for their cooperation in the research effort. The author concludes, however, that since reactive effects are present in all designs, and since "feedback" is a normal feature of therapeutic milieux, validity is not unusually threatened. Finally, the purposes of evaluation research are scrutinized.     

Evaluating and improving prevention programs: a strategy from drug abuse.

This paper describes a methodology used to overcome the difficulties in evaluating and improving prevention programs. The methodology was applied in this case to a drug abuse prevention program. The methodology includes using health-oriented rather than pathology-oriented outcome measures, random assignment to groups, multiple settings, multiple outcome measures, goal-free evaluation, and the recording of process variables. The first year results showed program effectiveness in one setting and not in the other. Process analyses pointed toward improvements that could be made. Those improvements were implemented and led to greater program effectiveness in the second year.     

Accountability to whom?

Social scientists are participating evermore frequently in the evaluation of social action programs. Participation, however, does not always imply social action. In keeping with the tradition of social sciences as critical and debunking activities, practitioners might consider the merits of conceiving themselves as agents of social change rather than technicians responding to emissaries of social control. This position also implies a view of applied research as an active enterprise through which reflective practitioners participate in the definition and direction of applied research efforts. After reviewing implications of non-reflective and reflective stances, the conclusion is that applied researchers adopting a social action perspective actually maximize their obligations as scholar-scientists even though a non-neutral posture is assumed.     

Systems engineering: A problem-solving approach to improving program performance

Evaluation methodologies have generally emphasized the measurement and assessment of program merit, as reflected by the achievement of program objectives. Less attention has been devoted to analyzing and explaining the reasons for deficient performance and, more importantly, what needs to be modified to improve performance. While evaluators do identify performance problems, the identified problems are not always analyzed. Consequently, the information reported to decision makers may be incomplete, inconclusive, and of limited utility in the decision-making process. This article describes an evaluation methodology based on problem solving techniques which can be effective and efficient in defining and analyzing problems which impair program performance. The model can be applied in evaluations where the purpose is to provide decision makers with information and recommendations to improve program performance and provide an alternative to those evaluation models which emphasize the assessment of program merit. The model also provides a potentially unique cost/benefit methodology for estimating the potential worth of improving program performance (i.e., solving the problem).     

The ideological context of child welfare

This paper examines consequences for child development and well-being of the prevailing institutional order and ideology of the United States and concludes that child development in our country is severely obstructed because of socially structured violence which permeates our way of life. The paper ends with comments on prospects for social transformation which seems necessary to assure the development and well-being of every child.     

The evaluator as program consultant

This paper presents a model for the role of the evaluator in which the evaluator acts as a Program Consultant who operates in three domains: Program Development, Planned Change, and Evaluation Technology. This permits the performance of evaluations that have greater validity and utility. The model is presented both from the theoretical perspective of linkage and from the context of the evaluation of a Case Management program operated by a Community Action Agency. Each of the domains is discussed with examples from the Case Management Evaluation. The paper concludes with a discussion of the contexts in which the model is best applied.     

Pre- and post-planned evaluations are complements not alternatives: A reply to Strasser and Deniston

Strasser and Deniston's own analysis shows that post-planned evaluations are unsuitable substitutes for pre-planned evaluations. When viewed as post-experimental interviews, however, post-planned evaluations can produce valuable information which complements traditional experimental and quasi-experimental evaluations.     

The evaluation of treatment in therapeutic communities: Problems of design and implementation

The evaluation of therapeutic communities provides an opportunity to explore the relationship between evaluation methodologies and the constraints imposed by the nature of the treatment setting. In most therapeutic communities the environment represents the primary vehicle of treatment and, as a result, virtually every aspect of the program's operations are, directly or indirectly, related to the treatment goals. This paper examines some of the problems this situation creates for the evaluation of such programs. It suggests that the need to develop an effective working relationship with the program becomes the primary concern of the evaluation process, and it analyzes the use of specific techniques involved in establishing this type of relationship. Such an approach may be useful in other settings where the unique characteristics of the program become the primary constraint in evaluation design and implementation.     

Institutional child abuse

Although various forms of child abuse have been extensively investigated, other forms of psychological and institutional abuse continue to exist. This paper will focus on a variety of experiences occurring in institutions and residential centers that, in effect, cause a variety of detrimental effects and harm to those in those agencies who are supposedly “in treatment.” The various forms of institutional abuse will be examined and ramifications and repercussions explored.     

Making methods choices

This article is a further discussion of methodological paradigms in evaluation research. More specifically, this article is a response to the attacks on paradigmatic perspectives made by Reichardt and Cook in the opening chapter of their edited book Qalitative and Quantitative Methods in Evaluation Research. They “suggest that part of this current debate over qualitative and quantitative methods is not centered on productive issues and so is not being argued in as logical a fashion as it should be.” For better or worse paradigm debates are, by their nature, only partly subject to logical analysis. Paradigms — methodological or otherwise — involve values, world view, empirical tendencies, and patterned responses. Because their central arguments rest entirely on a logical foundation, Reichardt and Cook may have done precisely what they accuse others of doing: “obscuring issues and unnecessarily creating schisms between the two methodtypes.”     

Complexity of process and role in the evaluation of an interdisciplinary mental health education program: The deacon's masterpiece

Research and literature on program evaluation must attend to problems of application, which can have major effect on the outcome. Increasing numbers of innovative, multifaceted programs operating in complex settings are presenting for evaluation. Evaluation process must become flexible, creative, and multifaceted in order to produce valid and comprehensive results, and to meet the needs of this field. Evaluators and evaluation, the evaluated program, and the environment become a mutually interacting system in the process of evaluation. Evaluation goals and roles must be conceptualized, agreed upon, sanctioned, and appropriately implemented or conflict and ineffectiveness will result. Three models of evaluation emerge: objective and independent, objective but serving one interest, and pseudo-scientific public relations. Evaluation of an interdisciplinary mental health education program illustrates these issues. Evaluation technology, process, and roles must all be attended to or the weak link undermines the rest.     

The dysfunctional side effects of quantitative indicator production: illustration from mental health care (a message from Chicken Little)

In the current context of economic conservation, accountability and retrenchment from social programs, justification is required in order to maintain human service capacity. In this effort, evaluation is a key component. Yet desire to base action upon information resulting from evaluation must be tempered by increased attention to the effects of the evaluation process. All too often premature application of quantitative indicators formerly used for individual assessment and research to bureaucratic decision making produces side effects that are dysfunctional in nature. Frequently these side effects influence service delivery directly. More insidious, because they are less easily discerned, are distortions introduced into the data by evaluation pressure. These not only delay effects on service but also impair understanding of the very process they are meant to illuminate. In order to illustrate this phenomenon, the author reviews a general literature and utilized examples from mental health care. Common themes are identified and a tentative theory of side effect generation proposed.