An analysis of variables affecting length of stay in foster care

The purpose of this paper was to find out whether the relationship between type of contact with agency and/or child in foster care and average number of contacts over time makes any difference in length of stay in foster care. Four types of contacts were examined: Family—Agency, Family Initiated—Agency, Social Worker—Initiated, and Family—Child. Results indicated that there were critical numbers for each contact type found to be statistically significant that would tend to reduce the length of the child's stay in foster care. The importance of contact during the first month of placement was emphasized.     

Problems in being responsive: Reflections on an evaluation of a program for training motorcycle riders

This paper discusses some methodological issues arising from an evaluation of the Queensland Road Safety Council Motorcycle Training Program. The basic orientation of the evaluation was responsive/illuminative; responsive in the sense that the evaluation proceeded interactively, with appropriate issues and procedures identified and modified progressively through direct observation of the program and discussion with the participants; illuminative in the sense that the final report attempted to provide a detailed account of the program operation and of its strengths and weaknesses as perceived by various stakeholders. An account is given of the main considerations in designing, implementing, and reporting the evaluation. Implications for future evaluations are discussed.     

The evaluation and selection of adequate causal models: a compensatory education example

Procedures for ascertaining relative model adequacy in latent variable structural relations models are discussed. Under diverse methods of estimation, this determination may be assessed using the chi square goodness of fit statistic, incremental fit indices for covariance structure models, and latent variable coefficients of determination. An example from evaluation research is taken (cf. Magidson, 1977; Bentler & Woodward, 1978). Numerical sensitivity of parameter estimates under alternative model specifications is demonstrated. Interpretive implications based on these procedures are discussed in terms of parameter sensitivity to alternative model specifications.     

On considering the unintended impact of evaluation: reactive distortions in program goals and activities

The relationship between program planning and evaluation can be viewed as bidirectional; that is, evaluation methods, procedures, instruments, and criteria not only are determined by, but also influence, program goals and activities. Within the human services context, several factors or sources of reactivity between evaluation and program planning can be identified. These involve (a) quantification of goals and activities, (b) preferences by different audiences for various kinds of evaluation data, (c) values and evaluation criteria, and (d) evaluation requirements and resource availability. Effects of these reactive features are discussed and illustrated with examples drawn from mental health evaluation and accountability practices. It is argued that for evaluation to be a credible and useful practice, evaluators should plan their efforts and assess their own effectiveness within the larger context of human service systems.     

Experimental and contextual models of program evaluation

Evaluation research is one of the most rapidly evolving fields of applied behavioral science. As demand for program assessment has increased, the number of alternative evaluation approaches has also grown. As a result, everyday practitioners have often lacked sufficient guidelines for the choice of appropriate evaluation strategies.The present paper articulates an underlying epistemological distinction between (a) experimental evaluation models which simplify program realities in generalizable analyses of discrete causes and effects, and (b) contextual evaluation models which holistically examine particular program operations. These two evaluation approaches are directed at different purposes and are applicable to different program settings. A topology of program characteristics (breadth of goals, scope of treatment, specificity of results, and clarity of theory) is developed and linked to the appropriateness of experimental and contextual evaluation.     

The evaluator as program consultant

This paper presents a model for the role of the evaluator in which the evaluator acts as a Program Consultant who operates in three domains: Program Development, Planned Change, and Evaluation Technology. This permits the performance of evaluations that have greater validity and utility. The model is presented both from the theoretical perspective of linkage and from the context of the evaluation of a Case Management program operated by a Community Action Agency. Each of the domains is discussed with examples from the Case Management Evaluation. The paper concludes with a discussion of the contexts in which the model is best applied.     

Using macro variables in program evaluation

This paper discusses the importance of using macro-level variables for program analysis. It is argued that three types of macro variables--cultural, economic, and demographic--provide a context for evaluation, and offer alternative explanations for the success or failure of a program. The problems of causality and availability of data are discussed. Macro variables are seen to be especially useful in "meta evaluation," and in integrating the results of specific program evaluations into general conclusions.     

Applications of social area analysis to program planning and evaluation

The development of social area analysis as a research technique is highlighted, and two illustrative applications are presented. The first example applies social area analysis to the study of an urban region's demography and the distribution of social problems found there. The second example illustrates how social area analysis can be integrated within a larger evaluation effort. Suggestions for undertaking social area analyses and related demographic studies are presented.     

The impact of general accounting office program evaluations on government

General Accounting Office (GAO) audit reports have an impact on government operations by encouraging growth in the size and scope of the state and by encouraging the Congress and executive agencies to attend to the auditor's dominant values of economy, efficiency, and effectiveness in government. Through its financial, management, and program audits, the GAO program evaluators seek to rationalize agency management and programs so that more resources are required. The GAO's program evaluation efforts also serve Congress by providing for oversight of public policy, a bludgeon to coax agencies into pursuing activities preferred by congressmen, and a tool for electoral activities. In the end, the evaluation auditors are doomed to frustration in achieving their ideals for effective democratic government. This is so because “practical business policy” is irrelevant to government operations and inconsistent with the core values of a democratic state.     

Thomas Szasz' “critical reflections on child psychiatry”: An alternative view through a systemic lens

In “Critical Reflections on Child Psychiatry,” Thomas Szasz (1979) argues that children are without the rights afforded adults, that children are often coerced into therapy, and that most children do not appear to benefit from psychotherapy. Szasz raises many pertinent questions for mental health professionals working with children. However, his arguments appear to be based on literature limited to traditional psychodynamic approaches. His position also neglects to consider literature in the developmental psychology and legal fields. This paper, written from the cognitive developmental and the family systems perspective, provides another examination of the important issues raised by Thomas Szasz.     

Dealing with unknown variables in policy/program evaluation

A leading objection to the feasibility of applying benefit-cost analysis in evaluation is that one often cannot know the benefits, the costs, or the probabilities of their occurring for a given proposal. Threshold analysis involves handling unknown variables by converting relevant evaluation problems into questions as to whether a given benefit, cost, probability, or combination of these elements is more or less than a threshold. Above the threshold, the proposed program would be relatively profitable; and below the threshold, it would be relatively unprofitable. Applying that analysis may require the drawing of simple graphs to stimulate the minds of knowledgeable persons as to the range of the actual benefits, costs or probabilities. The approach can be applied when the benefits or the costs are either monetary or nonmonetary, and when the problems involve either go/no-go decisions or conflicting-choice decisions.     

Mental health needs assessment of children and adolescents

Needs assessment is essential input for program planning. Little data are, however, available about the various approaches and whether strategies for needs assessment are viable. It was found by an inexpensive methodology that opinions about the importance of various services, opinions about the potential acceptance of services, services provided, and the total need for services are interrelated. A hybrid approach that utilized data from teachers and the people that provide mental health services was also a practical means to obtain information about the unmet needs of children and adolescents in the country. Data supported the value of this technique for the assessment of the unmet needs of children and adolescents. Thus, an inexpensive methodology can provide useful information from multiple perspectives.     

An organization-environment framework for assessing program implementation

Evaluations of health programs are typically conducted from a rational bureaucratic framework in which client change and client outcome are seen as logical outcomes of program activity. This paper proposes a theoretical framework for examining organizational consequences of program implementation. Factors in the internal environment (e.g., the hospital) and the external environment (e.g., the community) which contribute to the program's success or failure, are considered. Observational and structured interview data are used to compare two state funded perinatal projects, one an organizational "success" and the other a costly failure. Attributes of external environments which may be important to successful implementation include demand for services, provider competition, access to information, social beliefs, and professional commitment. Important features of the internal environment include host organization commitment and conflict, and structural attributes: complexity, formalization, centralization, and coordination. The paper concludes with theoretically-based questions which serve as guidelines for process evaluation.     

A cost-benefit analysis of a smoking cessation program

Smoking cessation programs are usually evaluated in terms of the percentage of participants who have stopped smoking at least 6 months after the program's completion. This paper shows how a relatively low rate of behavior change may actually result in a high ratio of benefits to costs. Cost-benefit analysis is done for a specific program but the procedures and cost estimates are generalizable to other smoking cessation programs. Benefits to the firm included reduced costs of insurance and the savings due to employee absenteeism and disability. Costs included the program costs as well as the opportunity costs of the participants. Data from the literature and from the specific program are combined to form conclusions about the effectiveness of the program.     

Behavioral evaluation of a state program of deinstitutionalization of the developmentally disabled

The purpose of this study was to evaluate one phase of a statewide program of deinstitutionalizing developmentally disabled individuals. Forty adults who had spent a minimum of six months in community placements were evaluated with respect to changes in behavior associated with movement out of state institutions and into community settings such as group homes and day activity centers. Pre- and post-deinstitutionalization measures were taken with the Behavior Development Survey. Post-deinstitutionalization measures on Cataldo and Risley's Resident Activity Manifest were compared to the same measures obtained on 159 developmentally disabled individuals scheduled for deinstitutionalization. A consistent pattern of positive changes on both instruments favored deinstitutionalization, but changes could not be unequivocally attributed to movement out of the institutions per se.     

Paternalism and the young

A critical analysis is made of the concept of paternalism, and the manner in which protective measures on behalf of children often result in a denial of rights to the young. The article challenges the assumption that parents invariably know the needs of their own children, and accordingly act on their behalf. Similar reservations are made as to the capacity of official caretakers acting as agents of the state, to replace parents who are at fault. Conversely, it is suggested that children often have greater capabilities to function independently than is generally acknowledged. However, attention is also drawn to the defects in the arguments of some child-rights advocates. In sum, the article argues for the need to seek out legitimate paternalistic practices where genuinely needed, where sufficient consideration has been given to the capacities of the young for an independent life, and where sufficient sensitivity has been shown to their aspirations.     

The goal-oriented approach to evaluation: Critique and case study from drug abuse treatment

The specification and measurement of program goals remains central to most evaluation research strategies, yet procedures for implementing this approach are not well-articulated. It is the purpose of this discussion to describe a stepwise procedure for programmatic goal setting and monitoring used in a demonstration drug treatment program for women. Three implementation steps are described: (a) goal setting, (b) checking for consistency, (c) monitoring and feedback. The advantages and limitations of this approach are discussed and useful complementary measurement strategies are suggested.