The value of unattractive characteristics in social entitlements.

Social programs legislated as entitlements are attractive to promoters of health and welfare services because of the programmatic security and stability such a status offers. Yet existing entitlements and other publicly funded services routinely fail to exhibit the high performance standards that an entitlement status requires. Such "flaws" as confusion over purpose, obstructive procedures, inefficient administration, and cost overruns can also serve important functional and political purposes. Therefore, administrators and evaluators who are in a position to witness these discrepancies will sometimes find their roles as manager and critic compromised. Instead of pursuing policies that would satisfy entitlement principles, they may only be in a position to implement and defend the "flaws" as reasonable compromises. There is the risk, however, that the public will remain skeptical about overall programmatic results despite optimistic evaluation reports. More careful legislation, more disciplined administration, and more neutral evaluation will be required if human services are to retain the public's confidence.     

The evaluation of treatment in therapeutic communities: Problems of design and implementation

The evaluation of therapeutic communities provides an opportunity to explore the relationship between evaluation methodologies and the constraints imposed by the nature of the treatment setting. In most therapeutic communities the environment represents the primary vehicle of treatment and, as a result, virtually every aspect of the program's operations are, directly or indirectly, related to the treatment goals. This paper examines some of the problems this situation creates for the evaluation of such programs. It suggests that the need to develop an effective working relationship with the program becomes the primary concern of the evaluation process, and it analyzes the use of specific techniques involved in establishing this type of relationship. Such an approach may be useful in other settings where the unique characteristics of the program become the primary constraint in evaluation design and implementation.     

Making a case for an interface between quasi-experimental designs and social impact analyses

This paper describes the community impact assessment conducted after Concorde operations were initiated at Dulles International Airport. Problems in the design and conduct of this study that make causal interpretations implausible are discussed. The present authors report an improved design to eliminate these problems as well as increase the external validity of the results. A discussion of how the careful implementation of quasiexperimental designs allows for treatment-related causal interpretations to be made is also included. Finally, it is emphasized that practical application of these data make it possible to improve the implementation of the treatment in communities receiving it at a later date.     

The foster care system simulation: Evaluation of a training innovation

Although foster care is often spoken of as a system with many interactive facets, foster care personnel often carry out their specific roles with little understanding or consideration of the ultimate impact of that system. A simulation of public foster care service delivery designed to sensitize participants to these interactions is described. Evaluation of the simulation's accuracy in portrayal of actual service delivery and a study of the kinds of learning demonstrated by participants at three points (pretest, posttest, and follow-up) are reported. Participants found the simulation an attractive training method, indicated a greater appreciation of system-wide foster care problems, and reported changes in foster care thinking and work, but other indicators of learning were inconsistent.     

Evaluation and social policy. Some reflections on ideas and institutions

This paper discusses the relationship between evaluation as a meta- and as a technical activity in the field of social policy, with special reference to the British experience. Evaluation as a meta-activity is defined to be the analysis of the values and criteria to be used: a process which assumes that the definition of problems is itself problematic. Evaluation as a technical activity is defined to be the application of agreed criteria to agreed policy problems. From this distinction, it is argued, some important implications flow for the role of the universities and the funding of research.     

How evaluators can deal with role conflict

Unclear professional norms create problems of role conflict for program evaluators. Four circumstances that produce role conflict and suggestions for dealing with each one are examined. The influence of the organizational context on role conflicts is also considered.     

The public and private purposes of case records

Social welfare agencies have official reasons and requirements for the construction of case records. Sociologists have detected a set of unofficial reasons that shape case records in practice. This paper suggests that case records that record the lives of children in child welfare systems are written, inter alia, to deny the failure of interventions, to justify the refusal to serve “bad clients,” and to justify the decision to extend hegemony over “good” clients.     

The role of the father in the family: Toward a gender-neutral family policy

A review of the literature suggests that: (a) Women are not the exclusive potential contributors to quality child care; (b) men can provide care equally well; (c) a growing number of men want to increase their involvement in child care. A number of obstacles impede, however, increased paternal involvement in child care; these include the existing societal value system, the present structure of the labor market, and the nature of many existing social services. Therefore, various aspects of social and family policy are examined and a variety of options are proposed.     

Comparative influence of political, cost/benefit, and statistical information on administrative decision making.

The influence of three types of evaluation information was studied in simulated decisionmaking situations within a community mental health center setting. Twelve administrators reviewed one of two decision problems (direct services or indirect services funding) and were then presented political, cost/benefit and statistical information. After each presentation, the administrators rated the importance of each type of information for decision making. Only the cost/benefit data were rated as significantly influencing their decisions. The influence of each type was not dependent upon the decision problem, nor to length of time in administrative positions. The results were interpreted in terms of “who says what, how, and to whom. ” If evaluators desire a greater impact on the decision making behavior of community health administrators, they should consider employing a pay off-based evaluation strategy and providing cost/benefit information.     

On the limiting properties of social indicators

Characteristics of the population and economic measures that constitute traditional social indicators are compared with more recent “quality of life” measures to demonstrate that social indicators are always value statements at the policy level. The possibility of alternative perspectives is illustrated. It is suggested that an awareness of the assumptions implicit in any given indicator is as important as the data they provide.     

An examination of methods in the evaluation of volunteer programs.

Although many volunteer programs have been called "successful" and the use of paraprofessionals urged, few program evaluations with a benefit-cost analysis have been reported. Several major studies of volunteer utilization are reviewed, and particular needs with reference to program evaluation are discussed. Three alternative evaluation methodologies are proposed. The importance of utilizing a comprehensive evaluation incorporating all three methodologies is discussed. An ongoing volunteer program in a community mental health center is used as an illustration.     

An analysis of variables affecting length of stay in foster care

The purpose of this paper was to find out whether the relationship between type of contact with agency and/or child in foster care and average number of contacts over time makes any difference in length of stay in foster care. Four types of contacts were examined: Family—Agency, Family Initiated—Agency, Social Worker—Initiated, and Family—Child. Results indicated that there were critical numbers for each contact type found to be statistically significant that would tend to reduce the length of the child's stay in foster care. The importance of contact during the first month of placement was emphasized.     

Measuring corporate social performance: The problem of values

Debates over the social performance of corporations have carried with them implicit evaluations of that performance. The apparent subjectivity and bias of these informal evaluations has led to the development of more formalized corporate social performance assessment methodologies — social audits. In this paper it is argued that these social audits are also influenced by subjectivity and bias considerations; and thru the case study of a social audit of a corporate safety program, the underlying value issues at the core of these influences are explored. The value implications of the various social audit methodologies in use are discussed; and several value sensitive stages/elements of social audit research and evaluation research in general are identified. Recommendations set forth for minimizing the negative consequences of value-influenced evaluations are a value-explicit evaluation approach, greater standardization of evaluation procedures, a dialectic evaluation approach, and better informed evaluation consumers.     

Professionalization of evaluative research: Conflict as a sign of health

Morell and Flaherty (1978) present arguments about the current status of evaluative research as a profession. This paper contains a critique, using a process model of professional development, of the perspectives employed by Morell and Flaherty. Issues discussed are the nature and substance of conflicts in evaluative research, including conflicts over methodological perspectives, the role of the evaluator in program planning, and the appropriate uses and users of evaluative information. The paper concludes with a discussion of the needs within the profession of evaluative research if it is to remain viable and responsive to change.     

Treatment participation and outcome in a program for problem drinker-drivers

Relationships were examined between 13 predictor variables and two criteria of improvement in a program for problem drinker-drivers. Four measures of social background (income, educations, age, marital status), four measures of psychopathology based on the Minnesota Multiphasic Personality Inventory (MMPI), one measure of general alcoholism, and four indicators of treatment involvement served as predictor variables. Two factorially derived criteria served as outcome indicators, one reflecting general improvement, the other reflecting improved social adjustment, but a failure to reduce drinking. This second factor was interpreted as indicating resistance. Analysis included partial correlation and multiple regression, from which explained variance was completely partitioned into sources unique to each predictor, and sources shared by combinations of predictors. Results suggested that with other factors controlled, treatment involvement was linked both to improvement and to resistance. Similar conclusions were suggested for social background, but slightly different ones for psychopathology and alcoholism. Psychopathology was uniquely related to improvement; alcoholism, to resistance.     

Dealing with unknown variables in policy/program evaluation

A leading objection to the feasibility of applying benefit-cost analysis in evaluation is that one often cannot know the benefits, the costs, or the probabilities of their occurring for a given proposal. Threshold analysis involves handling unknown variables by converting relevant evaluation problems into questions as to whether a given benefit, cost, probability, or combination of these elements is more or less than a threshold. Above the threshold, the proposed program would be relatively profitable; and below the threshold, it would be relatively unprofitable. Applying that analysis may require the drawing of simple graphs to stimulate the minds of knowledgeable persons as to the range of the actual benefits, costs or probabilities. The approach can be applied when the benefits or the costs are either monetary or nonmonetary, and when the problems involve either go/no-go decisions or conflicting-choice decisions.     

Documentation in evaluation research: Managerial and scientific requirements

Documentation in evaluation research consists of written material, in human- or machine-readable form, pertaining to the plans, activities, and results of the project. It is argued here that good documentation is essential for effective management of evaluations, and for responsible reporting of the research procedures and findings. Documentation relating to electronic data processing activity is especially important. The purpose of this paper is to stimulate consideration and discussion of documentation, and to emphasize its importance in evaluation research. The role of documentation in the planning and control functions of project management is reviewed, and the importance of documentation in the assessment of research quality with respect to objectivity, validity, and replicability is discussed. Reasons for poor documentation are considered. An outline of documentation required in different phases of research projects is provided, and recommendations for improving the quality of documentation are presented.