Triangulation in practice

An evaluation study was designed using a between-method triangulation strategy. Two study teams gathered and analyzed data about the effectiveness of a formal process for long range program planning. One team used a mail questionnaire and the other, on-site open-ended interviews. Initial plans for coordination between the two teams were misguided and, fortunately, were subverted during data collection and analysis. The evaluation strategy and its derailment are discussed in terms of the purposes of triangulation and the forces that work against sensible intentions for coordination in multimethod studies.     

Developing and transferring behavioral technologies for children and youth

Behavioral researchers have developed a variety of technologies for improving the capacities of children and their caregivers. Despite the availability of behavioral technologies for children and youth, their widespread adoption and sustained use has been limited. Characteristics for appropriate (e.g., sustainable, effective) technologies for children and youth are suggested here, and strategies for research and development of these methods are described. This paper outlines the process of transferring behaviorial technologies including the phases of awareness, interest, assessment, trial, implementation, maintenance, and reinvention. Finally, issues implicit in the process of transferring behavioral technologies for children and youth are discussed.     

The use of metaevaluation to plan evaluations of conservation programs

The U.S. Department of Energy (DOE) is sponsoring and will be required to evaluate a wide range of conservation programs. The development of evaluation plans for such programs can often be improved by reviewing evaluations that have been done on similar programs. The Residential Conservation Service (RCS), sponsored by DOE, is discussed as an example of how conducting metaevaluations can improve evaluation planning. Many utilities have sponsored programs that are similar to the RCS. A review of the utility program evaluations suggested key methodological issues and data problems that should be considered in the development of an RCS evaluation plan. The implications of these metaevaluation findings for an RCS evaluation plan are discussed.     

The diversion of delinquent youth: family-frcused intervention

This paper describes the practices of the Adolescent Diversion Project and presents some research evaluating the program's effectiveness. The youth were referred to the project as an alternative to further court processing. These youth were randomly assigned to the project or to a control group which received treatment as usual by the court. Project youth were assigned to work with college undergraduates for 18 weeks. The two basic intervention techniques employed were behavioral contracting and advocacy. Project youth were further randomly assigned to one of two intervention conditions within the project. The Multi-Focus group applied the above strategies to a variety of social domains of the youth, while the Family Condition focused entirely on the family. Research results focused on standard outcome criteria (police, court, and school data) as well as on the degree to which the experimental conditions differed in practice.     

Parents' searches for child care and the design of information services

This article reports selected findings from a study of the process by which urban parents search for child care. Analyses of interview data indicate that (a) parents used informal sources of information to a considerably greater extent than formal sources, (b) the general pattern was to begin the search with relatives and close friends and gradually move out to more peripheral ties, (c) there does not appear to be an effective informal system of neighbors or community persons who matched parents with child care providers, and (d) there was no single or several sources that were outstanding in effectiveness for a large proportion of parents. The article sets forth implications of these data for the design of child care information and referral services in urban settings.     

The use of the nominal group technique to isolate teacher job stressors

The nominal group technique, a small-group decision-making process which offers open and relatively unrestricted response to problem identification and resolution, was used as a means of identifying the major stressors of elementary school teachers and the concomitant solutions to their problems. Although the specific stressors that have been presented in this paper are not necessarily applicable to other school districts, the nominal group technique itself should be very useful in other school districts as well as in most any work setting.The active involvement of teachers in defining and solving their own problems is a critical aspect of the technique. Teachers, like most employees, will more actively solve problems and accept innovations if they can see some tangible benefit for themselves in doing so.“Intensity of work demands” was the most frequently mentioned job stressor and also received the highest combined ranking for the nine teacher groups. “Student misbehavior and lack of motivation” ranked a close second.     

The social “R & D” paradigm in child and youth services: Building knowledge convivially

Adaptation of agricultural and industrial research and development models offers new methods for bridging the worlds of research and practice in child and youth services. Social R & D methodologies provide effective means for generating new social technologies because they are product oriented and use a wide array of traditional and nontraditional data sources. This article discusses one social R & D paradigm—Developmental Research and Utilization. Permanency planning and community support systems research are used to illustrate the past and potential application of this methodology to the field of child and youth services.     

Applicability of standards for evaluations of educational programs, projects and materials in an international setting

This paper starts with a short discussion of the book Standards for Evaluations of Educational Programs, Projects and Materials by the Joint Committee on Standards for Educational Evaluation (McGraw-Hill, 1981). It then goes on to examine the usefulness in countries other than the United States of these standards and argues that evaluation of “programs” is more complicated and more politically biased than evaluation of “projects” and “materials.” In Europe, evaluation of educational programs (school systems) has proved to be highly correlated with the overriding objectives of different political parties, e.g., conservative, liberal, socialist, whereas educational policy in the United States is less bound to political parties, varying rather from case to case. In order to show how generally agreed upon educational goals can be interpreted according to different values, the author has reanalysed some studies of mean achievement in reading comprehension and science for 10- and 14-year-old students in 15 countries (IEA Project, 1973). The rank order of these countries is changed if, instead of national mean achievement, one takes proportion of students under a minimum level or above a prescribed top level as the criterion. Also the national (program) standard is evaluated in terms of between-school variance in results and in proportion of single-school units with exceptionally low or exceptionally high mean results. The conclusion is that the book on standards for evaluations of educational programs provides a good checklist of prerequisites for a reliable and valid evaluation. However, it is also concluded that—due to differences in values of program outcomes—such standards do not guarantee that the result of the evaluation will be indisputable.     

The marginal high school student: defining the problem and searching for policy

The central issue explored in this paper concerns those principles and objectives that should guide the formulation of policy aimed at the marginal high school student. Recent efforts to make these youth more employable through specialized and remedial programs are examined. After the general ineffectiveness of such programs is noted, an analysis is offered which links delinquency and adolescent development theory. The concept of adolescent social development is explored along with the effects of experiential education programs as a context for stimulating this development. Finally, to promote the use of programs which provide for adolescent social development, a policy of modified vouchers is advocated.     

Occupational stress management: Cognitive-behavioral approaches

Human service professionals have been identified as subject to burnout because of stress in the work environment. A number of theorists have emphasized the role of cognitions in psychological stress. Cognitive-behavioral programs which help individuals develop coping skills in order to increase ability to deal with job-related stress are reviewed. Implications for occupational stress management programs and directions for future research and evaluation in the area are discussed.     

The impact of general accounting office program evaluations on government

General Accounting Office (GAO) audit reports have an impact on government operations by encouraging growth in the size and scope of the state and by encouraging the Congress and executive agencies to attend to the auditor's dominant values of economy, efficiency, and effectiveness in government. Through its financial, management, and program audits, the GAO program evaluators seek to rationalize agency management and programs so that more resources are required. The GAO's program evaluation efforts also serve Congress by providing for oversight of public policy, a bludgeon to coax agencies into pursuing activities preferred by congressmen, and a tool for electoral activities. In the end, the evaluation auditors are doomed to frustration in achieving their ideals for effective democratic government. This is so because “practical business policy” is irrelevant to government operations and inconsistent with the core values of a democratic state.     

Educating handicapped children: The development of a special education ideology scale

The ideological belief system of actors who influence the direction of policies is an important dimension providing some estimate of a policy's feasibility. This paper, through the development and application of a Special Education Ideology (SEI) Scale, examines what a randomly selected group of local educational decision-makers in Massachusetts think regarding the education of handicapped children within the contexts of citizen/parent participation, intergovernmental relations, community/agency linkage, and normalization. Their belief system, and extent of mainstreaming in Massachusetts, are analyzed according to community type—urban, small city, suburban, and rural. With respect to Public Law 94–142, the Education for All Handicapped Children Act, implications and limitations of the SEI Scale are explored for education and human service professionals.