Triangulation in practice

An evaluation study was designed using a between-method triangulation strategy. Two study teams gathered and analyzed data about the effectiveness of a formal process for long range program planning. One team used a mail questionnaire and the other, on-site open-ended interviews. Initial plans for coordination between the two teams were misguided and, fortunately, were subverted during data collection and analysis. The evaluation strategy and its derailment are discussed in terms of the purposes of triangulation and the forces that work against sensible intentions for coordination in multimethod studies.     

The dysfunctional side effects of quantitative indicator production: illustration from mental health care (a message from Chicken Little)

In the current context of economic conservation, accountability and retrenchment from social programs, justification is required in order to maintain human service capacity. In this effort, evaluation is a key component. Yet desire to base action upon information resulting from evaluation must be tempered by increased attention to the effects of the evaluation process. All too often premature application of quantitative indicators formerly used for individual assessment and research to bureaucratic decision making produces side effects that are dysfunctional in nature. Frequently these side effects influence service delivery directly. More insidious, because they are less easily discerned, are distortions introduced into the data by evaluation pressure. These not only delay effects on service but also impair understanding of the very process they are meant to illuminate. In order to illustrate this phenomenon, the author reviews a general literature and utilized examples from mental health care. Common themes are identified and a tentative theory of side effect generation proposed.     

On considering the unintended impact of evaluation: reactive distortions in program goals and activities

The relationship between program planning and evaluation can be viewed as bidirectional; that is, evaluation methods, procedures, instruments, and criteria not only are determined by, but also influence, program goals and activities. Within the human services context, several factors or sources of reactivity between evaluation and program planning can be identified. These involve (a) quantification of goals and activities, (b) preferences by different audiences for various kinds of evaluation data, (c) values and evaluation criteria, and (d) evaluation requirements and resource availability. Effects of these reactive features are discussed and illustrated with examples drawn from mental health evaluation and accountability practices. It is argued that for evaluation to be a credible and useful practice, evaluators should plan their efforts and assess their own effectiveness within the larger context of human service systems.     

Quality of life in the evaluation of community support systems

As a result of the authors' experience in conducting an evaluation of the community support system (CSS) program in New York State, they have identified five reasons for focusing on quality of life (QOL) as a desired outcome for programs for the chronically ill. These reasons are presented and problems in QOL evaluative research are discussed. Psychological indicators are distinguished from social indicators of QOL, and two methods of operationalizing perceived QOL, the psychological well-being and life domains methods, are examined. A conceptual model is presented and the results of an exploratory study of the QOL of 118 chronic psychiatric patients receiving CSS services are presented.     

An analysis of nonpartisan election media expenditure decisions using limited dependent variable methods

In order to maximize the information disseminating function of campaigning, candidates for political office must make numerous decisions regarding the allocation of resources for various campaign activities. One of the most important financial decisions a candidate must make occurs in determining the extent and character of mass media expenditures. To determine mass media budget allocations, two decisions are required. First, the type(s) of mass media to employ in the campaign, and second, the amount of the budget to allocate to each media selected. These two decisions are not necessarily independent since high fixed costs are associated with employing electronic media.In this paper, we examine one particular form of mass media, television, and its use in nonpartisan elections. Because of the nature of the decision process in using television advertising an estimation technique for limited dependent variables is employed to predict first, the probability that a candidate will use television advertising in a campaign, and second, the amount spent on such advertising. The occurrence of these outcomes is considered to be a function of both the total financial resources available to the candidate and the type (importance) of the political office sought.     

Misogyny in New York custody decisions with parental alienation and child sexual abuse allegations

This article analyzes 24 New York (NY) published child custody cases decided between 2001 and 2017 that contained parental alienation and child sexual allegations. It addresses whether there was a tendency toward gendered decisions and the evidence on which the decisions were based. It reveals that most decisions favored alienation allegations over child sexual abuse allegations and transferring custody from mothers communicating sexual abuse allegations in court to fathers defending against them by alleging parental alienation. On appeal, these family court decisions were overwhelmingly upheld. The analysis also shows that the decisions were based as often on implicit misogynistic cultural assumptions in the absence of allegation-specific evidence as they were on allegation-specific evidence. The article adds to the growing understanding of sources of bias by proposing four errors that support biased reasoning. It concludes with suggestions about how experts, attorneys, and judges can question themselves and others to reduce bias.     

Protectionism: Old wine in new bottles

This paper addresses some new types of arguments that have recently arisen in defense of protectionism in developing and developing countries. The new sources of protectionism in developing countries discussed are: (1) recession-debt-crisis costs; (2) protection in the world economy; (3) wages and labor markets; (4) political requirements. The new arguments for protectionism in developed countries discussed are: (1) shift to the service economy; (2) defense; (3) threat of protection to induce foreign investment.     

The conduct of program evaluation reviews in the general accounting office

Professional behavior during the General Accounting Office's (GAO's) present era of oversight can best be understood by reviewing the audit types that populate village life in the GAO and the repertoire the office uses in conducting program evaluation audits. Four types of auditors are present, each of which has a role to play in the audit repertoire. This repertoire focuses on finding faults in government programs in order to catch or sustain congressional interest in GAO's work. The “audit trail” of each report may last up to 3 years and involves considerable organizational strife and conflict with outside groups. To sustain themselves through this arduous auditing process, auditors have developed a creed through which they conceptualize themselves as an elite with a mission to insure good government.     

Handling problems of ecological fallacy in program planning and evaluation

The issue of ecological fallacy is highlighted in the use of social area analysis as a method of program planning and evaluation. Data on psychiatric admissions from one catchment area in New York City show misleading cross level generalization to be a real and ever present danger that planners and evaluators should not ignore. A simple method of assessing the degree and major sources of difference between higher (geographic) and lower (sociological) level relationships is illustrated. Suggestions are made regarding ways of clarifying cross level generalizations.