Small area indicators and social change

This article examines the relevance of measures of social change for small areas and how these can be used in need assessment. Specifically, this article explicates the problems of selecting a limited group of indicators from among the broad panoply of indicators available, and of determining the different kinds of information provided by indicators controlling for the level of aggregation. The use of small area social indicators to do need assessment is illustrated using data from the 1970 and 1980 Health Demographic Profile System.     

Behavioral epidemiology and social area analysis: The study of blackout looting

This article describes the use of social area analytic techniques in the study of the looting which occurred during the Con Edison power failure of July 13 and 14, 1977. Detailed procedures are outlined of the systematic collection and analysis of data which were developed from available data. Analyses contrasting competing explanatory theoretical models for blackout looting are presented. The central dependent variable is ratio of looted to retail stores. Both the numerator and denominator are aggregated to the census tract level.     

Evaluation of the U.S. Air Force Family Support Center Program

This article provides an overview of the implementation and evaluation of Family Support Centers (FSC) in the U.S. Air Force. After describing the FSC model, the quantitative aspect of the evaluation is discussed, including the evaluation design, implementation methods, survey response rates, and procedures for data analysis. A summary of evaluation results also are presented as well as a discussion of administrative and methodological issues encountered in performing the evaluation. The results of the evaluation suggest that the FSC has become an integral part of the base human service delivery system, especially for married members and civilian spouses.     

To be or not to be: Control and balancing of type I and type II errors

The various costs of Type I and Type II errors of inference from data are discussed. Unfortunately, other things equal, Types I and II errors are inversely related. Six methods of minimizing Type I error in studies are presented, each of which may be employed even after data are collected. Type II errors may also be minimized by a combination of study design and analytic means. Although some of these techniques must be built into the study others can be undertaken when beginning work on a data set. Only one of these techniques will increase Type I error, and those involving data reduction may decrease both types of error.     

Pragmatic decisions in the evaluation of consumer satisfaction with mental health treatment

This article examines pragmatic choices which must be made in conducting consumer satisfaction assessment of mental health treatment. It is argued that choices involving sample, format, and procedure for examining satisfaction may influence the results of such research, and must be considered when creating or evaluating these efforts. These choices are examined in detail.     

The dysfunctional side effects of quantitative indicator production: illustration from mental health care (a message from Chicken Little)

In the current context of economic conservation, accountability and retrenchment from social programs, justification is required in order to maintain human service capacity. In this effort, evaluation is a key component. Yet desire to base action upon information resulting from evaluation must be tempered by increased attention to the effects of the evaluation process. All too often premature application of quantitative indicators formerly used for individual assessment and research to bureaucratic decision making produces side effects that are dysfunctional in nature. Frequently these side effects influence service delivery directly. More insidious, because they are less easily discerned, are distortions introduced into the data by evaluation pressure. These not only delay effects on service but also impair understanding of the very process they are meant to illuminate. In order to illustrate this phenomenon, the author reviews a general literature and utilized examples from mental health care. Common themes are identified and a tentative theory of side effect generation proposed.     

Evaluating a human service program: Employing the judicial evaluation model

The emergence of adversarial models as an approach to formative and summative evaluations is gaining recognition among educational research professionals. The implementation of the Judicial Evaluation Model (JEM), as described in this article, is the first application to a human service employment and training program. Evaluation questions raised within the study were designed to assess the efficacy of linkage arrangements between the Comprehensive Employment and Training Act (CETA) prime sponsors and education service providers in the Commonwealth of Virginia. The four stages of the JEM and its application to CETA are discussed, the panel findings are reported along with noted pitfalls and strengths, suggested guidelines for implementation, and a few recommendations.     

Constructing case vignettes for evaluating clinical judgment: An empirical model

Studies of clinical judgment are frequently based on analogue research, which uses experimenter-controlled stimuli to elicit judgments. The stimuli may be live models, audio- or video-taped models, or written case vignettes describing a clinical encounter. A major challenge of analogue research is to construct stimuli that maintain a balance between experimental rigor and clinical reality. An ideal set of case vignettes, for example, will contain summaries that resemble actual case histories and that are varied only on the specific clinical factors being studied. The model presented here demonstrates the empirical development of written case analogues in which several variables are studied simultaneously. The model can be adapted to any setting in which professionals are required to make judgments or decisions about individuals. The vignettes described here have been used in a variety of clinical settings to assess the reliability of clinicians'judgments and to aid in evaluation and program planning.     

Problem characteristics, decision processes and evaluation activity: a preliminary study of mental health center directors

This article reports on the results of a preliminary study on the characteristics of problems and problem solving in mental health centers. The major findings were that directors were generally quite satisfied both with the available level of information intended to help them make decisions, and with their ability to make good, even optimal, decisions. Much of this confidence resulted from the use of assumptions about the problems which reduced the need for empirical information. The problem solving process was highly dynamic and interactive, with changes over time in the characteristics of problems, constraints and alternative solutions. The directors played an active role in attempting to foster and influence the direction of many of these instabilities. Some implications for evaluation and research are discussed.     

“Paradigms lost”: Some thoughts on choosing methods in evaluation research

Patton agrees with us concerning the need for a catholic approach to method choices in evaluation. However, Patton criticizes us for setting up a straw man when we argued that quantitative and qualitative methods are not inevitably linked to different “paradigms” and that allegiance to a particular paradigm does not force the evaluator into a choice between mutually exclusive qualitative and quantitative methods. The first part of this article shows how some of the past literature creates the impression both of an inevitable method-paradigm linkage and of a forcedchoice between qualitative and quantitative paradigms. It is argued that such impressions could have pernicious consequences for evaluation practice by artificially restricting method choices. In the final section, we consider some of the difficulties of making method choices and suggest that the purposefully developed strengths of different methods should not be completely ignored.     

Evaluating and improving prevention programs: a strategy from drug abuse.

This paper describes a methodology used to overcome the difficulties in evaluating and improving prevention programs. The methodology was applied in this case to a drug abuse prevention program. The methodology includes using health-oriented rather than pathology-oriented outcome measures, random assignment to groups, multiple settings, multiple outcome measures, goal-free evaluation, and the recording of process variables. The first year results showed program effectiveness in one setting and not in the other. Process analyses pointed toward improvements that could be made. Those improvements were implemented and led to greater program effectiveness in the second year.     

Social area analysis in program evaluation and planning

This article defines social area analysis and identifies its conceptual roots and principal applications. By reviewing the uses of demographic data among several disciplines, the utility of social area analysis is assessed. The authors cite and briefly review the seminal and most recent contributions to the field. Several types of ecological fallacies and other problematic facets of social area analysis are considered, as well as specific issues in the derivation of statistical inferences from cross-level and same-level areal and case data.     

Applications of social area analysis to program planning and evaluation

The development of social area analysis as a research technique is highlighted, and two illustrative applications are presented. The first example applies social area analysis to the study of an urban region's demography and the distribution of social problems found there. The second example illustrates how social area analysis can be integrated within a larger evaluation effort. Suggestions for undertaking social area analyses and related demographic studies are presented.     

Pitfalls in planning: Contrasting perspectives on mental health and corporate planning

Planning is a management activity that has been linked to high quality services, organizational growth, and in austere times, to organizational survival. The present study was undertaken to identify the critical barriers that inhibit or derail effective planning. It was predicted that human service and corporate managers would generally agree about the most and least important planning pitfalls in their systems. Discrepancies would be related to differences in the structural characteristics of organizations in the two sectors. Rankings made by 57 senior mental health administrators of the most and least important pitfalls in planning were contrasted with those of 159 corporate executives. Substantial agreement between the two groups of executives was found.     

Evaluating correctional health care. The lessons of New York City.

New York City's correctional health care system is a complicated one, with many component parts. Correspondingly, the task of developing an evaluation process to assess this system turned out to be complex as well. The major responsibility for developing such a process and for keeping it in motion lies with the Contract Evaluation Unit of the Department of Health. Our experiences in meeting these challenges, as well as the historical and institutional factors that have influenced our efforts, should be of interest to professional groups facing similar tasks in other localities and to readers curious about how particular health service evaluation approaches take form. This article describes our method, how it emerged, and that it entails, with an eye toward identifying what might usefully be applied to the study of other correctional health care services and settings.     

A survey of mental health district structure and effectiveness

A national survey using a mailed questionnaire was undertaken to clarify issues of mental health district governance. CMHC, state hospital and district management and governance staff from 49 identified districts classified their districts on 20 types of decision authority and 14 measures of district effectiveness. Each respondent type thought their districts: (1) dealt with state money issues, seeking regional money, and coordinating/changing programs and services, and (2) effectively enhanced planning capabilities. Service providers did not find planning capabilities to be systematically related to specific decision authorities. CMHC management perceived less district control than district officials and, with state hospital management, perceived less effectiveness. District and CMHC officials see a direct relationship between district control and effectiveness. In a subsample of 11 districts each described by multiple respondent types, no item of authority or effectiveness was rated universally positive by all groups. District management subjectivity and legislative implications are discussed.     

The validity of indices for rural health manpower needs assessment

Population-to-practitioner ratios have long been the primary index in the designation of health manpower shortage areas. This paper documents that application of the widely used population-to-dentist index results in understatement of the need for dental health manpower in rural areas. Through the analysis of utilization data collected from a statewide health screening program in Colorado, the practice of sole reliance on the population-to-dentist indices as an indicator of need was tested. Another measure, the area-(square miles) to-dentist ratio was formulated, examined, and found to be a more useful referent of the need for additional health manpower in rural areas. Utilization of dental services in sparsely settled rural counties of Colorado was unrelated to population-to-dentist ratios. A strong, statistically significant association of utilization with land area-to-dentist ratios was found. The findings of this analysis suggest a need for reevaluation of needs assessment methodologies used in the designation of health manpower shortage areas. Indices more sensitive to consumer circumstance than to the number of health care providers available must be considered.