Quality of life assessment and drug abuse treatment program evaluation

Quality of life assessment is used as an outcome measure technique to evaluate a central medical intake (CMI) unit for drug treatment programs. While significant improvements across time are evident in both perceived well-being and in the standard federal funding criteria, no differences are observed between those referred into treatment by the CMI and those entering directly. Moreover, the improvements observed are the same regardless of whether the patient stayed in treatment or not.     

The impact of citizen evaluation review on community mental health center programs

A field study of techniques for involving local citizens in the evaluation of Community Mental Health Center (CMHC) services shows that lay citizens can make recommendations that service agencies will use to improve services. Seventeen citizen groups received evaluation information from participating CMHCs and developed 310 recommendations. Seventy-one percent of these suggestions were used by the CMHCs. Staff and governing board attitudes, fiscal pressures, and other organizational characteristics affected the success of the citizen groups.     

Complexity of process and role in the evaluation of an interdisciplinary mental health education program: The deacon's masterpiece

Research and literature on program evaluation must attend to problems of application, which can have major effect on the outcome. Increasing numbers of innovative, multifaceted programs operating in complex settings are presenting for evaluation. Evaluation process must become flexible, creative, and multifaceted in order to produce valid and comprehensive results, and to meet the needs of this field. Evaluators and evaluation, the evaluated program, and the environment become a mutually interacting system in the process of evaluation. Evaluation goals and roles must be conceptualized, agreed upon, sanctioned, and appropriately implemented or conflict and ineffectiveness will result. Three models of evaluation emerge: objective and independent, objective but serving one interest, and pseudo-scientific public relations. Evaluation of an interdisciplinary mental health education program illustrates these issues. Evaluation technology, process, and roles must all be attended to or the weak link undermines the rest.     

The validity of indices for rural health manpower needs assessment

Population-to-practitioner ratios have long been the primary index in the designation of health manpower shortage areas. This paper documents that application of the widely used population-to-dentist index results in understatement of the need for dental health manpower in rural areas. Through the analysis of utilization data collected from a statewide health screening program in Colorado, the practice of sole reliance on the population-to-dentist indices as an indicator of need was tested. Another measure, the area-(square miles) to-dentist ratio was formulated, examined, and found to be a more useful referent of the need for additional health manpower in rural areas. Utilization of dental services in sparsely settled rural counties of Colorado was unrelated to population-to-dentist ratios. A strong, statistically significant association of utilization with land area-to-dentist ratios was found. The findings of this analysis suggest a need for reevaluation of needs assessment methodologies used in the designation of health manpower shortage areas. Indices more sensitive to consumer circumstance than to the number of health care providers available must be considered.     

Consumer survey as needs assessment method: a case study

Needs assessment methodology has grown remarkably as a necessary part of evaluating human service programs. The consumer survey method of needs assessment is reviewed. A case study in which the training needs of board members of the eleven community mental health agencies in a rural New England state were examined by a questionnaire requesting ratings of the importance and priority of 11 possible topics is presented. Sixty-five of 300 questionnaires were returned (21.66%). The results clearly support the view that training programs for boards are needed, and suggest that citizens can make reasonable decisions about program content if given the opportunity. A training project, based on the results of the survey is described. Problems in designing and implementing consumer surveys and training programs are discussed.     

Enhancing program review for evaluation and management

Program review has not received the attention it warrants as a program evaluation tool despite its wide use for evaluation and management purposes. The use of the program review will probably endure on the strength of its face validity and irrespective of other developments in the field of program evaluation. Evaluators should realize this and, accordingly, attempt to improve its effectiveness. This paper presents one organization's approach to achieving this objective through the explication of development principles, implementation guidelines and review items. This paper also discusses benefits that can be expected from a systematic development of this tool and presents various research directions and potentials in this area.     

Evaluating sociodemographic data: implications for program planning and development

Sociodemographic data can have broad implications for program planning and development. Yet, such data are often obtained and analyzed in a non-rigorous methodological fashion. Four of these methodological issues are discussed in this paper. Their relevance for program planning and development are clarified, using examples from the utilization, needs assessment, illness behavior, and social functioning literature.     

A model for a comparative cost-effectiveness evaluation of two mental health partial care programs.

This study provides a model for comparing the cost-effectiveness of two mental health programs. The example cited examines the cost-effectiveness of a hospital-based and a community-based partial care program. Using each group as its own control, the authors compare patients' use of inpatient hospitalisation and emergency room services during one year prior to and one year subsequent to first admission in day care. In both day care programs, patients substantially reduced their number of inpatient admissions and days. Neither program, however, showed a significant decrease in patients' use of the emergency room. Reflecting the substantially lower costs of the community-based program, the benefit-cost and cost-effectiveness ratios were significantly better in the community-based program than in the hospital-based program. Based on this evidence the authors conclude that the lower-cost, equally effective community-based program should be carefully considered as an alternative for more partial care patients.     

Mental health needs assessment of children and adolescents

Needs assessment is essential input for program planning. Little data are, however, available about the various approaches and whether strategies for needs assessment are viable. It was found by an inexpensive methodology that opinions about the importance of various services, opinions about the potential acceptance of services, services provided, and the total need for services are interrelated. A hybrid approach that utilized data from teachers and the people that provide mental health services was also a practical means to obtain information about the unmet needs of children and adolescents in the country. Data supported the value of this technique for the assessment of the unmet needs of children and adolescents. Thus, an inexpensive methodology can provide useful information from multiple perspectives.     

A critique of social indicators analysis and key informants surveys as needs assessment methods

This paper reports data from a pilot study designed to determine the practicality and utility of two differing needs assessment methodologies: social indicators analysis and key informants surveys. The authors found social indicators analysis successfully identified differential areas of need within the SMSA which served as the research site. They suggest it is a practical and valid means for assessing human service needs at a general level. The key informants survey was judged to be less useful. Informants generally were unable to identify the extent of differing types of needs or their geographic distribution. It is suggested additional research utilizing differing designs must be completed before meaningful conclusions can be reached about the utility of the key informants survey as a needs assessment method.     

To be or not to be: Control and balancing of type I and type II errors

The various costs of Type I and Type II errors of inference from data are discussed. Unfortunately, other things equal, Types I and II errors are inversely related. Six methods of minimizing Type I error in studies are presented, each of which may be employed even after data are collected. Type II errors may also be minimized by a combination of study design and analytic means. Although some of these techniques must be built into the study others can be undertaken when beginning work on a data set. Only one of these techniques will increase Type I error, and those involving data reduction may decrease both types of error.     

Putting schools as socio/economic institutions back into educational evaluation: A response to Kean and McNamara

A number of individuals have proposed the gradual refinement of existing programs, rather than their continual closing down, as a safeguard against educational fads and political pressures. The gradual refinement thesis is extended, refined, and criticized in this article. A different unit of analysis is proposed, one which takes into account the role of educational institutions in assisting in the reproduction of both cultural and economic inequality. It is argued that decisions about the gradual refinement of programs and institutions are not only technical but ideological and politico/economic in nature. Criteria for deciding on gradual refinement, based on a strategy of nonreformist reforms, are suggested.     

Child deaths attributed to abuse and neglect: An empirical study

This study of 267 child deaths associated with abuse or neglect in Texas during 1975 through 1977 suggests a number of indicators for identifying potential child fatalities. Families where abuse or neglect is implicated in a child fatality are characterized by small family size, young parents, and under-utilization of community support services. Over three-fourths of the families in the study had never come to the attention of the state's child protective services agency. In addition, when fathers were present in the home, they were as likely to be involved in the abuse or neglect as mothers. Neglect was implicated in the death of a child as often as abuse. The study suggests the need for further examination of child fatality profiles associated with abuse and neglect and increased community outreach efforts to provide support to high-risk families, The study also has implications for social policy decisions, particularly in relation to efforts to resist attempts to exclude neglect from child protective service responsibility.     

A multi-dimensional client satisfaction instrument

Development of a client satisfaction measure through successive factor analyses is described. Five factors, each defined by five items and accounting for at least 11% of the total variance, were identified and replicated. A single second order factor also emerged in both studies. Reliability of the second order factor, and of three of the first order factors is sufficient for most purposes. The second order factor, which is a linear combination of the first order factors, was not significantly correlated with sex, age, race, or number of attempts made before receiving a client response. The distribution of scores was highly skewed, with most respondents scoring near the upper (satisfied) end.     

The optimal treatment approach to needs assessment

This paper describes a needs assessment technique which has not been considered in previous discussions. Basically the Optimal Treatment approach to needs assessment consists of comparing the most desirable set of services for a client with the services actually recommended or received. Discrepancies due to unavailable resources are noted and aggregated across clients. Systematic shortcomings may be taken to represent service needs.The advantages and disadvantages of this and other procedures are considered, especially with respect to political factors and the utility of data for planning and decision making in the local agency context.     

Evaluating a human service program: Employing the judicial evaluation model

The emergence of adversarial models as an approach to formative and summative evaluations is gaining recognition among educational research professionals. The implementation of the Judicial Evaluation Model (JEM), as described in this article, is the first application to a human service employment and training program. Evaluation questions raised within the study were designed to assess the efficacy of linkage arrangements between the Comprehensive Employment and Training Act (CETA) prime sponsors and education service providers in the Commonwealth of Virginia. The four stages of the JEM and its application to CETA are discussed, the panel findings are reported along with noted pitfalls and strengths, suggested guidelines for implementation, and a few recommendations.     

Institutional child abuse

Although various forms of child abuse have been extensively investigated, other forms of psychological and institutional abuse continue to exist. This paper will focus on a variety of experiences occurring in institutions and residential centers that, in effect, cause a variety of detrimental effects and harm to those in those agencies who are supposedly “in treatment.” The various forms of institutional abuse will be examined and ramifications and repercussions explored.     

Inpatient psychiatric treatment of the difficult child: Common practices and their ethical implications

This article reviews and discusses the available research literature pertaining to the prevalence of seclusion and restraint procedures in inpatient psychiatric treatment facilities for children and adolescence. The rationales for use, potential effects, and factors found to influence the application of these more extreme clinical interventions are considered. The author suggests that the supposed therapeutic function of these types of practices has not been substantiated and their implementation can vary widely from one setting to another. Results from several studies suggest that clearly defined criteria for use and more observational studies of prevalence and precipitant factors could considerably refine these procedures and reduce potential abuses. The psychopharmacologic approach to the treatment of difficult children is examined as a sophisticated counteraggression strategy which has yet to demonstrate therapeutic efficacy. These clinical procedures, which are utilized in scores of inpatient settings, generate serious ethical questions for the health professional concerned with the legal rights of those minors labeled as deviant members of families or the society at large.