Evidence-Based Social Work Practice with Mentally Ill Persons who Abuse Alcohol and Other Drugs

Abstract

The current article outlines a comprehensive approach to evidence-based social work practice, and applies it to persons with severe and persistent mental illness who also abuse alcohol and other drugs. Representative empirical literature is summarized within a framework that delineates the three major functions of evidence-based social work practice: assessment, intervention and evaluation. Assessment protocol, which incorporates the use of valid scales to complement the qualitative interview, is based on domain-specific research that outlines relevant psychosocial risk factors, and highlights those that are amenable to change. Intervention strategies are derived from the growing body of controlled research findings, but flexibility in implementation is recommended to accommodate clients' individual needs and the vagaries of daily practice. Naturalistic evaluation methods are used to capitalize on the use of brief, valid process and outcome measures to augment individual qualitative evaluation and to aggregate data for program evaluation. The implications of this integrated evidence-based strategy for social work practice are discussed.     

Network analysis in evaluation: some words of caution

Network analysis, a methodology derived from general systems' theory, can be utilized as a community mental health administrative-evaluation procedure. Evaluation parameters derived from analysis of patient data as they “flow” through a network of agency services, provide measures of systemic functioning. These parameters include “longest paths” and various ratio relationships as evaluation measures of service delivery. The limitations of network analysis are examined by means of conceptual analyses, and phenomena that emerged from research experience. The necessity for both quantitative and qualitative data to ensure a meaningful evaluation of mental health services is explained. Conclusions about the value of the network analysis approach are considered.     

Quantifizierung

The challenge in data analysis often lies in taking account of the multi-dimensionality and complexity of the data and at the same time discovering patterns, which requires reduction and simplification. Quantitizing qualitative data can serve as an additional step during data analysis to broaden the perspective and collect complementary views. Examples from research illustrate how information gathered during one data collection can generate different types of data or how qualitative data can be transformed to be analysed statistically. Practicalities of integrating qualitative and quantitative analyses are illustrated to offer good-practice examples for transformation designs (QUAL → QUAN). With the methodological reflection of research practice I evaluate consequences for the field of mixed methods research, in which the statistical analysis of codes created during an interpretive phase plays a minor role.     

Outcome measurement in homeless systems of care

Public and private interest groups are pressured to demonstrate that investments of resources, time, and effort have resulted in improvements in the lives of those experiencing homelessness and/or reductions in the scope of homelessness. Measuring desired outcomes can provide information to support accountability efforts. This article reviews the literature on outcome measurement in homeless systems of care, grounded in a conceptual model consisting of three elements: continuums of care, service programs, and clients.This review did not reveal the existence of any single comprehensive outcome measurement instrument that could be used for the homeless system of care. System-level outcomes include cost savings, reduction of access barriers, and organizational linkages. Service program-level outcome measurement is typically based on the aggregation of client-level outcomes. At the client-level of measurement, several instruments were identified in the literature that have potential for providing the basis of outcome measurement.     

Strengths, uses and problems of global scales as an evaluation instrument

A global scale can provided means for integrating a variety of client/patient assessment techniques to complement each other in a useful manner. Global scales have been applied to service planning and evaluation as well as being useful for the on-going communication of clients' clinical/functioning status. Global scales are attractive to many service programs because of their apparent ease of implementation and their apparent face validity. Furthermore, there is an extensive body of literature describing global scales as hightly reliable and valid when properly implemented and maintained. The literature also suggests that there are some serious pitfalls. While initial implementation of a global scale is easy, the maintenance of a reliable, valid and useful scale appears to require their active use in treatment planning, treatment review and clinical supervision processes. It also requires that more extensive multidimensional ratings be made at intake, review and termination. Furthermore, staff training and development sessions two or three times a year are needed to surface and deal with differences in clinician ratings. Given these sorts of supports, then, a global scale has been demonstrated as a useful tool in service program management as well as clinical process and outcome studies, including studies of cost effectiveness. Since the major use of a global scale is as an integrating construct, a quantitative model is offered to describe the relationship between global scale ratings and multidimensional facets influencing the global ratings.     

Global scales : Strengths, uses and problems of global scales as an evaluation instrument

A global scale can provided means for integrating a variety of client/patient assessment techniques to complement each other in a useful manner. Global scales have been applied to service planning and evaluation as well as being useful for the on-going communication of clients' clinical/functioning status. Global scales are attractive to many service programs because of their apparent ease of implementation and their apparent face validity. Furthermore, there is an extensive body of literature describing global scales as hightly reliable and valid when properly implemented and maintained. The literature also suggests that there are some serious pitfalls. While initial implementation of a global scale is easy, the maintenance of a reliable, valid and useful scale appears to require their active use in treatment planning, treatment review and clinical supervision processes. It also requires that more extensive multidimensional ratings be made at intake, review and termination. Furthermore, staff training and development sessions two or three times a year are needed to surface and deal with differences in clinician ratings. Given these sorts of supports, then, a global scale has been demonstrated as a useful tool in service program management as well as clinical process and outcome studies, including studies of cost effectiveness. Since the major use of a global scale is as an integrating construct, a quantitative model is offered to describe the relationship between global scale ratings and multidimensional facets influencing the global ratings.     

Implementation of evidence-based interventions: analyzing critical components for sustainability in community mental health services

This study analyses the implementation and sustainability of evidence-based community mental health services in the form of publicly financed Individual Placement and Support programs. Critical implementation components and program fidelity were assessed after one year. After two years, program fidelity was assessed once again. After three years, the programs’ sustainability was assessed and semi-structured interviews performed, in order to deepen the understanding of implementation. Interviews and documents provided the quantitative and qualitative data, which were analyzed by the use of the Supported Employment Fidelity Scale, the Sustainable Implementation Scale (which was developed in a connecting study), and qualitative content analysis. Despite promising fidelity results after one year, eight out of 14 programs were terminated within three years. Implementation of integrated evidence-based programs in community-based settings is a delicate undertaking. Implementing agencies can benefit from rigorous preparation before program start, especially concerning the circumstances at the organizational level, such as making plans for collaboration, financing and assessments of program fidelity.     

Bridging the qualitative–quantitative divide: Experiences from conducting a mixed methods evaluation in the RUCAS programme

Quantitative and qualitative approaches to planning and evaluation in education for sustainable development have often been treated by practitioners from a single research paradigm. This paper discusses the utility of mixed method evaluation designs which integrate qualitative and quantitative data through a sequential transformative process. Sequential mixed method data collection strategies involve collecting data in an iterative process whereby data collected in one phase contribute to data collected in the next. This is done through examples from a programme addressing the ‘Reorientation of University Curricula to Address Sustainability (RUCAS): A European Commission Tempus-funded Programme’. It is argued that the two approaches are complementary and that there are significant gains from combining both. Using methods from both research paradigms does not, however, mean that the inherent differences among epistemologies and methodologies should be neglected. Based on this experience, it is recommended that using a sequential transformative mixed method evaluation can produce more robust results than could be accomplished using a single approach in programme planning and evaluation focussed on education for sustainable development.     

Insights into proceeds from social investments: beyond numbers fetishism and relativism

How profitable are social investments? It is necessary for social professionals to gain and give insight into the proceeds of their work. Both clients and professionals will benefit from a results-oriented approach. Subsidising agencies and financiers will reap the fruits of such an approach as well. But how can we gather evidence on the proceeds of social investments? In this article we distinguish two mutually complementary ways of gaining evidence. The first is known as evidence-based policy and practice, mainly quantitative research on the effects of social interventions and on the causal relations between interventions and outcome. However, limitations are attached to quantitative research because of disruptive factors, a variety of side effects and the dynamics of social reality. Complementary evidence is therefore needed: practice-based evidence, based primarily on qualitative methods that provide insight into mechanisms and conditions under which effects occur. The practice-based method presumes systematically studying a series of small-scale and mindful practices in order to uncover tacit knowledge of professionals. It is exactly this combination of rigid quantitative research and flexible qualitative methods which does justice to the challenging undertaking of clarifying the proceeds from social investments.     

A Peek Inside the Box: How Information Flows Through Substance Abuse Treatment Agencies

This article describes how integrated health information technology affects the collection and flow of client-level intake and assessment data within substance abuse treatment agencies. We combined process mapping and qualitative inquiry methods to analyze staff interviews (N = 54) from eight agencies in four U.S. states. Integrated health information technology was related to expedited and improved flow of information, while nonintegrated health information technology systems were associated with double data entry, multiple eligibility screenings, and different intake processes across levels of care. Process mapping is an effective tool to identify barriers to efficient client-level data management and opportunities to leverage health information technology to streamline the intake process of substance abuse treatment agencies, improve staff productivity, and enhance the accuracy and information flow.     

Statusmasse light: Statusskalen bei unzureichenden Berufsangaben

Using prestige and SES measures requires information on the occupation performed. Because gathering and coding this information is time consuming and expensive, appropriate data are often missing. This article presents proxy-scales that may be used in this situation. These scales are derived from original status measures on the basis of commonly available variables. After presenting the derived status scales their validity is examined. It turns out that they capture approximately half of the variance of the original measures. Using a simple status attainment model, the utility of the derived status scales is demonstrated. Finally, we discuss how the scales may be adjusted to other data structures.     

The relationship between implementation fidelity and educational outcomes in a school-based family support program: development of a model for evaluating multidimensional full-service programs

There is increasing recognition of the need for evaluations that identify program processes or mediators and assess degree of program implementation rather than focusing solely on outcome evaluation. This paper describes the application of complementary qualitative and quantitative evaluation procedures to assess the degree of implementation of multi-component family support programs for improving educational outcomes for at risk youth, and to assess the relationship between program implementation and outcomes. The qualitative evaluation involved prolonged engagement to identify common program domains or mediators. Using a method called Innovation Configuration Analysis, levels of implementation of program domains were explicated as well as an overall Implementation Fidelity Index. Strong positive relationships were found between overall program implementation and program-level outcomes achieved by student participants.     

Conceptual problems of the monitoring and evaluation of qualitative objectives of rural development

Monitoring and evaluation are crucial aspects of all rural development projects in terms of ensuring that objectives are being met and assessing the impact of various interventions. Thus, monitoring and evaluation should be built into a project's organizational and implementation structure and constitute a continuous, systematic project activity. In need of further refinement, however, are techniques for monitoring and evaluating the nonmaterial, nonobjective dimensions of development projects such as community participation. Needed is an approach that is not based exclusively on the measurement of material impact, but is also able to explain what happens in a rural development project that seeks to promote participation. In dealing with participation, there is a concern not only with quantitative results, but also qualitative processes. Toward this end, project beneficiaries should have a role in describing the processes involved, analyzing the results, and making a judgment as to the outcome of project activities. The unmet need in this area is to devise indicators of economic, political, cultural, and vulnerable-group participation in rural development projects and to identify ways such indicators could be used to collect relevant data.     

Respite and family support services: Responding to the need

Increased focus on the needs of parents and guardians for structured opportunities for temporary relief from care of disabled persons has stimulated policy and implementation initiatives at the state government level on behalf of these families. This article presents qualitative and quantitative data on the need for respite services not only in terms of relief, but as a positive, supportive force in the prevention of permanent placement outside the home. Current services are outlined and issues important to clinicians and managers working with such families are discussed.Planning and Development     

Application Review of Journey Mapping

Journey Mapping is a unique Internet-driven program evaluation tool supporting both qualitative and quantitative data collection. Successful implementation of the tool requires minimal training. The Journey Mapping website is updated on a regular basis to include new features; all users log on and work with the latest version. Journey Mapping supports collection of program outcome data, as well as data for program refinements. Realizing the full potential and benefits of Journey Mapping may depend on integrating the system with staff job designs and the overall mission of the program.     

Language attitudes as predictors of morphosyntactic variation: Evidence from Catalan speakers in southern France

This article explores the potential role of attitudes as motivating factors in language variation and change. This study represents an important departure from existing scholarship through its consideration of quantitative measures of language attitudes as factors in multivariate analyses of linguistic data. Four morphosyntactic variables in Northern Catalan (an obsolescent variety spoken in France) will be examined in order to determine which factors significantly correlate with use of local or supralocal variants. Linear regression modelling reveals firm correlations for two of the variables, wherein positive evaluations of Catalan varieties on the status dimension (cf. Carranza & Ryan, 1975; Woolard, 1989, inter alia) align with supralocal variant use. We will see that attitudes can be helpful in uncovering motivations for language variation, but that qualitative ideological data are indispensable in the interpretation of findings.     

Looking at engagement and outcome from the perspectives of child protection workers and parents

This paper reports a study that sought to understand what facilitates engagement between parents and child protection workers and to ascertain the relationship between such engagement and intervention outcome. Quantitative and qualitative data were gathered through personal interviews with 131 worker-parent dyads (workers and corresponding parents receiving service) from 11 child protection agencies in Ontario, Canada. Measures included scales for engagement, parental well-being (depression and stress), and worker well-being (burnout, job satisfaction, stress). Outcome measures included perception of child safety, changed parenting practices and satisfaction. A relationship was found between workers' perception of parent engagement and parents' perception of their own engagement, as well as between the perceptions that workers and parents had around their own respective engagement. Workers who were satisfied with service outcomes were significantly more engaged than those who were unsatisfied. Parents thinking that their children were safer as a result of intervention were significantly more engaged than parents who thought that their children were less safe. The strongest reason given by parents for positive change was being able to trust their worker (p < .001) and believing that their worker was knowledgeable about parenting (p < .01). Qualitative data suggested that parents valued experienced workers because they thought such workers could better understand their problems and how to deal with them. Correspondingly, workers felt that experience enabled them to better understand clients' problems and provide more effective support. No relationship was found between parent engagement and parental depression or stress, but the mean depression score for parents placed them at risk of clinical depression. There was a negative correlation between worker stress and engagement. The study demonstrates that engagement between clients and workers is related to positive outcomes (as reported by workers and parents) and supports the contention that promoting engagement is integral to a successful child protection intervention.     

Medical sociology: some tensions among theory, method, and substance

Why is it that quantitative and qualitative researchers on health issues often have divergent findings and conclusions? Exploration of such differences can be a useful way of bringing separate intellectual enclaves in medical sociology together and also can stimulate future inquiries. Some differences can be resolved by more precise definitions, by comparable frequency and timing of measurement, and by careful evaluation of meaning contexts. The triangulation of methods, using diaries as a bridge between surveys and qualitative measurement, offers particular promise. Improved theory on the relationships of method to data, and more attention to behavioral sequences and the social context of measurement, can serve as a stimulant to innovative solutions.     

Use of Mixed Methods in a Study of Residence and Contact Disputes between Divorced and Separated Parents

This article discusses a study of residence and contact disputes in court which utilised a mixture of quantitative and qualitative methods in order to gain a multidimensional picture of these disputes. It is argued that in a study such as this, the different datasets are best viewed as complementary, rather than as a means of validating each other. The data from each stage address a different aspect of these court disputes, thus offering a better understanding of the complexity of what goes on at court. I examine the ways in which the mixed methods approach improved the research team’s ability to interpret the findings. I conclude that the study has highlighted that because there is not just one view of how well the courts work, a variety of criteria needs to be used in evaluating this: when, for whom, for which problems and on which measures.     

Training consumers to collect data in mental health service system evaluation research

Consumer satisfaction with services is an outcome measure. This report describes The Consumer Satisfaction Research Associate Training Program, which was designed for the education and supervision of recovered consumers who collected data during four studies at three psychiatric hospitals. The educational program had didactic and laboratory components. Consumer research associates collected quantitative and qualitative data using a survey tool for face-to-face interviews. Participant recruitment and management of confidential records were included in the role. Data collected by research associates from consumers who are in treatment can enhance the validity and trustworthiness of satisfaction surveys. Nurses can play an important role in the research process by serving as staff resources for consumers who collect data in various service system settings.