Evaluation approaches in mental health: Implications of the new federalism

Since the introduction of federal mental health legislation in 1963, there has been a changing emphasis on evaluation and accountability. With direct federal funding of community mental health services, accountability demands were met through expectations for local agency evaluation activities which were overseen by federal authorities. The advent of the New Federalism and the shift to block grant funding of mental health services to state mental health authorities have shifted responsibility for evaluation to the states and local programs. This paper reviews federal mental health statutes to trace the extent and locus of required evaluation activities and discusses two approaches to carrying out program evaluation: "top-down" where the evaluation topic, method, and data collection are mandated by an administering or funding body; and "bottom-up" approaches where the subject, method of study, and data to be collected are developed in response to a felt need at the local agency level. A case study of each approach as used at the state level in mental health is examined. Based on the literature and the case studies, conclusions are presented on the pro's and con's of each method in meeting accountability demands and the barriers which must be overcome for either method to be successful.     

A case study in defining developmental costs for quality assurance in mental health center programs

Issues related to the state-of-the-art of ambulatory mental health care review as well as to the resources necessary for planning and operating a quality assurance program may be hampering development of such programs in community mental health centers nationwide. Faced with the problem of responsible and prudent program management, it has become increasingly difficult for CMHC directors to implement innovative, but unproven, and often unpriced programs which have the potential to draw scarce professional resources away from direct service delivery. Pressure to implement CMHC program requirements that are not ordinarily considered revenue-producing in nature, for example, consultation and education services and quality assurance activities, may further create anxiety for mental health program directors, especially during periods when they are ostensibly confronted with the need to maximize CMHC revenues. Before mental health professionals and administrators of mental health programs can be expected to introduce quality assurance programs, they need to have some idea how much the system costs. The figures reported from three North Carolina mental health centers may offer some indication of what other mental health centers might expect to spend in developing a quality assurance program. Salary expenses for multidisciplinary staff participation in quality assurance activities are presented.     

Nursing in tele-mental health

This article discusses a cooperative project involving an academic health sciences center, a state university, a women's shelter, and a community mental health center in East Texas, a medically underserved, rural region of Texas. The U.S. Department of Commerce provided grant funding to develop a telehealth network to provide an evaluation, referral, and treatment program for victims of domestic violence. Nurses were involved in all phases of the project, from initial conception through development, implementation, and evaluation. The authors concluded that all of the women involved in the study had significant mental health issues and other health concerns that were not being addressed by the existing health care delivery system. Without the use of telehealth, these women would have had limited access to primary health care and virtually no access to mental health services. The East Texas Tele-Mental Health Network demonstrates the value of the clinical and organizational skills that nurses bring to innovative models of mental health service delivery.     

A medicare adult day care model: proposed criteria and available supply

The study on which this article is based addressed the issue of the proportion of adult day care centers (ADCs) existing in 1986 that would qualify for Medicare funding under the 1989 U.S. Senate Medicare Adult Day Care Amendments. It also estimated the impact of the criteria on two policy-relevant subgroups of ADCs-that is, Alzheimer's vs. non-Alzheimer's and rural vs. urban-using data from a 1986 national census survey of ADCs. The five proposed Medicare criteria and the percentage of ADCs meeting them were: services to be provided directly, 14.6%; multi-disciplinary team, 20%; services to be provided directly or indirectly, 16%; program activities, 42%; and other, 53%. Only 3% met all five criteria while 13% met four out of five. Alzheimer's centers met the criteria more often than non-Alzheimer's centers, while urban centers qualified more often than rural centers. Based on the findings, implications for Medicare funding policy are discussed.     

A Medicare Adult Day Care Model

The study on which this article is based addressed the issue of the proportion of adult day care centers (ADCs) existing in 1986 that would qualify for Medicare funding under the 1989 U.S. Senate Medicare Adult Day Care Amendments. It also estimated the impact of the criteria on two policy-relevant subgroups of ADCs-that is, Alzheimer's vs. non-Alzheimer's and rural vs. urban-using data from a 1986 national census survey of ADCs. The five proposed Medicare criteria and the percentage of ADCs meeting hem were: services to be provided directly, 14.6%; multidisciplinary team, 20%; services to be provided directly or indirectly, 16%; program activities, 42%; and other, 53%. Only 3% met all five criteria while 13% met four out of five. Alzheimer's centers met the criteria more often than non-Alzheimer's centers, while urban centers qualified more often than rural centers. Based on the findings. implications for Medicare funding policy are discussed.     

Family Reunification of Youth in Foster Care with Complex Mental Health Needs: Barriers and Recommendations

This study presents findings of a formative evaluation of an innovative pilot program designed to reunify and reintegrate foster youth with complex mental and behavioral needs in residential treatment centers or therapeutic foster care with their families in the community. Data collection methods included in-depth structured case file reviews and semi-structured interviews with the youth, as well as their caregivers, Child Protective Services caseworkers, and pilot program staff. The participants provided important insights regarding system, program, and case-level barriers to the successful reunification of these youth back into the community. Training, practice, and policy recommendations are discussed.     

Building Community Capacity to Help Persons with Mental Illness

Abstract

In a time of crisis, persons with mental illness who encounter the police are often sent to jail or are involuntarily committed for psychiatric evaluation when mental health services are not readily available. To better serve these persons and the community, law enforcement and mental health professionals in one Iowa county joined together to ensure these individuals received the assessment and treatment needed. The results of a mixed methods program evaluation are detailed and suggest that the process to develop the program was as critical as the conduct of the program itself. Implications for program development are discussed.     

Resource Scarcity in Mental Health: Crisis or Opportunity for Social Work Education?

This paper suggests that the funding crisis faced by mental health provides an opportunity for structural and policy changes that can benefit the entire system. The effect of increasing state control, trends in mental health treatment, new technologies, and changing higher education constituencies on social work education are examined. Stress is placed on the need for social worker educators to become more involved in the determination of state priorities and policies in order to shift the emphasis of mental health from institution to community, from medical model to psychosocial treatment, and from physician dominance to multidisciplinary control.     

Meeting the Challenge of Preparing Social Workers for Integrated Health Practice: Evidence From Two MSW Cohorts

ABSTRACT

Changes in health care policy have led to an expansion of integrated care models that rely on collaboration among interprofessional health teams. Recent federal funding has encouraged the development of innovative training models to prepare social workers for integrated health practice. This article presents evidence from the first two MSW cohorts of the Integrated Mental and Behavioral Health Training Program funded by the Health Resources and Services Administration. Twelve students completing the program in 2014 (n = 5) and 2015 (n = 7) participated in evaluation activities at the time of program completion and at 1-year postgraduation. Findings highlight key components of the training program instrumental in student attainment of program goals and competencies. Implications for MSW education and training for practice in integrated care environments are discussed.     

Developing a public health policy-research nexus: An evaluation of Nurse Practitioner models in aged care

A frustration often expressed by researchers and policy-makers in public health is an apparent mismatch between respective priorities and expectations for research. Academics bemoan an oversimplification of their work, a reticence for independent critique and the constant pressure to pursue evaluation funding. Meanwhile, policy-makers look for research reports written in plain language with clear application, which are attuned to current policy settings and produced quickly. In a context where there are calls in western nations for evidence based policy with stronger links to academic research, such a mismatch can present significant challenges to policy program evaluation. The purpose of this paper is to present one attempt to overcome these challenges. Specifically, the paper describes the development of a conceptual framework for a large-scale, multifaceted evaluation of an Australian Government health initiative to expand Nurse Practitioner models of practice in aged care service delivery. In doing so, the paper provides a brief review of key points for the facilitation of a strong research-policy nexus in public health evaluations, as well as describes how this particular evaluation embodies these key points. As such, the paper presents an evaluation approach which may be adopted and adapted by others undertaking public health policy program evaluations.     

On considering the unintended impact of evaluation: reactive distortions in program goals and activities

The relationship between program planning and evaluation can be viewed as bidirectional; that is, evaluation methods, procedures, instruments, and criteria not only are determined by, but also influence, program goals and activities. Within the human services context, several factors or sources of reactivity between evaluation and program planning can be identified. These involve (a) quantification of goals and activities, (b) preferences by different audiences for various kinds of evaluation data, (c) values and evaluation criteria, and (d) evaluation requirements and resource availability. Effects of these reactive features are discussed and illustrated with examples drawn from mental health evaluation and accountability practices. It is argued that for evaluation to be a credible and useful practice, evaluators should plan their efforts and assess their own effectiveness within the larger context of human service systems.     

The Connections of Mental Health Problems,Violent Life Experiences,and the Social Milieu of the “Stroll” with the HIV Risk Behaviors of Female Street Sex Workers

Abstract

This paper examines the connections of mental health, victimization, and sexual risk behaviors among a sample of 278 street-based female sex workers in Miami. Using targeted sampling strategies, drug-using sex workers were recruited into an HIV prevention research program. Data were collected by trained interviewers, and focused on drug use and sexual risk for HIV, childhood abuse, recent victimization, and mental health. More than half of the participants reported histories of physical (51.1%) or sexual (53.1%) abuse as children, 37.4% were classified with moderate or severe anxiety symptoms, and 52.9% had symptoms of moderate or severe depression. Logistic regression analyses demonstrated significant associations between mental health issues and engagement in recent unprotected vaginal and oral sex. The program development and policy implications of these findings are discussed.     

Barriers and facilitators to evaluation of health policies and programs: Policymaker and researcher perspectives

Our research sought to identify the barriers and facilitators experienced by policymakers and evaluation researchers in the critical early stages of establishing an evaluation of a policy or program. We sought to determine the immediate barriers experienced at the point of initiating or commissioning evaluations and how these relate to broader system factors previously identified in the literature.We undertook 17 semi-structured interviews with a purposive sample of senior policymakers (n = 9) and senior evaluation researchers (n = 8) in Australia.Six themes were consistently raised by participants: political influence, funding, timeframes, a ‘culture of evaluation’, caution over anticipated results, and skills of policy agency staff. Participants also reflected on the dynamics of policy-researcher relationships including different motivations, physical and conceptual separation of the policy and researcher worlds, intellectual property concerns, and trust.We found that political and system factors act as macro level barriers to good evaluation practice that are manifested as time and funding constraints and contribute to organisational cultures that can come to fear evaluation. These factors then fed into meso and micro level factors. The dynamics of policy-researcher relationship provide a further challenge to evaluating government policies and programs.     

Working across borders to promote positive mental health and well‐being

World health reports highlight the increasing prevalence of mental health problems and the need to recognize that mental well‐being is a fundamental aspect of any health policy. The scale and cost of mental health problems mean that appropriate policies and strategies must be developed and implemented. In Ireland this is a period of substantial review and reflection on mental health policy and practice, but to date little consideration has been given to the development of a cross‐border strategy. Cooperation and collaboration has been largely dependent on the leadership of a number of individuals supported by short‐term European Union funding. This paper is informed by a study which was concerned with examining the issues associated with promoting mental health across the Irish border. Insights were provided by 38 individuals who were involved in the delivery of cross‐border mental health services in the North West region. The article argues that there is substantial support for working across borders and developing sustainable cross‐border health strategies. In the absence of a statutory all‐Ireland coordinated policy overseeing the development of mental health promotion, the sustainability of projects emerged as a key issue. The results suggest that whilst much innovative and creative work has been undertaken by committed individuals, what is now required is a strategic response from both the British and Irish governments which will ensure the needs of service users in these deprived border regions can be effectively addressed. Also, at a policy level this strategic development would be seen as a reflection of the value placed on promoting mental health and well‐being.     

Medicare Funded Mental Health Social Work: Better Access to What?

ABSTRACT

Better Access to Psychiatrists, Psychologists and General Practitioners through the Medicare Benefits Scheme initiative (Better Access), is an Australian Federal Government scheme aiming to improve access to mental health services. Accredited mental health social workers have been involved in the delivery of services under Better Access for more than a decade. In this time, there have been significant changes in the field of mental health services, with consequent increases in size and costs for Better Access. Better Access now represents public spending of more than one billion dollars per annum, yet there is no ongoing account of its impact. In this paper we consider the policy chronology and evaluation of the program, using current available data to question the impact of Better Access on both the service system and the nature of mental health social work practice. This case highlights the importance of a clear articulation of the scope of mental health social work, and ongoing monitoring of the impact of policy in a policy environment increasingly characterised by individualised funding and service delivery structured according to the principles of New Public Management.

IMPLICATIONS

• Social workers' increasing involvement with Better Access has reshaped the scope of mental health social work practice.

• The impact of Better Access on outcomes for service users and mental health social work is not currently a focus of research attention.

• Better Access is a case study illustrating the potential of the policy context to directly shape social work practice, therefore critical engagement with the impact of Better Access is an essential professional imperative.

     

Breastfeeding interventions in Kansas: A qualitative process evaluation of program goals and objectives

Across the state of Kansas, eighteen public health departments received funding through the 2011 Breastfeeding Grant Initiative to start a breastfeeding intervention. The main objective of this study was to evaluate the progress toward program goals and objectives. This study was a process evaluation. Qualitative data were collected from recipient health departments at two time-points during the program year. Structured, open-ended questions were asked through telephone interviews. This study examined: (1) progress toward program goals and objectives, (2) problems encountered during implementation, and (3) evaluation measures employed to assess program impact. All health departments reported making significant progress toward program goals and objectives and reported successful collaboration with other healthcare providers. The use of breast pumps, educational classes, and professional training of staff were reported as providing the best outcome in the promotion of breastfeeding. The majority of respondents did not measure program impact. From a public health perspective, it is important that infants receive breast milk for the first six months of life. It appears that goals and objectives set a priori guided health departments with the administration of their breastfeeding program. Results may be used to enhance and sustain delivery of breastfeeding support programs in Kansas communities.     

Clinical Social Work Practice with Former System Youth with Mental Health Needs: Perspective of Those in Need

The purpose of this study is to describe the types of programs and mental health services former system youth with mental health histories would be inclined to engage in to manage their mental health difficulties, along with the factors that might hinder them from engaging in these services. A series of closed and open-ended questions on potential programs and services were asked, as part of a larger study. Participants were former system youth; specifically 18?C30?year olds who were diagnosed with a mood disorder and were involved with public mental health and social services (e.g., public welfare, child welfare, juvenile justice) during childhood. Responses to the open-ended questions were categorized and percentages are reported from the yes/no items. Eighty-three percent and 76% reported that they would be enticed to come to a support group and panel discussion on mood disorders, respectively, while only 46% reported that they would attend a family support group. Talking with others who have had similar experiences, material possessions and creative expression were the most common responses regarding what would entice them to get involved. Further, peer disrespect, transportation, and daycare were common barriers to potential participation. As the field continues to build knowledge on system-wide strategies to improve agency-based mental health care for transitioning youth and young adults, new evidence-based approaches may benefit from listening to the specific needs, preferences, and suggestions of these youth themselves.