Analyzing free-text survey responses: An accessible strategy for developing patient-centered programs and program evaluation

Despite widespread availability of yoga in the Veterans Health Administration (VA), it remains unclear how to best evaluate yoga programs. This is particularly problematic for programs aimed at veterans with mental health concerns, as evaluation typically focuses narrowly upon mental health symptom severity, even though program participants may have other health-related priorities. We analyzed responses to free-text questions on 237 surveys completed by veterans with mental health concerns enrolled in a yoga program at six VA clinics in Louisiana to characterize veteran participants’ experiences with yoga. Qualitative analysis resulted in 15 domains reflecting veterans’ individual health-related values and priorities. We use results to illustrate the potential for analysis of free-text responses to reveal valuable insights into patient experiences, demonstrating how these data can inform patient-centered program evaluation. The approach we present is more accessible to those responsible for decision-making about local programs than conventional methods of analyzing qualitive evaluation data.     

Integrative Intensive Retreats for Veteran Couples and Families: A Pilot Study Assessing Change in Relationship Adjustment,Posttraumatic Growth,and Trauma Symptoms

We assessed a brief, systemic retreat‐style intervention that was developed to address concerns about the utilization of services for veterans coping with traumatic stress. A total of 76 dyads (N = 152) were assessed before and after a 4‐day retreat, which included psychoeducation, group and conjoint therapeutic sessions, and recreational relaxation components. Overall, participants reported a reduction in trauma symptoms, but only support persons experienced a significant increase in posttraumatic growth from pretest to posttest. Both veterans and their romantic partners reported an increase in relationship adjustment after the retreat. Opportunities to address the needs of this population by removing barriers to treatment and reducing feelings of isolation, as well as implications for similar treatments are discussed.     

Strengthening implementation success using large-scale consensus decision-making - A new approach to creating medical practice guidelines

A study involving over 2000 stoma care nurses in the development of best practice guidelines for the assessment of peristomal body profiles, patient engagement and patient follow-up was conducted in 2018. The study was designed to develop guidelines for stoma care treatment and product selection and to decrease the evidence to implementation gap. The project built on research evidence from a literature review of 77 articles. The results of the literature review were then used to inform a series of Delphi surveys sent out to stoma care nurses through Association and industry list-serves in 11 languages. The Delphi surveys were followed by a face2face professionally facilitated discussions among nurse-research experts. The project concluded with a facilitated consensus dialogue among 960 stoma care nurses from 25 countries, resulting in an implementation plan to ensure the guidelines become a normal part of routine patient care. The study resulted in a set of medical practice guidelines for stoma care nurses, designed to improve patient outcomes and patient quality of life, that were accepted and adopted into routine medical practice across 25 countries. This article describes the study and how the process used, coined the ‘Modified Delphi Process’ by the process designer, led to faster implementation than is generally experienced in the medical community.     

Enclosed versus open nursing stations in adult acute care psychiatric settings: does the design affect the therapeutic milieu?

Specific efforts by hospital accreditation organizations encourage renovation of nursing stations, so nurses can better see, attend, and care for their patients. The purpose of this study was to examine the effect of nursing station design on the therapeutic milieu in an adult acute care psychiatric unit. A repeated cross-sectional, pretest-posttest design was used. Data were collected from a convenience sample of 81 patients and 25 nursing staff members who completed the Ward Atmosphere Scale. Pretest data were collected when the unit had an enclosed nursing station, and posttest data were collected after renovations to the unit created an open nursing station. No statistically significant differences were found in patient or staff perceptions of the therapeutic milieu. No increase in aggression toward staff was found, given patients' ease of access to the nursing station. More research is needed about the impact of unit design in acute care psychiatric settings.     

Teaching Note—Keeping It Real: Program Evaluation Projects for an Undergraduate Research Class

ABSTRACT

This article describes a teaching innovation that focused on the redesign of an undergraduate social work research class. Students enrolled in the redesigned class had an opportunity to conduct program evaluation projects in community agencies. The projects included (a) pretest and posttest evaluation of reminiscence approach in improving the well-being of older adults in a long-term care facility, (b) evaluation of family support services at a Head Start center, and (c) pretest and posttest evaluation of an awareness program to promote community inclusion of individuals with an intellectual disability. Steps in course development, implementation, and evaluation as well as the positive student learning outcomes are discussed. The article concludes with limitations and potential areas for further improvement.     

Herpes: Coping and Caring—A Total College Health Program

Abstract

This article seeks to assist college health nurses in developing appropriate protocols for patient care. In addition, the authors outline factors influencing protocol content, design, and development and describe steps in creating individualized protocols by practice setting. To provide college health nurses with a basic understanding of protocols, the authors offer definitions and examples of appropriate and inappropriate use, and show how protocols can influence the delivery of patient care. Although protocols may provide college health nurses with directives for managing specific health problems, they require a sophisticated level of judgment and skill in their implementation.     

Do videotapes improve knowledge and attitudes about AIDS?

Three popular videotapes were assessed for their effectiveness in improving knowledge and attitudes about acquired immune deficiency syndrome (AIDS). One hundred forty-four university students completed knowledge and attitude questionnaires and then watched either an AIDS or a control tape. Half the students completed posttest questionnaires immediately after seeing the tape, and half were tested 1 month later. Students who viewed an AIDS tape showed increased knowledge at immediate and 1-month posttesting (p less than .01); the amount of increase declined over the month, however (p = .012). Students who saw two of the AIDS tapes showed an increase in compassion toward people with AIDS when tested immediately after the tape (p less than .01); but 1 month later, only those students who had viewed a third AIDS tape showed an increase in compassion (p less than .01). The AIDS tapes failed to bring about improvements in attitudes toward preventive behavior. The authors concluded that tapes about AIDS can lead to modest changes in knowledge and perhaps in attitudes; more significant changes, however, probably depend on multimedia interventions that include guided discussions as well as audiovisual and other components.     

Vietnam returning to the scene of the trauma

1. All veterans participating in this project were suffering from post-traumatic stress disorder. All had received counseling for their PTSD, including group sessions for 8 months prior to the trip. 2. One goal of this project was to provide new memories of a Vietnam at peace that could replace or stand beside the 20-year-old recollections. 3. One of the most dramatic occurrences was the re-humanizing of former enemies; the Vietnamese were seen as people and no longer enemies. 4. Some of the gains for the participants included improved sleep, gainful employment, less fear, anger and grief, and a resolution of suicidal intentions.     

Gambling decisions: an early intervention program for problem gamblers

An implementation and one-year follow-up of the Gambling Decisions program attempted to answer several important questions. First, is controlled gambling a viable treatment option for some gamblers? Can earlier stage problem gamblers be separated for treatment from those with more severe problems? Finally, would problem gamblers utilize a community health agency for treatment of their excessive gambling? A pretest/posttest design was chosen where the efficacy of the program was assessed using repeated measures ANOVA analysis. Results showed that an average loss of $608 over a 4-week period was reduced to $113 immediately after the 6-week program and to a loss of $73 at 12 months. The average number of hours spent gambling per 4 weeks was significantly reduced from 23.5 at pretest to 6.5 at the 12 month posttest. Significant decreases were also observed in the number of days per week that clients gambled, and clients reported significant reductions in everyday life problems related to gambling after completing the program.     

Usability testing,initial implementation,and formative evaluation of an evidence-based intervention: Lessons from a demonstration project to reduce long-term foster care

The field of child welfare faces an undersupply of evidence-based interventions to address long-term foster care. The Permanency Innovations Initiative is a five-year federal demonstration project intended to generate evidence to reduce long stays in foster care for those youth who encounter the most substantial barriers to permanency. This article describes a systematic and staged approach to implementation and evaluation of a PII project that included usability testing as one of its key activities. Usability testing is an industry-derived practice which analyzes early implementation processes and evaluation procedures before they are finalized. This article describes the iterative selection, testing, and analysis of nine usability metrics that were designed to assess three important constructs of the project's initial implementation and evaluation: intervening early, obtaining consent, and engaging parents. Results showed that seven of nine metrics met a predetermined target. This study demonstrates how findings from usability testing influenced the initial implementation and formative evaluation of an evidence-supported intervention. Implications are discussed for usability testing as a quality improvement cycle that may contribute to better operationalized interventions and more reliable, valid, and replicable evidence.     

Two‐Year Follow‐Up of the Efficacy of the PREP Training on Iranian Newlyweds

The aim of the present study was to replicate the efficacy of the Prevention and Relationship Education Program (PREP) training on marital satisfaction and dyadic adjustment among a group of Iranian newlyweds. One hundred and sixty heterosexual couples (N = 320 participants) were randomly assigned to an experimental or a wait list control group. Data from one hundred and forty‐eight couples were analyzed. Participants completed the ENRICH Marital Satisfaction Scale (The Family Journal, 1, 1993, 196–207) and Dyadic Adjustment Scale (The Journal of Sex Research, 39, 1976, 190–196) at pretest, posttest, 1‐year follow‐up, and 2‐year follow‐up. Results indicated that mean marital satisfaction and dyadic adjustment scores between the experimental and control groups were significantly different at posttest. Therefore, PREP training improved marital satisfaction and dyadic adjustment of couples at posttest. The data suggests that PREP training program can be introduced as an effective relationship education program for Iranian newlyweds.     

Applying RE-AIM to evaluate two community-based programs designed to improve access to eye care for those at high-risk for glaucoma

IntroductionGlaucoma is a leading cause of vision loss and blindness in the U.S. Risk factors include African American race, older age, family history of glaucoma, and diabetes. This paper describes the evaluation of a mobile eye health and a telemedicine program designed to improve access to eye care among people at high-risk for glaucoma.MethodsThe RE-AIM (reach, efficacy, adoption, implementation, and maintenance) evaluation framework was used to harmonize indicators. Both programs provided community-based eye health education and eye services related to glaucoma detection and care. Each program reported data on participants and community partners. An external evaluator conducted site visit interviews with program staff and community partners. Quantitative and qualitative data were integrated and analyzed using the RE-AIM dimensions.DiscussionBy targeting high-risk populations and providing comprehensive eye exams, both programs detected a large proportion of new glaucoma-related cases (17–19%) – a much larger proportion than that found in the general population (<2%). The educational intervention increased glaucoma knowledge; evidence that it led people to seek eye care was inconclusive.ConclusionsEvaluation findings from the mobile eye health program and the telemedicine program may provide useful information for wider implementation in public health clinics and in optometrist clinics located in retail outlets.     

An evaluation of stakeholder capacity in the implementation of millenium village primary school meal project

The paper focused on evaluation of stakeholder capacity to implement the millennium village primary school meal project in Kenya. The study which was anchored on the stakeholder theory as well as the Context Input Process Product (CIPP) model of evaluation used mixed methods research design with ex-post facto and case study as its two research elements. The mixed methods was adopted because when both qualitative and quantitative approaches are used in a single study, the strengths of both produce a research synergy in which the collective benefits are greater than what is obtained from either approach when used alone. A sample size of 186 participants was selected from seven (7) primary schools using stratified random sampling. The participants selected purposively included pupils, teachers, head teachers, parents, Area Education Officer (AEO) and project staff. Data was collected using questionnaires, face to face interview guide, Focus Group Discussion (FGD) and document analysis guide. The study findings indicated that the project was implemented with great success as indicated by stakeholders’ capacity to understand project objectives, their involvement in project planning and implementation and their positive attitude towards the project. This has contribute to poverty reduction in the community through the project and sustainable approaches for project implementation.     

The foster care system simulation: Evaluation of a training innovation

Although foster care is often spoken of as a system with many interactive facets, foster care personnel often carry out their specific roles with little understanding or consideration of the ultimate impact of that system. A simulation of public foster care service delivery designed to sensitize participants to these interactions is described. Evaluation of the simulation's accuracy in portrayal of actual service delivery and a study of the kinds of learning demonstrated by participants at three points (pretest, posttest, and follow-up) are reported. Participants found the simulation an attractive training method, indicated a greater appreciation of system-wide foster care problems, and reported changes in foster care thinking and work, but other indicators of learning were inconsistent.     

Does affective forecasting change motivation for disaster preparedness? Motivation one month after a hypothetical earthquake / ¿Influye la predicción afectiva en la motivación para la preparación ante las catástrofes? La motivación un mes después de un terremoto hipotético

The present study examined how disaster preparedness would change after people predicted their affective reactions in the event of a major earthquake. In Study 1, participants (N = 255) completed an assessment of disaster preparedness. One week later, participants predicted their thoughts and feelings if a major earthquake were to strike, and then responded to the motivation measure. Study 2 (N = 129) examined the status of participants’ motivation after one month. The results showed that greater motivation was only seen immediately after affective forecasting, and the same level of motivation was not maintained after one month. When people imagine a potential earthquake the expected negative affective reactions are overestimated. Disaster preparedness allows people to mitigate their future negative affective reactions. In order to avoid these affective reactions, affective forecasters had greater motivation for disaster preparedness.     

Early assessment of implementing evidence-based brief therapy interventions among secondary service psychiatric therapists

This implementation study was part of the Ostrobothnia Depression Study, in Finland, which covered implementation of motivational interviewing (MI) and behavioral activation (BA) within regional public psychiatric secondary care. It aimed to evaluate the mid-term progress of implementation and related factors. Altogether, 80 therapists had been educated through the implementation program by the point of the mid-term evaluation. Eligible information for evaluation was gathered using two questionnaires (q1, q2) with a one-year interval.A total of 45 of the 80 therapists completed q1, 30 completed q2, and 24 completed both questionnaires. Professional education was the only background factor associated with adopting the interventions (q1: p = 0.059, q2: p = 0.023), with higher education indicating greater activity. On the basis of trends such as changes in overall usefulness score from q1 to q2, the most involved therapists were slightly more likely to adopt MI/BA. Our experience so far suggests that encouraging staff to begin using new interventions during education is very important. The Consolidated Framework for Implementation Research was found to be a useful tool for constructing the evaluation.     

Finding the Love Guru in You: Examining the Effectiveness of a Sexual Communication Training Program for Married Couples

One specific area of communication that has the potential to elicit a wide range of outcomes, such as enhancing a couple's feelings of intimacy, offending one or both partners causing conflict, improving the communication climate within the relationship, or embarrassing the partners involved, is the discussion of sex. The current project theorized how to initiate and maintain sexual coaching conversations, taught participants the components of skillful and unskillful messages in an online training program, and then evaluated the overall effectiveness of said program. Forty couples participated in an online sexual coaching training program and completed several pretest and posttest measures. Results reveal that participation in the training program led to an increase in sexual and relationship satisfaction, a decrease in sexual communication apprehension, and an increase in sexual coaching knowledge and skill. These results, along with the value of interventive research and the implications of this training program, are discussed.     

Conceptual problems of the monitoring and evaluation of qualitative objectives of rural development

Monitoring and evaluation are crucial aspects of all rural development projects in terms of ensuring that objectives are being met and assessing the impact of various interventions. Thus, monitoring and evaluation should be built into a project's organizational and implementation structure and constitute a continuous, systematic project activity. In need of further refinement, however, are techniques for monitoring and evaluating the nonmaterial, nonobjective dimensions of development projects such as community participation. Needed is an approach that is not based exclusively on the measurement of material impact, but is also able to explain what happens in a rural development project that seeks to promote participation. In dealing with participation, there is a concern not only with quantitative results, but also qualitative processes. Toward this end, project beneficiaries should have a role in describing the processes involved, analyzing the results, and making a judgment as to the outcome of project activities. The unmet need in this area is to devise indicators of economic, political, cultural, and vulnerable-group participation in rural development projects and to identify ways such indicators could be used to collect relevant data.     

An education intervention for families of people with schizophrenia in China: development and evaluation

In China, nurses and physicians are the main care providers for people with schizophrenia. This care is provided primarily in institutions because community services are in their infancy, and families carry the burden of care. In the absence of published nursing research in the area, this article reports the rigorous development and evaluation of a culturally sensitive patient/family intervention, which was implemented in a large psychiatric hospital in Beijing, China. A random sample of 15 nurses responded to an open-ended questionnaire to explore what they believed people with schizophrenia and their family members should learn. A convenience sample of 51 family members who lived with a person with schizophrenia were also interviewed to explore what they understood and needed to know about schizophrenia. Common learning needs were integrated with the literature and presented as the Comprehensive Patient/Family Education Guide, which was implemented with an experimental group of 42 family members, while a control group of 45 received the usual hospital treatment. The effect of the intervention was evaluated by interviewing a random sample of 19 of the family members. The results revealed some useful information for the future planning and implementation of such programs, and although focused on a Chinese sample, do offer insights for nurses around the world.