An Evaluation of Training Procedures for Animal Shelter Volunteers

This study evaluated the effects of 3 training procedures on the correct implementation of a dog walking and enrichment protocol (DWEP). During the shelter’s typical training, volunteers correctly implemented just over half of all DWEP steps (M = 55.2%). Correct implementation of the DWEP procedure improved when participants completed a video-based self-training package (M = 75.3%) but did not reach the preestablished mastery criterion of 85% fidelity with 0 safety errors. Correct implementation improved during coaching (M = 90.6%), which consisted of modeling and positive and corrective feedback, and was maintained during 1-week and 1-month follow-up probes. Criterion performance was demonstrated by 2 of 3 participants at the conclusion of the study.     

Improving Program Outcome with Process-Based Performance Feedback

Abstract

Two studies, using an across participants multiple baseline design, assessed effects of different amounts of process and outcome performance feedback on instructor and child behavior in an early intervention program. Seven instructors, mostly university students, and one child with autism were observed over a 4 month period on measures of instructors' verbal instructions, prompts, and consequences, and correct child behavior. In Study 1, performance feedback contained verbal, written, and graphic information on instructor (process) and child (outcome) behavior, whereas in Study 2, only verbal process information was provided. Results show that both feedback procedures were equally effective in increasing correct instructor performance and correct child behavior. Conclusions are (a) that verbal process feedback alone was as effective as an extensive process plus outcome feedback, but easier to use and more efficient, and (b) that measures of performance error proved useful in detecting feedback effects on instructor behaviors that varied across time and program changes.     

A Parametric Analysis of Rehearsal Opportunities on Procedural Integrity

Behavioral skills training (BST) involves instruction, modeling, rehearsal, and feedback. Despite its empirical support, the literature provides little to no guidance regarding the best way to implement rehearsal to ensure accurate implementation of behavioral procedures. Thus, the purpose of this study was to conduct a parametric analysis of rehearsal opportunities within a BST package. We evaluated the effects of 1, 3, and 10 rehearsals with feedback on correct implementation of functional analysis (FA). Regardless of the number of rehearsals, 18 undergraduate participants effectively learned to implement FA conditions. The most efficient training involved the use of single rehearsals with feedback until meeting mastery criterion. Participants maintained integrity during a follow-up session and rated the use of rehearsal as highly acceptable.     

The importance of employee participation and perceptions of changes in procedures in a teamworking intervention

The powerful positive results of implementing teamwork are not always achieved. It has been suggested that attempts to implement theories regarding teamwork do not always lead to those theories being put into practice, and as a result positive outcomes are not always found. The participation of employees in the development and implementation of an intervention may help to ensure that changes take place. In this longitudinal study (N = 583) of teamwork implementation in Denmark we examined the links between pre-intervention working conditions and well-being, levels of participation in planning and implementation, employees' reports of changes in procedures, and intervention outcomes. Pre-intervention levels of autonomy and job satisfaction predicted the degree of employee participation in the planning and implementation of the intervention. Pre-intervention well-being and social support were linked directly to the degree to which employees reported changes in existing work practices concerning teamwork. In addition, participation and changes in work procedures were significantly associated with post-intervention autonomy, social support and well-being. The results indicate that employee participation in intervention processes is crucial in what appears to be an important association with perceived changes in procedures and, therefore, in intervention outcomes.     

Ecological Risk Ranking: Development and Evaluation of a Method for Improving Public Participation in Environmental Decision Making

This article reports an extension of the Carnegie Mellon risk-ranking method to incorporate ecological risks and their attributes. On the basis of earlier risk-perception studies, we identified a set of 20 relevant attributes for describing health, safety, and environmental hazards in standardized risk summary sheets. In a series of three ranking sessions, 23 laypeople ranked 10 such hazards in a fictional Midwestern U.S. county using both holistic and multiattribute ranking procedures. Results were consistent with those from previous studies involving only health and safety hazards, providing additional evidence for the validity of the method and the replicability of the resulting rankings. Holistic and multiattribute risk rankings were reasonably consistent both for individuals and for groups. Participants reported that they were satisfied with the procedures and results, and indicated their support for using the method to advise real-world risk-management decisions. Agreement among participants increased over the course of the exercise, perhaps because the materials and deliberations helped participants to correct their misconceptions and clarify their values. Overall, health and safety attributes were judged more important than environmental attributes. However, the overlap between the importance rankings of these two sets of attributes suggests that some information about environmental impacts is important to participants' judgments in comparative risk-assessment tasks.     

Assessing Observer Effects on the Fidelity of Implementation of Functional Analysis Procedures

Instructing and training others to implement functional analysis (FA) procedures can be a cumbersome and time-consuming task. Students and practitioners are required to learn all of the various components to establish conditions and analyze results while also learning to conduct the sessions. The current study assessed the fidelity of individuals implementing FA conditions after observing and rating the fidelity of videos models, using a multiple-baseline design across FA conditions. Video models of each condition were provided throughout; however, participants were only asked to provide fidelity ratings for one video. Results demonstrated the intervention was successful in increasing participant performance above baseline levels for 16 of 17 participants, with 7 participants requiring an additional intervention in which they were asked to observe their own performance. Further research should evaluate the degree to which this procedure may effectively prepare individuals working with non-confederate children and carry over to other responses and settings.     

Effectiveness of Job Aids and Post Performance Review on Staff Implementation of Discrete Trial Instruction

Discrete-trial instruction (DTI) is a well-established instructional method for teaching children with autism. Accurate implementation of DTI procedures requires that staff be well trained. This study examined the effectiveness of job aids followed by post performance review of strengths, errors, and accuracy on discrete trial instructional accuracy in three participants who conducted DTI within their current job positions. Additionally, primary training procedures were embedded within a meta-shaping procedure, which involved the gradual and systematic introduction of three levels of behavioral components required for accurate implementation of the DTI sequence with mastery criteria set at 90% accuracy across all three levels. As demonstrated in a changing criterion within a multiple baseline design, staff demonstrated increases in DTI accuracy following the initiation of the job aid condition; however, some degree of performance-based feedback was required to establish high levels of procedural fidelity across the DTI sequence. Performance accuracy on maintenance probes remained at high levels. These findings provide support that job aids followed by performance-based feedback may be an effective and efficient method for shaping high levels of DTI procedural fidelity in staff. Additionally, gradual shaping of the DTI accuracy may scaffold performance, minimize errors, and increase social validity.     

A Comparison of Two Types of Antecedent Control over Supervisory Behavior

The effects of a staff development program on supervisory task completion were compared with the impact of assigning a target date for task completion. Task completion was not affected by the staff development program, but improved upon implementation of a target date procedure. The implications for developing antecedent stimulus intervention procedures are discussed.     

An Evaluation of Train-to-Code to Teach Implementation of PECS©

We evaluated the impact of a customized training program developed using an observation and behavioral coding software system (i.e., Train-to-Code) to teach implementation of Phase 3A of the Picture Exchange Communication System to four undergraduate students. The training program coached participants on all relevant steps of the procedure. To accomplish this, participants viewed multiple video exemplars of correct and incorrect implementation of each step in the sequence of behaviors, and coded these behaviors in accordance with a taxonomy developed by the experimenters. The training program provided prompts and feedback in real time based on participants’ current level of performance and required mastery of seven levels of training, each with fewer prompts and feedback, until an expert level of unprompted coding performance was demonstrated. A multiple baseline design across participants was used to evaluate the effectiveness of the training program on levels of procedural integrity during performance test probes conducted with a confederate learner before and after the completion of the training program. Results showed improved performance relative to baseline following training, and maintenance of performance at 2–4 weeks follow-up. A conceptual analysis of these findings will be discussed, along with implications for staff training in applied settings.     

A Case Study in the Programming and Maintenance of Institutional Change

While behavior change projects and component analyses have been critical to the development of behavioral technology, behaviorists working in human service institutions have not focused enough attention on the "packaging" and systematic application of a technology for institutional change. In the present study, a package of organizational behavior management procedures was systematically introduced in a human service setting. The goal of intervention was to promote and maintain institutional change sufficient to support the wide scale implementation and maintenance of state-of-the-art behavioral procedures for program management and service delivery. The specific procedures utilized in the study included the following: (1) planning; (2) establishing functional organizational structure; (3) recruiting, orienting, and training management and direct service personnel; (4) contracting for staff performance; (5) scheduling, supervising, and evaluating personnel; (6) obtaining and managing financial resources; (7) evaluating program operation; (8) ensuring legal safety; and (9) disseminating results. Sixteen dependent variables, chosen to illustrate the quantity and quality of institutional change, were measured for the three years prior to, and five years following, intervention. Data from these measures indicated that intervention resulted in a positive change in virtually every aspect of program operation and that change resulting from the study was durable over time. Within the limitations imposed by case study design, these results were interpreted to suggest that institutional change is possible and that the technology required to effect such change is currently available.     

Increasing Following Headway in a Driving Simulator and Transfer to Real World Driving

ABSTRACT

The effects of a package intervention including prompts, goal setting, feedback, education, and behavioral self-monitoring to increase following headway (decrease tailgating) of three young drivers were evaluated in a simulated driving environment. Another objective of the present study was to determine if the effects of the package intervention would maintain in the simulator and transfer to real-world driving by assessing driving behavior recorded using a black box video camera in the participants’ vehicles. During intervention, drivers were prompted to increase following headway and were provided a specific target for following headway. The participants were asked to estimate following headway after each session and when the session ended were given feedback on actual following headway. The introduction of the treatment package in the simulator was associated with an increase in following headway for all participants. During the reversal phase maintenance occurred for all participants. The effects transferred to real-world driving for all participants. Teaching young drivers in a simulator to increase following headway may be one strategy to decrease the risk of crashes.     

Evaluating Preference for Graphic Feedback on Correct Versus Incorrect Performance

The current study evaluated preferences of undergraduate students for graphic feedback on percentage of incorrect performance versus feedback on percentage of correct performance. A total of 108 participants were enrolled in the study and received graphic feedback on performance on 12 online quizzes. One half of participants received graphic feedback on percentage of correct performance on the first four online quizzes and then received feedback on percentage of incorrect performance on the following four quizzes. The order was reversed for the other half of participants. Participants were then asked whether their feedback on the final four quizzes should depict percentage of correct or incorrect performance. Participants who completed the study overwhelmingly preferred feedback on correct performance. The type of feedback students received for the first four quizzes did not affect subsequent quiz performance, and feedback preference was not related to performance on quizzes.     

The Effects of Individual Monetary Incentives With and Without Feedback

Abstract

This study examined the effects of an individual monetary incentive system with and without feedback to determine if feedback would supplement the effects of incentives. Participants were seven college students who performed a computerized task called SYNWORK. SYNWORK presented four sub-tasks concurrently: memory, arithmetic, visual monitoring and auditory monitoring. Participants earned points for correct responses. The dependent variable was the number of points earned. An ABAC design was used with A = individual monetary incentives without feedback, B = individual monetary incentives with feedback, and C = hourly pay with feedback. Sessions were 90 minutes, and there were 5 to 10 sessions per phase. The point scores of six of the seven participants increased when feedback was added to the incentive system but stabilized or continued to increase when feedback was removed. The feedback intervention was staggered in time across participants, and performance increased when feedback was added, hence the data suggest that feedback enhanced the effects of the incentives. One possible reason for the reversal failure is that feedback evoked higher levels of performance that were then maintained by the additional incentives. Because performance did not reverse, however, the results must be viewed cautiously.     

Food Safety in the Domestic Environment: The Effect of Consumer Risk Information on Human Disease Risks

The improvement of food safety in the domestic environment requires a transdisciplinary approach, involving interaction between both the social and natural sciences. This approach is applied in a study on risks associated with Campylobacter on broiler meat. First, some web-based information interventions were designed and tested on participant motivation and intentions to cook more safely. Based on these self-reported measures, the intervention supported by the emotion "disgust" was selected as the most promising information intervention. Its effect on microbial cross-contamination was tested by recruiting a set of participants who prepared a salad with chicken breast fillet carrying a known amount of tracer bacteria. The amount of tracer that could be recovered from the salad revealed the transfer and survival of Campylobacter and was used as a measure of hygiene. This was introduced into an existing risk model on Campylobacter in the Netherlands to assess the effect of the information intervention both at the level of exposure and the level of human disease risk. We showed that the information intervention supported by the emotion "disgust" alone had no measurable effect on the health risk. However, when a behavioral cue was embedded within the instruction for the salad preparation, the risk decreased sharply. It is shown that a transdisciplinary approach, involving research on risk perception, microbiology, and risk assessment, is successful in evaluating the efficacy of an information intervention in terms of human health risks. The approach offers a novel tool for science-based risk management in the area of food safety.     

Facilitation and Dialogue as Methods of Reflective Practice in Professional Education

Students may fear sharing their learning in public because classrooms are not traditional places where we teach skills needed for mutual understanding, such as facilitation and dialogue. This is a pedagogical challenge for planning, policy, and design fields because contemporary teaching requires collaboration among participants. We propose that learning to work collaboratively in the classroom has broader relevance for practice because it mirrors the learning that happens in organizations. To address this opportunity, we trained students in facilitation and dialogue. In this article, we discuss the design, implementation, and outcomes of this intervention.     

Description and Evaluation of an Approach to Implementing Programs in Organizational Settings

A broad-based approach to implementing programs is described and evaluation information is reported from two investigations of its use by program managers in educational organizations. The approach consists of a framework of program implementation factors from which implementation procedures are derived. The procedures then are used by a manager to help facilitate implementation of a program according to plan. In the first investigation, three program managers were trained in the approach to assist them in implementing a behavioral group counseling program in three public high schools. In the second investigation, a program manager was trained to help implement a case management system in an elementary school. Results of both investigations suggest that managers were able to utilize the program implementation procedures, with their respective programs being more completely implemented when the procedures were applied. Reactions aboul the approach obtained from human service program managers, following both investigations, suggest that it may be a worthwhile managerial strategy. Advantages and limitations of the approach are considered and its relationship to organizational behavior management is discussed.     

Consistent Moment Selection Procedures for Generalized Method of Moments Estimation

This paper considers a generalized method of moments (GMM) estimation problem in which one has a vector of moment conditions, some of which are correct and some incorrect. The paper introduces several procedures for consistently selecting the correct moment conditions. The procedures also can consistently determine whether there is a sufficient number of correct moment conditions to identify the unknown parameters of interest. The paper specifies moment selection criteria that are GMM analogues of the widely used BIC and AIC model selection criteria. (The latter is not consistent.) The paper also considers downward and upward testing procedures. All of the moment selection procedures discussed in this paper are based on the minimized values of the GMM criterion function for different vectors of moment conditions. The procedures are applicable in time-series and cross-sectional contexts. Application of the results of the paper to instrumental variables estimation problems yields consistent procedures for selecting instrumental variables.     

Improving the Quality of Hotel Banquet Staff Performance

The banquet staff at a north Texas hotel were responsible for setting up 11 different functions for conferences and meetings. The functions were often set up late and items were often omitted. An analysis suggested that performance problems were Ihe result of weak antecedents, inefficient work procedures, inadequate training, and weak performance contingencies. A treatment package consisting of task checklists, feedback, goal setting, monetary bonuses for quality performance, training and job aids was designed to enhance the accuracy and timeliness of function setups. Performance increased from an average of 68.8% on a quality measure (accuracy plus timeliness of setups) in baseline to 99.7% during the treatment phase. Performance decreased to 82.3% during a second baseline phase in which the intervention was discontinued by hotel management. Performance increased to 99.3% with the reintroduction of the treatment package. Customer satisfaction ratings of banquet setups and service were high during the interventions. Treatment effects are discussed with respect to rule-governed behavior.     

Further Evaluation of the Performance Diagnostic Checklist-Safety (PDC-Safety)

ABSTRACT

We further evaluated the Performance Diagnostic Checklist- Safety (PDC-Safety) by comparing the effectiveness of a PDC-Safety indicated intervention with a PDC-Safety non-indicated intervention for three employees at a clinic serving children with intellectual disabilities. The interventions targeted participants’ hand washing before working with clients. The results of the PDC-Safety suggested that the antecedents and information domain was most problematic. First, a non-indicated intervention, which included access to additional materials (i.e., hand sanitizer), was implemented and found to be ineffective. Next, an indicated intervention, which included prompting, was found to be effective to increase safe performance to acceptable levels for two of the three participants; one participant required feedback to substantially improve and maintain safe performance.     

Behavior Skills Training with Voice-Over Video Modeling

The present study used behavior skills training (BST) to teach three adult participants to implement a video modeling intervention aimed at teaching social skills to adults with autism spectrum disorder (ASD). During baseline trials, participants were given access to written instructions before role-play with two actors (who simulated a quiet conversation) and one confederate (who played the role of an adult with ASD). During treatment, participants were given a video model with voice-over instruction depicting how to implement the video modeling intervention to teach social skills, repeated roleplay trials, and feedback following their performance. All participant scores (percentage of steps correctly implemented in each session) increased from baseline to treatment, and generalization was demonstrated with an actual consumer diagnosed with ASD. Additionally, after training participants to use a video model to teach one social skill, there was generalization to teaching as many as three additional novel social skills. Participants showed maintenance of skills during a treatment study that involved training adult clients with ASD to engage in the social skills.