An Evaluation of the Effects of Very Difficult Goals

In 1968 the first cohesive theory on goal setting proposed that difficult goals produce higher levels of performance than easy goals and that specific goals produce a higher level of performance than “do your best” goals. While over 40 years of research supports this theory, there has been some discrepancy regarding the use of very difficult goals. This study was designed to examine the effects on performance of different levels of performance improvement goals and two different types of feedback, as feedback is often used in conjunction with goal setting. A group design was used with participants receiving one of two goal levels, and two different types of feedback. While no significant effects were found between the two types of feedback, goal level produced significant results in terms of performance and accuracy.     

The Effects of Social Comparison and Objective Feedback on Work Performance Across Different Performance Levels

This study investigated the interaction effects of feedback content type (social comparison feedback vs. objective feedback) and two different performance levels (high vs. low) on work performance. One hundred fifty participants were recruited and asked to perform a simulated work task in a preliminary session. Based on their performance, the upper 40% (high performance) and lower 40% (low performance) groups were selected for the main experiment. Participants in each group were randomly assigned to the two different experimental conditions: objective feedback and social comparison feedback. For the participants in the objective feedback condition, information on the number of correctly completed work tasks was provided. For the participants in the social comparison feedback condition, rank information on their performance was provided. The results indicated that social comparison feedback was more effective than objective feedback for the high performers, but was less effective for the low performers.     

Evaluating the Temporal Location of Feedback: Providing Feedback Following Performance vs. Prior to Performance

In order to make feedback as effective as possible, it is important to understand its function in the three-term contingency (TTC) and the impact of various factors involved in delivering feedback. Timing of feedback is one factor that can affect the impact of feedback on learner’s behavior. An analysis of timing of feeback may help us getting closer to better understanding how feedback functions in the TTC. The purpose of this study was to evaluate the effects of feedback at different temporal locations. Specifically, feedback was provided either immediately (a) after a teaching session or (b) before the following teaching session, on teaching performance of undergraduate psychology students. The results indicated that feedback provided before the teaching session was more effective in improving teaching skills than feedback that was provided after the session. These findings suggest that feedback may function primarily as an antecedent to future performance and not necessarily as a consequence for past performance. However, the behavioral mechanism that explains these results is not yet clear. Future studies should investigate this further by manipulating the content of feedback prior to the teaching performance.     

False Alarms and Missed Events: The Impact and Origins of Perceived Inaccuracy in Tornado Warning Systems

Theory and conventional wisdom suggest that errors undermine the credibility of tornado warning systems and thus decrease the probability that individuals will comply (i.e., engage in protective action) when future warnings are issued. Unfortunately, empirical research on the influence of warning system accuracy on public responses to tornado warnings is incomplete and inconclusive. This study adds to existing research by analyzing two sets of relationships. First, we assess the relationship between perceptions of accuracy, credibility, and warning response. Using data collected via a large regional survey, we find that trust in the National Weather Service (NWS; the agency responsible for issuing tornado warnings) increases the likelihood that an individual will opt for protective action when responding to a hypothetical warning. More importantly, we find that subjective perceptions of warning system accuracy are, as theory suggests, systematically related to trust in the NWS and (by extension) stated responses to future warnings. The second half of the study matches survey data against NWS warning and event archives to investigate a critical follow‐up question—Why do some people perceive that their warning system is accurate, whereas others perceive that their system is error prone? We find that subjective perceptions are—in part—a function of objective experience, knowledge, and demographic characteristics. When considered in tandem, these findings support the proposition that errors influence perceptions about the accuracy of warning systems, which in turn impact the credibility that people assign to information provided by systems and, ultimately, public decisions about how to respond when warnings are issued.     

Approaching Error-Free Customer Satisfaction Through Process Change and Feedback Systems

Employee-based errors result in quality defects that can often impact customer satisfaction. This study examined the effects of a process change and feedback system intervention on error rates of 3 teams of retail furniture distribution warehouse workers. Archival records of error codes were analyzed and aggregated as the measure of quality. The intervention consisted of a process change where teams of 5 employees who had previously been assigned a specific role within the process were cross-trained to know and help with other team members' functions. Additionally, these teams were given performance feedback on an immediate, daily, and weekly basis. Team A reduced mean errors from 7.47 errors per week during baseline to 3.53 errors per week during the intervention phase. Team B experienced a reduction in mean number of weekly errors from a baseline of 11.39 errors per week to 3.82 errors per week during the intervention phase. Team C did not experience significant error rate reduction.     

An Evaluation of the Effectiveness of an Automated Observation and Feedback System on Safe Sitting Postures

This study evaluated the effectiveness of an automated observation and feedback system in improving safe sitting postures. Participants were four office workers. The dependent variables were the percentages of time participants spent in five safe body positions during experimental sessions. We used a multiple-baseline design counterbalanced across participants to test the system's effectiveness in delivering two types of feedback. For two participants, delayed/low-density feedback was introduced after baseline and immediate/high-density feedback was added in the next phase. For the other two participants, the sequence of implementing the feedback types was reversed. Results indicated that both delayed/low-density feedback and immediate/high-density feedback consistently improved participant postures and that immediate/high-density feedback was more effective than delayed/low-density feedback. The benefits of the automated system of postural data collection are discussed.     

The Effects of Two Types of Automated Feedback on the Performance of a Community Mental Health Center Staff

To study the differential effects of two types of performance feedback described as either cueing or evaluative "knowledge of results," two monthly reports embodying one or the other type of feedback were distributed to staff in a community mental health center. Evaluative reports consisted mainly of summary statistics in the form of service delivery indices presented in a manner that allowed each service area to compare its current perform that allowed each service area to compare its current performance with its past performance and with the performancec of all other service areas. Although it was found that the effects varied among and within feedback groups and across the dependent variables, the indices fluctuated primarily as a function of instructional antecedents and evaluative feedback. Aside from the quantitative analyses it was widely observed that the performance feedback system had contributed extensively to orienting the Center toward data-based management.     

Effects of Erroneous Estimation of Activity Durations on Scheduling and Dispatching a Single Project*

This research examines the effects of erroneous estimation of activity durations on two different approaches for executing a resource-constrained project. The two approaches are scheduling and dispatching. This research is conducted in two phases. Phase one examines the performance of scheduling and dispatching rules in a variety of project environments. These project environments are categorized by three different factors, namely, the order strength of the precedence relationship, the level of resource availability and the size of errors in estimating the activity durations. The results show that scheduling generally produces a shorter project completion time than the heuristic dispatching rules. The results also indicate that project environment affects only the relative performance differences of the scheduling and dispatching rules but not their ranking. Phase two examines the effect of rescheduling. The results show that frequent rescheduling improves the project completion time, while the timing of rescheduling has little impact on the improvement in the project completion time. The results also identify the project environments where rescheduling is beneficial.     

When Knowing You Are Doing Well Hinders Performance

Summary

The effect of two types of verbal consequences, rule-following feedback and task performance feedback, on rule-induced insensitivity to programmed schedules of reinforcement were examined. Rule-following feedback could be either accurate or non-contingently positive. The task involved moving a sign through a grid using telegraph keys operating on a multiple DRL 6/FR 18 schedule of reinforcement in the presence of an initially accurate rule. After acquisition, the multiple schedule was changed without notice to a FR 1/FI Yoked schedule. Accurate rule-following feedback plus feedback on task performance produced striking insensitivity to the DRL 6 to FR 1 schedule change, the opposite of what might be expected by a common sense analysis of task performance feedback, even after controlling for contact with the changed contingency. It is argued that findings such as these can only be understood by considering the mutual verbal relations evoked by the combinations of rules and feedback, rather than treating feedback as a simple consequential event or as a verbal consequence whose effects do not depend on the relations sustained with other events.     

Using Task Clarification,Goal Setting,and Feedback to Decrease Table Busing Times in a Franchise Pizza Restaurant

The current study investigated the effects of task-clarification, and manager verbal and graphic feedback on employee busing times at a pizza restaurant. Using an ABC design, task-clarification was provided in a memo, which described the process, priority, and goal time of busing. The busing time decreased slightly, from an average of 315 seconds during baseline to an average of 284 seconds after the task clarification memo. The majority of this change was accounted for by servers decreasing the time it took them to arrive at the table to begin busing. Subsequent to the employee memo, group feedback was administered in the form of verbal feedback and a weekly graph of average server busing times. This feedback was associated with reductions in busing time to an average 152 seconds. Changes in the feedback phase were caused almost exclusively by reductions in the time it took to clean the table once the server was there. A reversal to near-baseline busing times was observed in a follow-up phase. Prebusing and cross-busing were not associated with shorter busing times.     

The Interaction Effects of Frequency and Specificity of Feedback on Work Performance

ABSTRACT

This study investigated the interaction effect between specificity (specific vs. global) and frequency of feedback (frequent vs. infrequent) on the quality of work performance. Eighty participants were recruited and randomly assigned to one of the four groups: specific and frequent feedback, global and frequent feedback, specific and infrequent feedback, and global and infrequent feedback. A 2 × 2 factorial design was adopted. Participants were asked to work on a simulated order-fulfilling task and attended 24 sessions. The dependent variable was the error rate of the completed tasks. The results showed that more frequent feedback was more effective and specific feedback was more effective than global feedback in improving the quality of performance. Furthermore, an interactive effect between feedback frequency and specificity was found. Specific feedback was more effective than global feedback when the feedback was infrequent, but global feedback was comparable to specific feedback when it was frequent.     

Effects of error factors and prior incremental practice on spreadsheet error detection: an experimental study

Previous research has shown that spreadsheet errors are common and are not easily detected. In this paper, an experiment was conducted to examine the rate of detection of both quantitative errors and qualitative errors in two domain-free spreadsheets. A detailed list and explanation of the types of common spreadsheet errors are presented. Results showed that the ability to detect various types of errors appears to be dependent on the type and prominence of errors as well as prior incremental practice with spreadsheet error detection. Implications of the findings are discussed.     

Effectiveness of Job Aids and Post Performance Review on Staff Implementation of Discrete Trial Instruction

Discrete-trial instruction (DTI) is a well-established instructional method for teaching children with autism. Accurate implementation of DTI procedures requires that staff be well trained. This study examined the effectiveness of job aids followed by post performance review of strengths, errors, and accuracy on discrete trial instructional accuracy in three participants who conducted DTI within their current job positions. Additionally, primary training procedures were embedded within a meta-shaping procedure, which involved the gradual and systematic introduction of three levels of behavioral components required for accurate implementation of the DTI sequence with mastery criteria set at 90% accuracy across all three levels. As demonstrated in a changing criterion within a multiple baseline design, staff demonstrated increases in DTI accuracy following the initiation of the job aid condition; however, some degree of performance-based feedback was required to establish high levels of procedural fidelity across the DTI sequence. Performance accuracy on maintenance probes remained at high levels. These findings provide support that job aids followed by performance-based feedback may be an effective and efficient method for shaping high levels of DTI procedural fidelity in staff. Additionally, gradual shaping of the DTI accuracy may scaffold performance, minimize errors, and increase social validity.     

The Relative Effects of Different Frequencies of Feedback on Work Performance

Abstract

This laboratory simulation examined the relative effects of two frequencies of feedback on work performance under hourly pay and incentive pay. The study had four experimental conditions: feedback delivered after every session under hourly pay and under incentive pay, and feedback delivered after every fourth session under hourly pay and under incentive pay. Thirty-five college students were randomly assigned to one of the four conditions. Each participant attended 24 thirty-minute sessions. Participants performed a simulated work task on the computer that consisted of computer-related activities such as dragging, clicking, and typing. The dependent variable was the number of correctly completed units of work. An analysis of covariance was conducted to analyze the data using pretest scores as a covariate. Participants who received feedback every session completed significantly more work units than participants who received feedback every fourth session. In addition, an interaction between feedback frequency and pay systems was found: Feedback delivered every session was more effective than feedback delivered every fourth session under the incentive pay system, but not under the hourly pay system. The results suggest that the relative effects of feedback frequency may depend upon the extent to which feedback is correlated with differential consequences for performance.     

Improving Program Outcome with Process-Based Performance Feedback

Abstract

Two studies, using an across participants multiple baseline design, assessed effects of different amounts of process and outcome performance feedback on instructor and child behavior in an early intervention program. Seven instructors, mostly university students, and one child with autism were observed over a 4 month period on measures of instructors' verbal instructions, prompts, and consequences, and correct child behavior. In Study 1, performance feedback contained verbal, written, and graphic information on instructor (process) and child (outcome) behavior, whereas in Study 2, only verbal process information was provided. Results show that both feedback procedures were equally effective in increasing correct instructor performance and correct child behavior. Conclusions are (a) that verbal process feedback alone was as effective as an extensive process plus outcome feedback, but easier to use and more efficient, and (b) that measures of performance error proved useful in detecting feedback effects on instructor behaviors that varied across time and program changes.     

An Evaluation of Train-to-Code to Teach Implementation of PECS©

We evaluated the impact of a customized training program developed using an observation and behavioral coding software system (i.e., Train-to-Code) to teach implementation of Phase 3A of the Picture Exchange Communication System to four undergraduate students. The training program coached participants on all relevant steps of the procedure. To accomplish this, participants viewed multiple video exemplars of correct and incorrect implementation of each step in the sequence of behaviors, and coded these behaviors in accordance with a taxonomy developed by the experimenters. The training program provided prompts and feedback in real time based on participants’ current level of performance and required mastery of seven levels of training, each with fewer prompts and feedback, until an expert level of unprompted coding performance was demonstrated. A multiple baseline design across participants was used to evaluate the effectiveness of the training program on levels of procedural integrity during performance test probes conducted with a confederate learner before and after the completion of the training program. Results showed improved performance relative to baseline following training, and maintenance of performance at 2–4 weeks follow-up. A conceptual analysis of these findings will be discussed, along with implications for staff training in applied settings.     

Provider–user differences in perceived usefulness of forecasting formats

This paper aims to examine potential differences in perceived usefulness of various forecasting formats from the perspectives of providers and users of predictions. Experimental procedure consists of asking participants to assume the role of forecast providers and to construct forecasts using different formats, followed by requesting usefulness ratings for these formats (Phase 1). Usefulness of the formats are rated again in hindsight after receiving individualized performance feedback (Phase 2). In the ensuing role switch exercise, given new series and external predictions, participants are required to assign usefulness ratings as forecast users (Phase 3). In the last phase, participants are given performance feedback and asked to rate the usefulness in hindsight as users of predictions (Phase 4). Results reveal that regardless of the forecasting role, 95% prediction intervals are considered to be the most useful format, followed by directional predictions, 50% interval forecasts, and lastly, point forecasts. Finally, for all formats and for both roles, usefulness in hindsight is found to be lower than usefulness prior to performance feedback presentation.     

Fusion of Intelligence Information: A Bayesian Approach

The attack that occurred on September 11, 2001 was, in the end, the result of a failure to detect and prevent the terrorist operations that hit the United States. The U.S. government thus faces at this time the daunting tasks of first, drastically increasing its ability to obtain and interpret different types of signals of impending terrorist attacks with sufficient lead time and accuracy, and second, improving its ability to react effectively. One of the main challenges is the fusion of information, from different sources (U.S. or foreign), and of different types (electronic signals, human intelligence. etc.). Fusion thus involves two very distinct and separate issues: communications, i.e., ensuring that the different U.S. and foreign intelligence agencies communicate all relevant and accurate information in a timely fashion and, perhaps more difficult, merging the content of signals, some "sharp" and some "fuzzy," some dependent and some independent into useful information. The focus of this article is on the latter issue, and on the use of the results. In this article, I present a classic probabilistic Bayesian model sometimes used in engineering risk analysis, which can be helpful in the fusion of information because it allows computation of the posterior probability of an event given its prior probability (before the signal is observed) and the quality of the signal characterized by the probabilities of false positive and false negative. Experience suggests that the nature of these errors has been sometimes misunderstood; therefore, I discuss the validity of several possible definitions.     

Speed and Accuracy of Data Analysts' Behavior Using Methods of Equal Interval Graphic Data Charts,Standard Celeration Charts,and Statistical Process Control Charts

Abstract

In response to a debate concerning effectiveness of alternative data handling methods in the hands of experts, the exploratory research reported here was conducted. Experts worked individually except for the SPC expert who was assisted by another expert. Two equal interval graphing experts, two standard behavior or celeration charting experts, and one team of SPC experts were presented with four sets of data that they graphed one point at a time. Their task was to plot, analyze, and identify the location of an intervention within each data set. A total of N = 8 series were plotted and analyzed by each analyst. Average percent of interventions correctly identified by equal interval, celeration, and SPC chartists were respectively, 63%, 32%, and 25%. Rank order of mean hours spent on the exercises was from fewest to most hours, celeration chartists, equal interval chartists, and charting SPC charting team. Regardless of method used, poorer quality baseline data in data series resulted in poorer performances among chartists. Implications are discussed in terms of future research into charting behavior and practical consequences of establishing or failing to establish “good” baseline control prior to introducing interventions in the OBM tradition.     

Should You Order the Feedback Sandwich? Efficacy of Feedback Sequence and Timing

This study sought to investigate the efficacy of feedback sequence—namely, the feedback sandwich—and timing on performance. Undergraduate participants performed simulated office tasks, each associated with a feedback sequence (positive–corrective–positive, positive–positive–corrective, corrective–positive–positive, and no feedback), presented in a counterbalanced fashion. Half of the participants received individual verbal feedback delivered privately by the researcher immediately after each session, and the remaining participants received the same type of feedback immediately before each session. The aggregate data suggested no feedback was the most efficacious for participants who experienced feedback prior to performance, and the corrective–positive–positive sequence was the most efficacious for participants who received feedback following performance. Differences in feedback timing were not significant except for the no feedback condition. These results document that the feedback sandwich was not the most efficacious sequence, despite claims to the contrary.