Dimensions and design criteria for developing training reaction evaluations

Although there is research on training program evaluation, there is little systematic research on the design, development and use of training reaction evaluations. In order to obtain meaningful information from a program evaluation, evaluation professionals must be rigorous in the design and development of all aspects of an evaluation study, including instrumentation. The purposes of the study were to classify the dimensions of information sought using reaction evaluations and to establish design criteria for developing training reaction instruments. There were three major phases of the study: (1) classification of the dimensions and questionnaire design criteria used in reaction evaluations; (2) validation of the classified dimensions and the questionnaire design criteria by subject matter experts; and (3) assessment of a sample of training reaction instruments currently used in training programs in US corporations. The research findings were: eleven dimensions for reaction evaluation were identified and classified by purpose. Five overall design criteria, each consisting of several sub-criteria, were judged important in the design of reactionnaires. These include: introduction and directions; question format; question construction; questionnaire layout; and data analysis. It was concluded that a well-designed training reaction instrument integrates the proper application of design criteria with appropriate reaction dimensions. Most training reaction instruments used by US corporations consisted of questions representing only a few dimensions. The instruments varied in form and length. Few of them properly utilized the established questionnaire design criteria.