Measuring the Effectiveness of TeamSTEPPS
Implementing and sustaining a team training program such as TeamSTEPPS can result in transformational change in the patient safety culture of a hospital. This change can be measured using the Hospital Survey on Patient Safety Culture. Because TeamSTEPPS is a training program, it can be evaluated using Kirkpatrick's four-level taxonomy of training criteria, which is widely used to evaluate the effectiveness of training programs.
- Level 1: Reaction to training, including satisfaction and assessment of the utility of the training.
- Level 2: Learning,including immediate post-training changes in knowledge, knowledge retention, and demonstration of skill.
- Level 3: Transfer of learning to behavior that is applied in the work environment (transfer is synonymous with adoption of new behaviors).
- Level 4: Results, the degree to which desired outcomes are achieved as a result of transfer.
Success at Levels 1 through 3 increases the likelihood of achieving Level 4—desired results such as transformational change in safety culture. Our research indicates that team training can result in transformational change in safety culture when the work environment supports transfer of learning to behavior.
To determine whether hospital employees have learned the TeamSTEPPS tools and strategies and transferred that learning to their behaviors in the work environment, we developed 10 items for addition to the Hospital Survey on Patient Safety Culture: one item to measure respondents’ extent of TeamSTEPPS training, four items to measure learning (Level 2) as knowledge of TeamSTEPPS tools, and five items to measure transfer (Level 3) as a respondent’s perception of the frequency with which TeamSTEPPS tools and strategies were used. The learning and transfer items are internally consistent (alpha = 0.71 and 0.79 respectively).