Global Healthcare Company Conducts Award-Winning Impact Evaluation

2019 Brandon Hall Bronze Award, Excellence in Learning Best Use of Learning Measurement

A multi-billion-dollar global healthcare company based in France is transforming scientific innovation into healthcare solutions around the globe. Dedicated to supporting people through their health challenges, the Company employs more than 100,000 people to provide healthcare solutions in 176 countries. When it recognized a need to measure the effectiveness and value of its employee training, the Company contacted Innovative Learning Group (ILG) to help accomplish this effort and adopt an impact evaluation approach.

Situation

The Company wanted to uncover specific qualitative and quantitative data for given learning solutions:

1. Extent to which the training built the skills and knowledge needed by learners to perform their roles
2. Degree to which learning resulted in on-the-job application of learning to perform critical actions
3. Extent to which performance of these critical tasks produced meaningful organizational impact
4. Factors within the training and organizational environment that either supported or impeded learning, application, and meaningful organizational results
5. Ways to improve the training and harmonize it across locations

The first program the Company engaged ILG to evaluate was the Root Cause Analysis curriculum, for these reasons:

• Root cause analysis skills and knowledge, which are used to identify the underlying causes of deviations from standard procedures, are critical to the organization. Deviations can lead to patient health risks and regulatory noncompliance, either of which can put a company in jeopardy.
• The curriculum has a broad, large audience, including all employees whose roles require them to participate in, lead, and/or review root cause analyses as part of investigating deviations.
• The training was conducted differently across U.S. locations.

Solution

ILG began the Root Cause Analysis evaluation by reviewing and analyzing existing background and course materials from the U.S. locations. The ILG team wanted to understand what goals the current training intended to accomplish, apply instructional design principles to determine how likely the training is to meet the goals, and identify how closely the training is aligned to what the Company needs the training to accomplish.

The ILG analyst then had conversations with the Company’s evaluation process owner and the training’s key business stakeholders. The conversations identified the critical actions learners should be able to perform as a result of the training, as well as the organizational results this performance should help achieve.

The ILG analyst next developed and delivered an online survey for employees who had completed the Root Cause Analysis training. The survey’s primary purpose was to gather broad data on the extent to which learners performed the intended critical actions, the level (high, moderate, or low) and nature of impact they achieved by performing the critical actions, and the factors that supported or hindered performance and impact.

Once the final survey results were received, the ILG analyst sorted survey respondents into high, moderate, and low impact categories and selected a random sample of participants for follow-up interviews. These interviews guided participants to tell the impact story, probing further into critical actions, results achieved, and supporting or hindering factors.

Finally, ILG analyzed and combined the data from the materials review, conversations, survey responses, and follow-up interviews to produce findings and recommendations.

Results

While impact evaluation doesn’t measure return on investment as traditionally defined, it does provide data on the individual performance and organizational impact resulting from a learning solution. In the case of the Root Cause Analysis training, the impact evaluation accomplished the following:

• Assembled findings and case studies documenting the value that the Company’s Learning and Development group is providing through its programming.
• Did the heavy lifting required to determine how to redefine Root Cause Analysis training to meet the organization’s current needs.

Once the evaluation process owner and stakeholder team agreed on the evaluation recommendations to implement, the Company was able to proceed immediately into design and development of a new Root Cause Analysis blended solution. Immediate design and development was possible because:

• The behaviors required for effective root cause analysis were identified during the evaluation; these behaviors directly informed the learning objectives for the new solution.
• The key findings in the evaluation report detailed which content and instructional strategies in the existing training to leverage, and identified new content and instructional strategies to include.

As a result of the evaluation, all key recommendations were implemented, including a standardized blended solution consisting of three e-learning courses, an electronic performance-support tool, and a practice-focused workshop.

Go to Innovative Learning Group’s Medical Technology and Life Sciences page to learn more about the solutions we’ve provided to this exciting industry. To learn more about how Innovative Learning Group can create custom learning solutions to help improve your business results, contact us at info@innovativeLG.com.