Evaluation Planning and Data Collection

As the learning and development profession has matured, accountability of learning has increased. In the past, we could provide learning solutions, including technology-enabled learning, to clients and measure the success of that solution based upon self-reports. But in a complex world of increased scrutiny, less time, and fewer resources, this is no longer enough. Instead, the very real demands for learning require a shift from an activity-based approach to a results-based approach to learning through technology, as outlined in Table 3-1. In some cases, executives want to see the financial ROI.

Table 3-1. Activity-Based vs. Results-Based Approach



Business need is not linked to the learning.

Program is linked to specific business impact measures, such as revenue, productivity, quality, cost, time, and new customers.

No assessment of performance issues that need to change to meet business needs.

There is an assessment of performance that needs to improve to meet business needs.

Smart objectives are not developed for application.

Smart objectives for change and the related business impact are identified.

Participants are not fully prepared to achieve results from the program.

Results expectations are communicated to, and in partnership with, participants.

Work environment is not prepared to support the transfer of learning to application and business impact.

Work environment is prepared to support transfer of learning to application and business impact.

Partnerships with key managers to support the participants have not been identified and developed.

Partnerships are established with key managers prior to learning to ensure participation and support.

Results or ROI analysis in real, tangible, and objective measures—including monetary impact—are not captured.

Results and ROI analysis are measured.

Planning and reporting is input focused.

Planning and reporting is outcome focused.

For more detail on this shift, see The Value of Learning: How Organizations Capture Value and ROI and Translate Them Into Support, Improvement, Funds (Phillips and Phillips, 2007, Pfeiffer).

For learning programs to lead to results, they must first be positioned for success. Positioning occurs through the establishment of alignment, the starting point for implementation. Alignment drives objectives, the first step in the ROI process, as shown in Figure 2-3 (found in chapter 2). Objectives drive the design of learning programs by describing how participants should react to the program, what they will learn, what they will do with what they learn, and the impact their behavior change will have on key business measures. A variety of data collection methods are available to collect reaction, learning, application, and impact data. This chapter explores alignment, objectives, evaluation planning, and data collection.


Objectives are core to business alignment. As shown in Figure 3-1, they evolve from the needs assessment process and drive the evaluation process. Objectives serve as the catalyst between what stakeholders want and need and what they get from a program or initiative. The first step toward developing objectives is clarifying stakeholder needs. First, business alignment starts as the learning program is connected to the business need. Next, alignment continues with impact objectives. Participants focus on the business impact during implementation. Finally, business alignment is validated as the learning program's contribution to the business impact is calculated (isolating the impact of learning).

< Prev   CONTENTS   Next >