Log in / Register
Home arrow Education arrow Measuring the Success of Learning Through Technology

Determining Program Objectives

Program objectives reflect the needs of stakeholders. When implementing technologybased learning, it is important to develop objectives at all five levels of evaluation. These objectives tie the learning to meaningful outcomes (reaction, learning, application, impact, and ROI). Program objectives represent the chain of impact, ensuring that designers, developers, participants, supervisors and managers, senior leaders, and evaluators are aware of the potential for success. Objectives should detail specifics about quality, accuracy, and time.

Level 1 reaction objectives describe expected immediate satisfaction with the program. They describe issues that are important to success, including the relevance of the program and importance of the information or content. In addition, these objectives describe expected satisfaction with the logistics of the learning, from delivery to expected use. Table 3-2 shows some typical reaction objectives.

Level 2 learning objectives describe the expected immediate outcomes in terms of knowledge acquisition, skills attainment, and awareness and insights obtained through the learning experience. These objectives set the stage for preparing participants for job performance transformation. It is important to note that even performance support tools (a nonlearning solution) will still have a learning component and thus learning objectives. Table 3-3 shows some typical learning objectives.

Table 3-2. Typical Reaction Objectives

At the end of the program, participants should rate each of the following statements at least a 4 or 5 on a 5-point scale:

• The program was organized.

• The delivery of the content was appropriate.

• The program was valuable for my work.

• The program was important to my success.

• I will recommend this program to others.

• The program was motivational for me personally.

• The program had practical content.

• The program contained new information.

• The program represented an excellent use of my time.

• I will use the content from this program.

Table 3-3. Typical Learning Objectives

After completing the program, participants will be able to:

• Identify the six features of the new policy in three minutes.

• Demonstrate the use of each software routine in the standard time.

• Use problem-solving skills, given a specific problem statement.

• Determine whether they are eligible for the early retirement program.

• Score 75 or better in 10 minutes on the new-product quiz.

• List all five customer-interaction skills.

• Explain the five categories for the value of diversity in a work group.

• Document suggestions for award consideration.

• Score at least 9 out of 10 on a sexual harassment policy quiz.

• Identify five new technology trends explained at the virtual conference.

• Name the six pillars of the division's new strategy.

• Successfully complete the leadership simulation in 15 minutes.

Level 3 application objectives describe the expected intermediate outcomes in terms of what the participant should do differently as a result of the technology-based learning. Objectives at this level also describe expectations as to the time at which participants should apply knowledge, skills, and insights routinely. Table 3-4 presents some typical application objectives.

Level 4 impact objectives define the specific business measures that should improve as a result of the actions occurring through the learning process. Improvement in these intermediate (and sometimes, long-term) outcomes represent changes in output, quality, costs, and time measures, as well as “softer” measures, such as engagement, satisfaction, and brand. Objectives at this level answer the question, “So what?” as it relates to the investment in learning. They describe to stakeholders the importance of learning through technology. Table 3-5 offers some examples of typical impact objectives.

Table 3-4. Typical Application Objectives

When the project is implemented:

• At least 99.1 percent of software users will be following the correct sequences after three weeks of use.

• Within one year, 10 percent of employees will submit documented suggestions for saving costs.

• The average 360-degree leadership assessment score will improve from 3.4 to 4.1 on a 5-point scale in 90 days.

• 95 percent of high-potential employees will complete individual development plans within two years.

• Employees will routinely use problem-solving skills when faced with a quality problem.

• Sexual harassment activity will cease within three months after the zero-tolerance policy is implemented.

• 80 percent of employees will use one or more of the three cost-containment features of the healthcare plan in the next six months.

• By November, pharmaceutical sales reps will communicate adverse effects of a specific prescription drug to all physicians in their territories.

• Managers will initiate three workout projects within 15 days.

• Sales and customer service representatives use all five interaction skills with at least half the customers within the next month.

Last, the Level 5 ROI objective defines for stakeholders the intended financial outcome. This single indicator sets the expectation for how the benefits of learning will relate to the cost. (Will the improvement in impact generated from the program recoup the costs of its implementation?)

An ROI objective is typically expressed as an acceptable return on investment percentage that compares the annual monetary benefits minus the cost, divided by the actual cost, and multiplied by 100. A 0 percent ROI indicates a break-even program. A 50 percent ROI indicates that the cost of the program is recaptured and an additional 50 percent “earnings” (50 cents for every dollar invested) is achieved.

For some programs, the ROI objective is larger than what might be expected from the ROI of other expenditures—such as the purchase of a new company, a new building, or major equipment. However, the two are related, and the calculation is the same for both. For many organizations, the ROI objective for a learning program is set slightly higher than the ROI expected from other “routine investments” because of the relative newness of applying the ROI concept to these types of programs. For example, if the expected ROI from the purchase of a new company is 20 percent, the ROI from a team leader development program might be in the 25 percent range. The important point is that the ROI objective should be established up front and in coordination with the sponsor.

Table 3-5. Typical Impact Objectives

After project completion, the following conditions should be met:

• Sales for upgrades should reach $10,000 per associates in 60 days.

• After nine months, grievances should be reduced from three per month to no more than two per month.

• The average number of new accounts should increase from 300 to 350 per month in six months.

• Tardiness should decrease by 20 percent within the next calendar year.

• An across-the-board reduction in overtime of 40 percent should be realized for front-ofhouse managers in 60 days.

• Employee complaints should be reduced from an average of three per month to an average of one per month.

• By the end of the year, the average number of produce defects should decrease from 214 per month to 153 per month.

• The employee engagement index should rise by one point during the next calendar year.

• Sales expenses should decrease by 10 percent in the fourth quarter.

• A 10 percent increase in brand awareness should occur among physicians during the next two years.

• Customer returns per month should decline by 15 percent in six months.

Evaluating the Program

This final phase of the alignment process is the basis for this book. Evaluation is based on meeting the objectives of the learning program. The more specific the objective, the easier it is to plan the evaluation. From clear objectives, the evaluator can determine what measures to collect during the evaluation process, the sources of the data, the timing of data collection, and the criterion for success. A critical step in the evaluation phase that validates the alignment of the program to the business need is isolating the effects of the program. As you will explore in the next chapter, this step is an imperative to report credible, reliable, and valid results. A variety of techniques are available to isolate the effects of learning.

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
Business & Finance
Computer Science
Language & Literature
Political science