Menu
Home
Log in / Register
 
Home arrow Education arrow Measuring the Success of Learning Through Technology

RESULTS

The first 25 people signed up within three days of the program's announcement. Level 1 reaction data were collected at the end of the fifth module and are presented in Table 11-2. Reactions were as expected, with the exception of intent to use, which was very high.

Level 2 learning seemed appropriate, and quiz scores were above targets, as shown in Table 11-3. Scores were slightly lower than desired for the quiz for implementation and support. This was not a huge concern, as this information is clearly in the documents and the product brochures.

Level 3 application data seemed to be on track as shown in Table 11-4. Identifying pricing options and explaining implementation and support were off a little, but overall the objectives were met. The time for the first call was below the objective, which was very good. As expected, there were some barriers and enablers to success, as shown in Table 11-5. The barriers were minimal. However, there was a concern that 9 percent of sales associates were not encouraged by their managers to use the program. As expected, the number one enabler was management encouragement.

Table 11-2. Reaction Results

Rate the Following

Rating

Relevant to my work

4.3

Important to my success

4.1

Intent to use

4.7

Recommend to others

4.2

Target: 4.0

Average: 4.3

(Scale: 1 = not at all; 2 = some; 3 = average; 4 = above average; 5 = very much)

Table 11-3. Learning Results

Module

Quiz Categories

Avg. Number of Correct Responses

1

Rationale for the upgrade

4.3

2

Key features of the upgrade

4.2

3

How upgrade will increase client profit

4.3

4

Pricing options

4.1

5

Implementation and support

3.9

Possible score: 25

Average: 20.8

Target: 20


Table 11-4. Application Results

Extent of Use in One Month

Rating

Make the first call (target: 5 days)

3.5 days

Explaining the rationale for upgrade

4.2

Identifying key features of upgrade

4.3

Describing how the upgrade increases client profit

4.1

Identifying pricing options

3.9

Explaining implementation and support

3.8

Target: 4.0

Average: 4.06

(Scale: 1 = not at all; 2 = some use; 3 = moderate amount of use; 4 = significant amount of use; 5 = very significant amount of use)

Table 11-5. Barriers and Enablers

Barriers

% Reporting

Lack of time to be involved

12%

Lack of management encouragement

8%

Took too long to complete

8%

Technology issues

4%

Other

4%

Explaining implementation and support

3.8

Enablers

% Reporting

Management encouragement

60%

Easy to use

52%

Timely

48%

Convenient

40%

Relevant

36%

Other

16%

Isolating the Effects of the Program

An important and perhaps the most challenging issue was to determine the effects of this program from other influences. The best way to do this was to use a classic experimental versus control group, although that became tricky with this program and it was questionable whether or not it would work throughout the program. At first, 25 sales associates were selected based on the first 25 to complete the modules. This was accomplished within the first three days. This group of 25 was matched to a group of other sales associates. The first challenge was finding an appropriate match; the second was to have enough remaining in the control group, since they drop out of the experiment when they complete the program. Some people would wait weeks or months to take the program, or they would choose not to take it at all. And some dropouts are a reality when mobile learning is left entirely up to the individual. Instead of forcing participants to use the program, the team wanted the program to be taken voluntarily. The communication for the program, which emphasized that learning is needed in order to sell this upgrade properly, was presented in hopes that this plea would cause them to sign up.

To select the control group, the factors that should affect sales were considered. The four most important considerations were:

1. current sales level, on an annual basis

2. tenure with the company

3. performance rating in human resource system

4. total selling experience.

Although the fourth item was a little more difficult to determine because it counted selling experience in other companies, a quick review of human resource records revealed the total number of years of selling experience. Given these factors, more than 80 associates matched with the 25 in the trial group. Fifty of those were selected randomly as the control group, recognizing that some of them may drop out of the control group when they started this mobile learning program.

 
Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
 
Subjects
Accounting
Business & Finance
Communication
Computer Science
Economics
Education
Engineering
Environment
Geography
Health
History
Language & Literature
Law
Management
Marketing
Mathematics
Political science
Philosophy
Psychology
Religion
Sociology
Travel