Level 3, Application Results

Self-assessments suggested the self-coaching participant managers from production appeared to be applying their newly acquired knowledge and skills more frequently at Day 180, with a mean score of 112.6, than they were at Day 90, with a mean score of 107.6. While work engagement in both business units was on the increase three months after the program, production maintained a higher response from its direct reports than did maintenance. Managers at PolyWrighton have suggested that the gain in the control group, the maintenance unit, probably was the result of supervisory changes made during that period.

Barriers to Application

Engaged leadership by the lead manager in production and closer communication among all managers in that business unit prevented any notable barriers to application.

Enablers to Application

Production unit leadership was a driving force in enabling the transfer of learning by the participating managers to the job. The lead manager participated in the learning program and practiced the five skills alongside her direct reports, the four line managers. In addition, regular meetings of these managers to discuss application of the five skills, and an improved system of tracking unit performance initiated by these managers, supported the continued use and development of the learning.

Level 4, Business Impact Results

The business impact measures included production's controllable rework and waste summarized in Table 7-1. Also measured were production's work engagement levels compared to those of the maintenance unit. Work engagement data were not converted to monetary values but were treated as intangible benefits.

Table 7-1. Production Controllable Rework and Waste

Pre Intervention 8-Month Average3,6

Projected Trend4

Post Intervention 8-Month Average5,6

Percent

Value

Percent

Value

Percent

Value

Percent Change7

Average Monthly Cost8,10

12-Month Cost Projection9,10

Controllable Rework1

2.69%

$94,000

0.0%

$0

1.87%

$65,450

1.87%

+ $65,450

+ $785,400

Controllable Waste2

0.73%

$178,850

1.5%

$367,500

.22%

$53,900

-1.28%

$313,600

$3,763,200

Total Program Benefits (Controllable Waste and Rework Combined)11

$248,150

$2,977,800

Impact Estimate12

Confidence13

Adjusted Estimate14

Total Benefit Adjusted for Impact15

Isolation of Impact on Total Program Benefits

50.0%

85.0%

42.5%

$1,265,565

Notes:

1. $35,000 is the monetary value of 1 percent of controllable rework per total product produced. Monetary values provided by PolyWrighton.

2. $245,000 is the monetary value of 1 percent of controllable waste per total product produced. Monetary values provided by PolyWrighton.

4. The projected trends are based on the preprogram percent waste and rework data collected during the eight-month period preceding the initiative. A linear trend line, generated in Microsoft Excel and extended through the end of the post-program period, returned the projected percentages shown. The dollar values referenced in notes 1 and 2 were then multiplied by the trend percentages to return the projected values.

5. Post-program monthly percentage and cost averages are based on the data collected during the eight-month period following the initiative.

6. Data points, whose causes were not assignable to the production business unit, were treated as outliers and removed from the calculation of the preand post-program eightmonth averages. PolyWrighton management made the final determination of which data points would be treated as outliers. Examples of causes not assignable to production included new product specifications, ongoing technical issues stemming from the plant fire recovery, an unexpected line freeze attributable to severe weather, and worse than usual supplier material shortages.

7. Percent change is the post-program percentage minus the projected trend percentage for both controllable rework and waste.

8. Average monthly cost is the post-program value minus the projected trend value for both controllable rework and waste.

9. Average monthly cost multiplied by 12 months for both controllable rework and waste.

10. Negative values represent PolyWrighton cost avoidances.

11. Total benefit before factoring in participant and management impact estimates corrected for confidence error.

12. Combined manager and participant estimate of the program's impact on improvement.

13. Combined manager and participant confidence of their estimate of the program's impact on improvement.

14. Combined adjusted impact estimate based on combined manager and participant estimate and confidence.

15. (12-month Total Program Benefit expressed as a positive value) x (Adjusted Estimate expressed as a decimal value) = (Total Benefit Adjusted for Impact):

$2,977,800 X 0.425 = $1,265,565

Preprogram monthly percentage and cost averages, based on data collected during the eight-month period preceding the program, projected trends for controllable waste and rework through the end of the eight-month period following the initiative. A linear trend line returned the projected percentages shown in Table 7-1. The projected trend values through the end of the post-program period were 0 percent for rework and 1.5 percent for waste.

All rework and waste data the cause of which was not assignable to the production business unit were treated as outliers and removed from calculation of the preand post-program eight-month averages. PolyWrighton management made the final determination of which data points would be treated as outliers. Examples of causes not assignable to production included new product specifications, ongoing technical issues stemming from the plant fire recovery, an unexpected line freeze attributable to severe weather, and worse-than-usual supplier material shortages. Controllable rework during the preprogram period averaged 2.69 percent. At a cost of $35,000 for every 1 percent, preprogram rework averaged $94,000 per month. The projected trend value through the end of the post-program period was 0 percent for rework. The post-program monthly average for rework, however, was 1.87 percent, for a cost of $65,450.

Controllable waste during the preprogram period averaged 0.73 percent. At a cost of $245,000 for every one percent, preprogram waste averaged $178,850 per month. The projected trend value through the end of the post-program period was 1.5 percent for waste, or $367,500. The post-program monthly average for waste was $53,900, based on 0.22 percent. The difference between the post-program average and the projected trend represented an average monthly cost decrease of $313,600 for product waste.

Combined, the cost of rework and savings on waste totaled an average monthly cost savings of $248,150, before factoring in participant and management impact estimates and correcting for confidence error, for a total projected annual cost savings of $2,977,800.

Isolation

Also shown in Table 7-1, the combined participant and management estimates of the impact of the initiative on business results, corrected for estimate error, were used in conjunction with the trend analyses of controllable waste and rework. In this case, the 12-month total program benefit (expressed as a positive value of $2,977,800), multiplied by the adjusted estimate of the program's impact on program benefits (expressed as a decimal value of 0.425), returned an annual cost avoidance of $1,265,565 before expenses. This is the total benefit, adjusted for impact, before subtracting total program costs.


Data Conversion to Money

PolyWrighton provided standard monetary values for waste and rework. The company estimated the value of 1 percent of rework at $35,000, and 1 percent of controllable waste at $245,000. The average monthly percentages were multiplied by these values to determine monetary value.

Program Costs

The learning program was provided free-of-charge to the company, but the estimated cost for the program was calculated at $253,761. While the firm's actual cost for the initiative was $3,360, this study provided the more conservative return on investment calculation to allow PolyWrighton managers to make a better-informed decision when considering whether to extend the program to the other 13 business units.

Level 5, ROI

The total projected annual benefit adjusted for impact, before subtracting total program costs, was $1,265,565. The $1,265,565 benefit less the program cost of $253,761 was $1,519,512. The net benefit of $1,519,512 divided by the total program cost of

$253,761 was 5.98. Multiplied by 100, the Level 5 ROI calculation was 598 percent.

ROI = $1,265,565 – $253,761/ $253,761 x 100 = 598%

Intangible Benefits

Work engagement was studied throughout the program plus six months afterward. Work engagement is a positive psychological state of mind that researchers have linked to employee satisfaction and superior job performance. Higher levels of work engagement are associated with positive feelings of individual well-being (vigor); a strong sense of commitment to the organization and its mission, goals, and objectives (dedication); and the employees' full concentration and involvement with the work itself (absorption). Work engagement is measured by the frequency an employee experiences the three psychological sub-states of vigor, dedication, and absorption at work. The self-coaching skills taught during the learning program were intended to help the participating production managers create and sustain a more favorable environment for work engagement to positively affect employee motivation and performance. A series of statistical tests examined work engagement in the production and maintenance units using the UWES in a quasi-experimental research design.

An independent sample t-test for differences between the production (test group) and maintenance (control group) means was not statistically significant at Day 1 of the program; the difference between the two groups' work engagement was too small to matter. However, the gap between these two business units widened considerably by the end of the program. By Day 90, mixed-design analysis of variance statistical testing indicated that the difference between production and maintenance work engagement was statistically significant and powerful. Further, Cronbach's alpha testing showed the statistical reliability of the UWES was high. Given the organizational context, including a disruptive plant fire during the final 30 days of the study, the program can perhaps be seen as more preventive in nature. That is, it may be the production managers learned to handle the high stress situation more effectively than their maintenance counterparts.

By Day 270, six months after the program, work engagement in both business units was on the rise. Managers at PolyWrighton indicated the gain in the control group, maintenance, was probably the result of key supervisory changes made after the plant fire that occurred in the last 30 days of the program. While the post-program gap continued to narrow, production maintained high enough levels of work engagement to remain statistically significant at Day 180 and Day 270. In essence, production managers held that increased work engagement led to greater employee satisfaction, improved teamwork and communications, and better decision making.

 
< Prev   CONTENTS   Next >