II Evaluation in Action. Case Studies Describing the Evaluation of Learning Through Technology Programs
Measuring ROI in Work Engagement: A Blended Learning Solution
John Kmiec, Sandra Dugas, Cyndi Gaudet, Heather Annulis, Mary Nell McNeese, and Susan Bush
This case study describes the evaluation of a blended learning program designed to enhance the capabilities of immediate managers to positively influence line employees' work engagement. The study used quasi-experimental research design to analyze changes in work engagement for line employees assigned to two of 14 business units at PolyWrighton, a manufacturer of highquality, lightweight plastics. The test group of immediate managers in the production business unit received the learning initiative. The maintenance business unit managers did not receive the initiative. Also evaluated were the production unit's participant reaction, learning, application, and business impact data. Production's adjusted return on investment estimate was 598 percent, or $1,265,565.
PolyWrighton makes high-quality, lightweight plastics used to package a wide variety of food, beverage, and personal care products. Meeting rigorous hygienic, chemical, and environmental safety standards and specifications for these products requires constant monitoring and testing, state of the art technology and an extensively trained, highly skilled workforce. The chemicals used in the manufacturing process are both toxic and flammable. The plant machinery is very complex, massive in size, and hazardous in its own right. The product itself is processed under high heat and pressure. These conditions combine to demand heightened operational and safety awareness by all employees.
Adding to the complexity of the operation are the costs associated with product waste and rework. In a highly competitive market where raw materials are expensive and frequently in short supply, it is imperative that PolyWrighton generate as little waste and rework as possible. Rework is defined as product that fails to meet customer expectations for quality, and therefore must be reprocessed. Product waste is unusable, because it cannot be reprocessed and must be discarded. The costs associated with rework and waste result in an additional $35,000 for every 1 percent of product rework and $245,000 for every 1 percent of waste per total product produced. The larger of the two expenses, product waste, costs PolyWrighton about $600 per minute for every minute waste is generated.
As with many manufacturing processes, most waste and rework can be prevented, although a smaller amount cannot be prevented and may be considered normal. Controllable waste and rework, for instance, are the result of assignable causes. That is, their causes are identifiable, can be eliminated, and prevented from reoccurring. For example, some controllable waste in the production business unit may represent the cost of a single human error in judgment or decision making that occurred during the manufacturing process. The waste in this example can be traced to the specific cause, and the cause can be diagnosed and eliminated by appropriate intervention. The same holds true for an unexpected equipment failure that must be diagnosed and repaired by the maintenance unit. If the breakdown is preventable, it is controllable. Depending on the nature of the problem, the cause of the mechanical breakdown may be assignable to the maintenance unit (such as improperly performed or neglected servicing procedure), the production unit (for example, operating the equipment improperly), or both. On the other hand, common waste and rework are random and their causes are unknown. Common waste and rework are considered normal byproducts of production, as long as they remain within normal limits of the manufacturing process.