Measuring ROI in a Sales Program: An E-Learning Solution

United Petroleum International

Patrick Whalen


This case addresses measuring the effectiveness and return on investment of an e-learning solution in an international sales environment. This can be especially challenging when management wants the program to pay for itself in the first year. This case demonstrates that, with a proper needs assessment and support from the organization, a well-designed e-learning program can influence business measures significantly. The program contribution to sales and other business measures determined by using one or more methods to isolate the effects of the program. The $500,000 projected price tag of the training was a key factor in management's decision to support an impact study to determine the return on investment.


United Petroleum International (UPI) is an international organization headquartered in the southwestern United States. UPI operates several refineries and engages in the sales and service of petroleum products worldwide. UPI has approximately 17,500 employees. International sales of petroleum products have plummeted during the last three quarters, and the outlook shows this trend will continue.

Increased competition abroad and a diminishing quality of sales relationships with customers/prospects were determined to be the major reasons for the lack of performance. The results from quarterly customer satisfaction surveys revealed specific areas of low performance. The executive vice president (EVP) of international sales asked for an assessment of the performance improvement needs of the UPI International Sales Organization (ISO). International Sales has 117 sales engineers and eight sales managers. They are supported by 50 administrative employees who maintain the customer/prospect database, develop sales quotes for the sales engineers, maintain pricing and inventory lists, and provide HR services.

A senior representative from corporate HR and two of UPI's internal consultants teamed with an external consultant to implement the Performance Assessment and Analysis Process to identify problems, opportunities, and solutions in ISO. The report provided to the EVP identified overall findings, performance gaps, and recommended solutions. At the end of the presentation, the EVP agreed to fund an intense improvement effort, including sales training and restructuring of the ISO incentive pay plan, which was no longer competitive in the changing markets. Funding was also made available for the consultant to design and implement a comprehensive evaluation system to determine business impact and return on investment. The EVP was particularly interested in knowing the ROI for the program. A business objective was established to improve three business measures. Measures to be tracked were identified as sales, monthly closing ratios, and customer satisfaction. Because measurement is an inherent component of the process, the methods and timing were designed and put into place. Baseline data were collected from UPI's performance records.


The HR department worked with the design team to design and implement a more appropriate and competitive incentive plan. This new incentive plan was designed after a review of several models and an analysis of application to UPI's markets. The plan was approved and scheduled for implementation in June.

The second solution, addressing the skill and knowledge needs within the sales force, was more difficult to design and implement. Client workload, time constraints, and the scattered locations of the sales engineers were impediments to implementing traditional instructor-led learning. Electronic learning methods were considered a viable alternative. A plus for this delivery method at ISO was that all sales engineers had online capabilities on their laptop computers. Another plus was that the flexibility of the electronic delivery method allowed it to be available at any time of the day. This flexibility is attractive to participants who are compensated principally through incentive pay and who desire to spend their available time making customer contacts. The decision was made that the 117 sales engineers and eight sales managers would receive an electronically delivered interactive program to improve their skills and effectively achieve the business objectives. During the performance analysis, it was discovered that the corporate HR group had identified sales competencies from a previous project and had already begun developing a curriculum. Much of this in-work product served as an important input for the new initiative and greatly assisted the on-time completion of the project. The design called for a more focused e-learning effort, paying specific attention to the sales relationships engaged by sales engineers and allowing for significant practice of the required skills. The program had to present numerous job scenarios and challenges currently being encountered in the marketplace. The EVP of International Sales assigned the project to the manager of sales training, who subsequently established a project team to provide the coordination, design, and development of this project.

Several modules were developed with the support of corporate professionals, including technical writers, learning technology specialists, graphic designers, information technology specialists, and consultants. The team consisted of five full-time employees and four external consultants. Given the short timeframe for completion (management allowed a few months to design and implement the program), work began immediately to develop focused e-learning programs based on the desired business impact (the business objectives), job performance competencies, and field sales encounters. Several members of the design team were concerned that traditional faceto-face learning methods could not be replaced by an interactive e-learning program. The learning technology specialists addressed these concerns, and field testing established the design as a success in achieving learning goals. The e-learning program that was developed for the sales engineers became known affectionately as the TLC program, the Technology Learning Competency program. After design completion, it was implemented in June and July, shortly after the new incentive plan was implemented.

The Technology Learning Competency (TLC) Program

The TLC program was an interactive, self-paced learning process designed to assess current skill level and needs of the sales engineer. Each module was designed to build on a specific set of UPI sales skills (that is, client partnerships, product pricing and contracting, selling more profitable products, uncovering objections, handling objections, defining product features as unique benefits for the customer, expanding existing contracts, handling dissatisfied customers, building community awareness of UPI, and UPI product awareness/knowledge).

The TLC program was designed to allow the participant to respond to various salesrelationship scenarios and to determine the appropriate decision to move closer to a sale. Each decision made by the engineer activated another scenario, which allowed additional choices or decisions to be made. The program continued on a predetermined path initiated by the engineer until a string of choices confirmed the responses as appropriate or until the decision was redirected. Video of a subject matter expert provided analysis of decision choices and helpful suggestions. This took maximum advantage of learning opportunities presented when a participant worked through the program. The engineer experienced real-world issues and situations, had the help of an expert, and was able to learn from mistakes in a nonthreatening manner. A pretest at the beginning of each module was used to determine the skill areas that needed improvement and to load the appropriate learning modules. All the 117 sales engineers were pretested to establish a baseline. The program then linked participants to recommended modules that addressed their skill gaps. Each engineer was allowed a two-month window to complete the required e-learning, either during or after hours as his schedule allowed. So that they could be more effective coaches, the eight managers completed all modules plus a coaching module.

The TLC program contained a programmed mechanism that captured the results from the various decision paths chosen by the participant. After each learning module, an individual report was generated, which highlighted the learning achievement and the decisions made by the engineer. This report was provided to each participant and his manager for discussion in the follow-up coaching session. This provided additional learning opportunities and a means for recognition and feedback. Sales engineers were asked to schedule the follow-up planning and coaching meeting with their managers to occur within two weeks of their TLC program implementation.

< Prev   CONTENTS   Next >