DEVELOP THE REPORT
There are five types of reports to develop to communicate the results of the ROI studies. These include the detailed report (which is developed for every evaluation project), executive summary, general audience reports, single-page reports, and macrolevel scorecard.
The detailed report is the comprehensive report that details the specifics of the program and the ROI study. This report is developed for every comprehensive evaluation conducted. It becomes the record and provides the opportunity to replicate the study without having to repeat the entire planning process. It is possible to save time, money, effort, and a great deal of frustration by building on an existing study. The detailed report contains six major headings:
O need for the program
O need for the evaluation O evaluation methodology O results
O conclusions and next steps
Need for the Program
Define and clarify the objectives for the program, making sure that the objectives reflect the five levels of evaluation. Objectives should relate to the participants' perspective, describe what participants are intended to learn, reflect how they are intended to apply what they have learned, and reflect the outcomes that the knowledge and skills gained in the program will have on the organization. Objectives also present the target ROI and how that particular target was determined.
Need for the Evaluation
Typically, if the program is intended to influence Level 4 measures, this presents a need. In some cases, it may be that the Level 4 measures were never developed so the intent of the evaluation is to understand the influence the program has had or is having on the organization. The intent of the evaluation may be to understand the extent to which the program successfully achieved the objectives. The need for the evaluation may depend on the request of an executive. Clearly state the reasons in the report. Although this report will be distributed to key audiences, it will also serve as the tool to refer to in future evaluations and to describe what happened during this particular evaluation.
This clear and complete description of the evaluation process builds credibility for the results. First provide an overview of the methodology. Then, describe each element of the process, including all options available at each step, which option(s) were chosen, the reasons for those choices, all actions and activities related to each element of the process, and each step taken. For the data collection section of the report, detail how the data were collected, why those data were collected, from whom the data were collected, why the data were collected from that particular source(s), when the data were collected, and why those data collection procedures were selected. Also, display a completed detailed copy of your data collection plan. After the data collection plan has been described, explain the ROI analysis procedures and why the isolation method was selected. Clearly state the various ways the effects of the program could be isolated and explain why the method was chosen. In essence, answer the question, “Why did you do what you did?” When explaining data conversion, describe how the monetary values for the Level 4 impact measures linked to the program were developed, again explaining the range of possibilities for data conversion. After describing the possible data conversion methods, clearly explain why the techniques selected were chosen. Address the cost issue and provide the cost categories included in the ROI analysis. At this point, do not include the actual cost of the program. If the cost of the program is introduced too early, the audience will focus solely on the cost and their attention will be lost. As with data collection, provide a detailed copy of the ROI analysis plan so that the audience can see a summary of exactly what happened.