Evaluating Alternatives

The second philosophical statement that defines evaluation presents it as the process of delineating, obtaining, and providing useful information for the purpose of selecting among alternatives. Thus it may not matter whether the program was efficiently conducted, effective, or had an impact on behavior or functions. Instead, the value of the evaluation is in its being able to compare one activity to another, one program to another, or one employee to another so that decisions can be made in the presence of empirically collected data. Search committees perform this kind of evaluation. In the course of their work, they describe job candidates' strengths, outline previous experiences, and acquire other useful information that makes it possible to choose among a number of candidates. A company planning to adopt and purchase a computer system will perform this kind of evaluation on all the systems it is considering. It will select the one that performs the best given the company's needs and resources.

Identifying Areas to Improve

Finally, there is a third way of defining evaluation as the identification of discrepancies between where a program is currently and where it would like to be. For example, an organization's marketing department may have as one of its goals at least one face-to-face visit with customers per year. Currently, its sales force sees fewer than half the customers in a year. Records of face-to-face calls indicate the discrepancy between where XYZ Corporation is currently and where the organization wants to be.

Personnel evaluations often take on this definition as well. A new employee's first evaluation may be an example of the first definition, that is, an evaluation against some minimal standard of performance. After this initial evaluation, certain performance goals are set for the employee, either mutually or by the supervisor or team. The next and all the subsequent evaluations of that employee are compared with those performance goals or standards. The discrepancies are identified and remediation strategies are developed.

Each of the individuals identified here who have some interest in or ownership of the evaluation are stakeholders in the organization being evaluated. As such, they have a specific focus on what is evaluated and a particular expectation of how they might use evaluation results. Project directors and project managers might focus their attention on how efficiently the project activities are performed and the effectiveness of these activities in reaching predetermined goals and objectives. Project staff (teachers, trainers, counselors, human resource workers, line staff, and volunteers) might focus their attention on how effective the project activities have been in achieving the short-term objectives of their clients and the impact of the activities in sustaining long-term changes in the client. Parent institution administrators and management might focus their attention on the impact of project activities on any significant changes in how the organization functions and the fundamental products of the organization.

Consequently, each of the stakeholders has specific responsibilities before, during, and after the evaluation. Project managers and directors need to determine what aspects of the project must be evaluated as stipulated by funding sources, accreditation requirements, or strategic plans. They also need to identify aspects of the project that should be evaluated to assist them in determining what (or who) is working, not working, no longer needed, and in need of updating. Also, it is usually their responsibility to appoint or hire the evaluator based upon that person's (or agency's) experience, capability, and reputation.

Project staff need to assist in helping the project managers and evaluator in identifying those aspects of the project that should be evaluated that would result in answers for them to improve their practice and product. Also, staff need to assist in data collection both during and after the project cycle to assure that data are collected in a consistent, rigorous, and relevant manner. They act as the reality filter for evaluators to communicate which data collection practices and instruments are realistic given the work being performed.

Organization administrators need to establish a credible atmosphere for the evaluation. They must communicate to all parties involved that the evaluation activities are meaningful, important, and necessary. This means that from the outset they have to get people to understand that the evaluation is not a punitive exercise, but one that will provide important insights to improving the operation of the organization. This sense of improvement must be maintained throughout the evaluation by informing those involved, making changes if activities have been identified as in need of change, and encouraging staff to use interim evaluation findings to improve their practice. At the end of the program cycle, they must use the evaluation findings to alter or maintain project operations, instead of just filing the report or forwarding it to the project sponsor.

< Prev   CONTENTS   Next >