Stage Seven. Definition of Criteria

To choose the best alternative, we need to use a measure of their quality. Such measures are called the criteria.

In the course of solving the problem, it will be necessary to compare the proposed options to assess the degree of achievement of the goal or deviation from it, as well as to monitor the course of events. This is achieved by highlighting some features of the objects and processes. These features should be associated with the features of the objects or processes of interest to us and should be available for observation and measurement. Then, based on the results of measurements, we will be able to carry out the necessary control.

Such characteristics are called criteria. Each study (including our one) will require criteria. How many, what, and how to choose criteria? This stage is devoted to the answers to these questions.

First is the number of criteria. Obviously, the fewer criteria needed, the easier the comparison would be. Hence, it is desirable to minimize the number of criteria, and it would be good to reduce it to one. Sometimes it works well. For example, UNESCO, in allocating funds to help underdeveloped countries in the humanitarian sphere, decided to provide subsidies to the most lagging behind. But the “lagging behind” is an evaluative term, and criteria are required by which to determine who is the lagging behind. It is remarkable that in a number of areas UNESCO managed to effectively solve this problem with a single criterion. The state of health is generally assessed in terms of child mortality, and the state of education in terms of the percentage of illiterate people in the country. At one university in the United States, a teacher’s supplement for the quality of teaching was made dependent on one criterion — the product of the number of students by hours in his classes. In Sweden, the effectiveness of the traffic police is measured by the number of deaths in road accidents. In all these cases, the single criteria proved to be workable.

Unfortunately, more often than not a single criterion fails to adequately assess the quality of the object in question. For example, the criterion of speed of arrival of firefighters does not characterize the fight against fires as it is not associated with a decrease in the number of fires. Spending per student does not measure the quality of schooling. The number of students per teacher does not fully characterize the quality of training at the university.

Therefore, you have to enter some more criteria, differently describing the object, and complement each other.

Here is an example of how to choose criteria.

A firm of systems analyst was ordered to analyze the problem of improving garbage collection in a big city. At the stage of forming the criteria, the following criteria were first proposed: the cost of garbage collection per apartment, the number of tons of garbage collected per person-hour, and the total amount of garbage collected. These criteria were rejected as being unrelated to the quality of work. Criteria such as the percentage of residential areas without diseases, the number of fires due to garbage burning, the number of complaints of residents about the accumulation of garbage, and the number of people being bitten by rats were considered more successful. This example illustrates the criteria requirement to be as consistent with the goal as possible.

Yet, what are the criteria and how many of them to choose? The answer will become obvious if we understand that the criteria are quantitative models of qualitative goals. In fact, the criteria formed in the future in some sense represent and replace the goals: optimization by criteria should provide maximum approximation to the goal. Of course, the criteria are not identical to the goal, but it is the similarity of the goal to its model. Determining the value of the criterion for an alternative is essentially a measure of its suitability as a means to an end.

Thus, it is clear that it is necessary to choose such criteria and so many of them that together they are an adequate model of the goal. (However, how to implement this recommendation will have to be decided separately in each case. It is not ahvays fully possible. But do not despair: as the ancient saying goes, “you can w'alk a lot in shoes that are a little tight”.) Consequently, we come to multicriteria tasks — not only because there are multipurpose tasks but also because one goal often has to be displayed by several criteria.

When choosing criteria, you can sometimes use the experience of previous work. For example, in the analysis and design of technical systems are usually used criteria such as financial (cost, profit, etc.), inventory (number of products, range, etc.), operational (efficiency, reliability, etc.), survivability (compatibility w'ith existing systems, adaptability to the environment, the speed of obsolescence, safety, etc.), environmental friendliness, ergonomics and a number of others. Another advice is to introduce at least three criteria for each characteristic described by the criteria: one should characterize the qualitative side, the other the quantitative side, and the third the temporal side. Such empirical lists are certainly useful and should be developed.

Criteria and limitations. Let us now pay attention to the fact that the set of criteria formed by us in the formulation of the optimality problem is divided into two subsets. Some criteria are subject to change as the situation approaches the desired state as close as possible. Others are subject to certain conditions, as a rule, fixing their values; these conditions must be met in the course of solving the entire problem. These criteria are called constraints. Recall (see Chapter 1, the notion of optimality) that constraints play no less a role in selection than maximized criteria. The difference between them is that the criteria maximized as if open up opportunities for the suggestion of more and more new alternatives in search of the best of them, and the restriction obviously reduces their number, prohibiting some of the alternatives. Some target criteria can be sacrificed for the sake of others, and the restriction cannot be excluded and must be strictly observed.

In the practice of systems analysis, there are cases when the imposed restrictions are so strong that they make it impossible to achieve the goal. Then the systems analyst must ask the decision-maker whether these restrictions can be relaxed or removed altogether. Recall the story of the bus company from Chapter

1. Another example is the imposition of strict requirements to the probability of false alarm, as presented to the developers of the radar station. This limitation would require unacceptably long periods of radar signal accumulation. Such a strict requirement stemmed from the reluctance to “too often” disturb superiors with false alarms.

Questions and Tasks

• 1. In what respect are the objectives and criteria?
• 2. What determines the set (number and nature) of the necessary criteria?
• 3. Discuss the similarities and differences between criteria and constraints.