BENCHMARKING

An alternative technique differs from back testing but serves the same purpose of insuring the reliability of the data. Benchmarking consists of comparing the outputs of some models to the outputs of alternative models that could be applied. For example, several "pricers," or valuation models, of market instruments apply to the same instrument. Alternatively, a single model applies to several instruments. This allows benchmarking the outputs of various models that apply to the same instrument or comparing which models fit better the calibration of a single instrument family.

Benchmarking is feasible only when there are alternative models. Some simpler models than those actually used might be less accurate but can serve at least for checking the orders of magnitude of outputs. Since there are several modeling options, it is generally feasible to design simple models that can serve as a benchmark. For credit risk, there are in-house models and vendors' models of economic capital. This allows benchmarking. For Basel 2 calculations, in-house models serve in "production mode," meaning for recurring calculations bank-wide. But Basel 2 calculations are easy to replicate for single transactions. This makes benchmarking feasible by sampling some transactions and comparing the outputs of the "production" model to the sample calculations. If outputs are similar, there is no issue. If not, the issue is to trace why there are discrepancies. Benchmarking implies traceability of calculations.

STRESS TESTING, HYPOTHETICAL SCENARIOS AND SENSITIVITY ANALYSES

Stress testing aims at investigating the possibility of exceptional losses by stressing the value of the risk factors. It is essentially the combination of a sensitivity analysis and of a "factor-push" technique, by stressing the factors that influence most portfolio values. The technique requires ranking the risk factors according to the portfolio sensitivity to each one of them. Since main risk factors might vary through time, the ranking changes.

Any stress test requires sensitivity analysis and is scenario based. For example, Basel 2 requires stress tests based on discrete scenarios in ALM, for the interest income and for the economic value. The benefit of scenarios is that they are not black boxes, as the VaR is. Selected "factor-pushes" are judgmental, based on either extreme values observed at certain periods and on judgment of how far the deviations could be. With a portfolio, it is not simple to identify which deviations of which parameters result in extreme losses because asset value changes offset within the portfolio.

We take the example of a credit portfolio. Various sensitivity analyses apply to portfolios. To proceed on an orderly basis, we need to refer to a unique base case, and change only one variable at a time and use a "factor-push" technique, or equivalently, a "what-if' technique.