Back Tests, Benchmarks and Stress Tests

Back tests serve for checking that the risk measures effectively capture actual observed variations. Stress tests serve for assessing what would happen under stressed conditions. Benchmarking aims at comparing how different models perform the same tasks. Back testing has not much to do either with stress tests, or with benchmarking. Back testing is part of model calibration (or "fitting") whether stress tests are designed to see what happens under stressed conditions. Benchmarking compares outputs of various models to alternate models serving as benchmarks.

Because they all serve for assessing the reliability of models and measure their effects on target variables, they can be addressed in a single chapter. The principles of back tests, benchmarking and stress tests are simple. However, technicalities and processes involved have become more critical. This has been highlighted by the financial crisis of 2007-2008, which demonstrated that reliability and compliance to sound best practices remained a persistent technical and organizational challenge.

A back test compares "before the fact" measures with "after the fact" measures. A stress test, in its simplest form, is a sensitivity analysis with a focus on critical variables. Benchmarking is essentially a comparison of models. They all require technical expertise and judgmental approaches. Stress testing does not simplify to inputting stressed values in risk factors. Its main challenge is in assessing the "second level" indirect effects, as explained in this chapter. Traceability of those effects makes stress testing a difficult issue.

Back testing is dealt with in Section 38.1. Benchmarking is briefly discussed in Section 38.2. Stress testing, sensitivity analyses and scenario-based factor-push techniques are discussed in Section 38.3. Finally, some validation issues are discussed briefly in Section 38.4. Note that in this chapter we address both market risk and credit risk. For credit risk, we do not need other concepts than those already defined. Moving from market risk to credit risk and vice versa can be easily inferred with examples provided.