Measuring the impact of research in CoEs
Given the large amounts of funding provided to CoEs, the impact of their research is of great importance to the REIs that fund them. Different strategies are used to account for the lasting effects of CoE research. Approximately 84% of responding CoEs stated that they were accountable to an external advisory board that supervises their research activities and performance (Figure 3.23). Less frequent is the use of metrics-based indicators to monitor research performance (51%) or agreements with individual units on research targets to be met within specific time frames (53% for the whole sample of CoEs,16 rising up to 57% for the CoEsSB).
Figure 3.23. CoEs and performance monitoring to measure impact
Note: *This percentage reaches 66% when Japanese CoEs are removed from the main sample (i.e. All CoEs). Source: OECD/RIHR Survey to Centres of Excellence, 2012.
There are differences in the way CoEs in different research fields assess their performance and the impact of their research. Overall, the role played by an external advisory board is important for all CoEs, regardless of their research focus. The use of an external advisory board is more frequent for CoEsLB in technical sciences (85% of the sample) than for those that focus on social sciences and humanities (77% of the sample). CoEsLB in technical sciences are also more prone to use a system of metrics-based indicators to evaluate research performance (Table 3.7). This may be linked to the challenges associated with quantitative measurement of the scholarly output of the social sciences and humanities (OECD, 2010).
Table 3.7. Impact assessment methods and fields of research
|
All CoEs |
CoEsLB |
CoEsSB |
||||
|
Technical sciences |
Social sciences and humanities |
Technical sciences |
Social sciences and humanities |
Technical sciences |
Social sciences and humanities |
|
|
The CoE uses a system of metrics-based indicators to monitor performance |
56% |
34% |
55% |
42% |
74% |
27% |
|
The CoE makes target agreements with its individual units (e.g. researchers or research groups within the CoE) |
53% |
56% |
52% |
65% |
85% |
50% |
|
The CoE has an external advisory board supervising the centre's performance |
85% |
83% |
85% |
77% |
83% |
85% |
Source: OECD/RIHR Survey to Centres of Excellence, 2012.
Respondents were able to describe in the comments section additional performance monitoring tools that were not mentioned in the survey questions. A number of CoEs described mid-term evaluations and how the results were used to determine continued funding. Others mentioned regular evaluations by international experts and periodic peer reviews. For some CoEs, impact assessment is continuous and undertaken by the centre’s steering group, research committee or advisory board. In some CoEs, individual researchers are required to meet key metrics (e.g. attract external research funding, publish in scientific journals), whereas in others, regular seminars are conducted to monitor performance.
