Recommendations for National Policy-Makers

To address the previously outlined general problem and its causes, many interviewees plea for systematic reflection on key dimensions of education and research. Table 6 gives an overview over the recommendations that we drew from the interviews. In this section, we will elaborate on the recommendations at national level, recognising that these will address the 'external' evaluations.

Objective 1: Simplify the procedures

Recommendation 1.1: Reduce the number of evaluation instruments and reports.

At the moment, several evaluations are being undertaken across universities in Romania, that absorb too much time and money. Therefore, we suggest that policy-makers integrate the existing evaluation instruments (see Tables 1 and 5) into a single, comprehensive evaluation scheme that will satisfy the need for quality assurance, quality improvements, and comparative quality analysis across institutions. A single evaluation system will reduce the amount of administrative and paperwork conducted at the moment by universities, and will make the standards and their assessments more transparent for professionals. Failing to do so—or worse, increasing the number of evaluations—is likely to further increase the bureaucracy that universities deal with on a daily basis.

Table 6 An overview over the recommendation to address the policy problem

Policy objective

1. Simplify the evaluation procedures

2. Allow professors and students to decide on quality standards

3. Apply a more consistent and open concept of 'quality'

Recommendations at national level

1.1 Reduce the number of evaluation instruments and reports

2.1. Focus on organising the evaluations without pre-defining all the standards

3.1. Reduce the number of standards on which evaluations are to be carried out

1.2. Evaluate the evaluation procedures as a whole every

5 years

1.3 Create new policy instruments to deal with gross misconduct

Recommendations at university and departmental level

1.4. Foster informal evaluation practices as well as formal practices

2.2 Enable a more flexible approach to evaluations within departments

3.2. Organise structured discussions about the meaning of quality in faculties and departments

3.3. Develop professional networks between people working on evaluations

Recommendation 1.2: Evaluate the evaluation procedures as a whole everyve years.

Evaluation procedures can never be perfect instruments to assess all aspects of the quality of higher education. However, that doesn't imply that they cannot be improved. As new priorities for higher education emerge, countries should invent new ways to evaluate universities. Consequently, we suggest the holistic assessment of the evaluation practice(s) on national level every 5 years. Current external evaluations clearly do not do so: they only review the quality assurance agency ARACIS, but hardly ever address other forms of evaluation. This time interval would give enough stability for the evaluation practices to be understood and effectively carried out by institutions, but also provide an opportunity for national level stakeholders to make small improvement where needed. Moreover, involving university leaders, faculty and students in this process is crucial, since they are the ones who deal with quality assurance regularly.

Recommendation 1.3: Take misconduct out of the evaluations.

Misconduct (i.e. bribery, plagiarism, etc.) is recognised as a major problem, but interviewees question whether evaluation instruments are the right tools to address it. The problem is one of effectiveness: evaluation instruments do not respond quickly or directly with individual cases of misconduct. Instruments that would be more effective in dealing with misconduct should aim at distributing power within the university and increasing transparency (after all, academic misconduct is abuse of power). Moreover, some innovative tools are now available such as anti-plagiarism software to review previously published and new scientific publications. Cases of bribery in relation to exams can be dealt with more effectively by providing external reviews of students' (dissertation) work or using standardized tests carried out by external examiners.

Objective 2: Allow professors and students to influence the standards for evaluation

Recommendation 2.1: Focus on organising the evaluations without pre-dening all the standards.

The quality assurance agency ARACIS sets two types of standards for universities: a list of minimum standards, and a set of 'reference standards'. Other evaluation practices prescribe similar—or even higher—levels of performance based on which institutions and people are assessed. While these standards are often meant as 'minimum quality', they in fact crowd initiatives of universities, departments and faculty to define quality according to their own terms and standards. It would be more effective if professors and students set many of the standards on which they want to be assessed themselves; it would encourage organisational actors to conceptualize quality and engage in a search for relevant benchmarks. This is also the direction taken in the revised European Standards and Guidelines that are to be adopted in the Bologna Process. Failing to allow professors and students to define more of the standards themselves will continue to create perverse incentives where individuals trick the system, as is currently the case.

Objective 3: Apply a more consistent and open concept of 'quality'

Recommendation 3.1: Reduce the number of criteria on which evaluations are to be carried out.

We have shown above that there are 10 evaluation instruments, with a combined load of close to 300 standards on which the universities have to comply. Every evaluation is based on an implicit (or explicit) idea of what quality is. This preconception is reflected in the criteria or standards set by the external agency that is in charge of carrying out the activity. The criteria vary across the evaluation procedures applied in Romania, which results in an unwanted level of confusion among universities and individuals. The more criteria are predefined, the more limited the possibility of universities to supplement the assessment of quality with additional aspects, tailored to their own needs Hence, the reduction of criteria on which evaluations are carried out can reduce the existing formal inconsistencies, while simultaneously broadening up the discussion on the meaning of quality.

< Prev   CONTENTS   Next >