Start of the Academic Year 2019-2020

2 September 2019

The annual festive celebration of the Start of the Academic Year takes place at 2 September in the Dom Church. This year's theme is recognising and valuing. Stan Gielen, President of the Dutch Research Council, is one of the speakers. In his speach Stan Gielen highlights the necessary measures to change the way we assess researchers and research proposals. The current situation calls for unorthodox matters.

Ladies and gentlemen, members of the Executive Board,

It is a great honour to be asked to speak to you at this Start of the Academic Year of Utrecht University, the university where I was appointed as an Assistant Professor in 1980, and got an opportunity to build my own research group for the first time. I still have many fond memories of that time.

Today’s theme, ‘Recognising and Valuing’, is an important and topical one. It should be considered alongside another pressing issue: how can we reduce the workloads of academic staff? As a result of the many demands placed on staff in Dutch universities, pressure and workloads have risen to unacceptable levels.

Reducing workloads and, in relation to that, answering the question, ‘how should we recognise and value the performance of university staff?’, is no simple matter, not least because the various aspects of the issue cannot be considered in isolation. Let’s take an example: To limit high workloads, which have come about due to high student intakes, and to guarantee educational quality in the science and technology area, some institutions have decided to set intake restrictions.

We thus have a teaching problem at the knowledge institutions. Because there really is a need right now for more highly-trained people in these areas, the Minister of Education, Culture and Science has allocated funds for a science and technology plan to create additional staff positions. But with this investment comes an estimated 350 new staff members in the science and technology area, who are expected to conduct research as well as teach. And so they submit research funding applications to the NWO. The result is more proposals that have to be assessed by other researchers, increasing the workloads of NWO staff. By approaching a teaching problem in this way, we have thus created a series of new problems for both university researchers and for the NWO.

Reducing workloads and the number of funding applications are therefore not problems that can be solved by the universities or the NWO alone; they must be tackled collectively by knowledge institutions, the NWO and the government.

As Chair of the NWO and with regard to the theme of Recognising and Valuing, I’m going to focus today on the research component in particular, and specifically on two aspects of that: the assessment of researchers and research groups, and the assessment of research proposals.

Assessing researchers is mainly about the question: when are you a good researcher? If someone tells you that he or she has had an article published in The Science of Nature, the usual reaction is to think that you’re talking to an excellent researcher. However, many publications in The Science of Nature have few citations; some have none at all. Furthermore, recent research has shown that there is no significant correlation between the impact factor of a journal in which a researcher publishes and the number of citations to the author’s articles. Someone who’s been published in The Science of Nature is therefore not necessarily an excellent researcher.

The idea that we should stop using the impact factors of journals when assessing researchers and research proposals has already been advocated by Science in Transition and in particular by Frank Miedema of this university. The emphasis on quantitative criteria means that the evaluation of researchers is very one-sided, with aspects such as creativity, connecting links between research disciplines, team science and social relevance fading into the background.

NWO recently signed the San Francisco Declaration of Research Assessment (DORA). We are no longer going to ask for an h-index or for the impact factors of the journals in which researchers have been published. Instead, we want to know about a researcher’s contribution to science and about the scientific or social impact of that contribution. The answers to that question can be many and varied. The contribution might be an entirely new scientific insight, but it might also be a special database that can be used by other researchers, or a new experimental protocol or algorithm for data analysis. Science is so broad and so wide-ranging that it can’t be captured in just a couple of criteria. Accordingly, for a number of funding tools the NWO has recently introduced a step that gives research funding applicants the opportunity to indicate what their contribution to science has been.

Now let’s look at the second aspect of research assessment: how do we assess research proposals in terms of their quality, originality and potential scientific and/or social impact? This is becoming increasingly difficult for research funders. The quality of applications is increasing, which means it’s getting harder and harder to select, out of the very best proposals, those proposals which should be funded. The number of research funding applications is also increasing, sometimes to an absurd extent. As an example, in the first round of the Dutch Research Agenda in 2018, 54 million euros was available for research consortia. There was a big uproar when it emerged that the acceptance rate was extremely low. Why was that? If you consider that a typical university employs approximately 10% of all researchers in the Netherlands, then you might expect that university to receive an allocation of around 5.5 million euros. Well, there are universities whose researchers requested more than 100 million euros for their projects. That’s an oversubscription by a factor of 20. So I agree entirely with the criticism that ensued: this is a waste of time, and it must change.

It is in all of our interests to reduce the number of applications:

  1. To prevent researchers spending too much time writing and assessing applications that have little chance of being accepted
  2. For institutions, so that more time can be spent on teaching and research, and
  3. For the NWO, to make the assessment process more efficient.

The high number of applications is an international problem, not unique to the Netherlands. But let me just say a few words about the Dutch situation. According to data from the VSNU, the number of PhD candidates at Dutch universities has risen 17% in the last ten years (from 7,425 in 2008 to 8,681 in 2017). The numbers of Other Academic Staff assigned to research (such as post docs) has risen by 27% (from 3,043 in 2008 to 3,858 in 2017). There have never been so many PhD candidates and post-doc researchers working in Dutch universities. Then because of this, there is a drive to submit a substantially higher number of research funding applications; so many, in fact, that even at a reasonable acceptance rate of 25 to 30%, it would lead to a more than doubling of the number of PhD candidates and post docs.

To start with, as research funders and knowledge institutions we need to put our own house in order first. We use the number of grants obtained as a criterion to establish the quality of a research proposal or research group. A study by Science Europe, the organisation which represents most of the research funders in Europe, shows that in nearly all countries, successful acquisition is an important parameter in research group assessment and researcher promotion. At many universities, the awarding of an externally-funded project leads to an extra financial contribution to a faculty or research group, and successful acquisition is usually a prerequisite for promotion. Increasingly, therefore, the primary driver of research grant applications is a financial or personal interest. The means (namely a grant application to be able to do good research) is thus elevated to become the end. You can’t blame researchers for playing this ‘game’, if I can call it that: their future depends on it.

The Van Rijn Commission recently stated that the funding model for education based on student numbers had given rise to perverse incentives and was encouraging a ‘race to the bottom’. I would say that connecting a financial or personal interest to successful research funding applications is another perverse incentive. It’s a catch-22 situation, and young researchers are its main victims. Faced with these imposed incentives, there is nothing they can do but submit applications.

What should we do about it?

The NWO is working hard with knowledge institutions and with you to reduce academic workloads and the number of applications. In consultation with the universities, we’re going to adjust and update the assessment criteria. The NWO is consulting with the VSNU in this regard. The NWO (and the VSNU too, I am sure) will ensure that any adjustment of the criteria is done in an international context: it makes no sense for the Netherlands to be all alone on this issue. Our researchers work in a global community and must be assessed on the basis of the same criteria as their colleagues abroad.

The NWO has already abolished the use of the h-index and impact factors. New criteria have also been introduced in the NWO’s calls for proposals, which will be better tailored to specific funding tools. Naturally, this will be done in close consultation with research funders in other countries.

The NWO is also considering removing the question about the number of previously obtained grants as a criterion of quality, or at least aligning it with the goal of the grant programme concerned. A Gravitation application will now be assessed quite differently from a Veni application. The NWO will give explicit instructions to this effect to assessment panels and reviewers.

Assess your staff on their research performance

I also want to call on universities to remove the acquired grants criterion from their evaluations of researchers and research groups, and to not make researcher promotions contingent on the number of grants obtained. Assess your staff on their research performance (and on their teaching performance, of course, but the NWO has no say on that).

The NWO will also critically evaluate the effectiveness and efficiency of its funding tools. To that end, the NWO will exchange its data with other research funders so that we can learn from each other and improve the effectiveness of our tools.

Finally, the NWO will also meet with the universities to discuss how we can reduce the number of grant applications. It makes no sense for the number of grant applications to be twenty times higher than the number of grants awarded. That’s frustrating for the applicants, for the reviewers, for the assessment panels, and also for the NWO.

I hope that I’ve been able to outline for you the issues we’re facing, and show that we’re working hard to do something about them. We can only do that if we work together: researchers, universities, the NWO and the Ministry.

The current situation calls for unorthodox measures. Asking for more money is an option, and one that I think would be justified to help restore order to the situation. If we want the Netherlands to remain a world leader, we must continue to make investments. But the Minister would then be entitled to expect us to show that we’re serious about changing the way we do things.

Thank you for your attention.

Stan Gielen

Read more

Source: NWO