A collection of superstars does not make a top university

Science can improve in terms of quality and become even fairer if scientists are assessed on the basis of the content and societal impact of their work. And that can quite easily be measured according to the chair of the Young Academy Belle Derks and vice rector Research at Utrecht University, Frank Miedema.

credits: Van Santen & Bolleurs

Text: Malou van Hintum, photography: Van Santen & Bolleurs

Many scientists seem to agree that the current practice of assessing researchers based on their publication scores and citation index will not continue to exist for much longer. But what is the alternative then? ‘Take time to read an article from a colleague instead of just counting the number of publications’, says Professor Belle Derks, chair of the Young Academy. ‘Then you can assess the work on the basis of its content.’ Furthermore, it should be universities that once again perform that assessment. That is where it belongs, in her opinion.­­­­

Neither seen, nor recognised

‘Now researchers must convince NWO and if they are successful, they receive a grant for their research. Science has become a competition in which superstars, who are good at selling themselves, flourish, at least if they do not get a burnout. Researchers who are also very capable, but lose the rat race for the big money, regularly give up because the system fails to see and recognise them. Does such a competitive system guarantee qualitatively good research? And do all of those individual scientists working on their superstar status, together ensure a qualitatively good university?’­­­­­

What exactly is quality?

Derks would like to see far more team science. Not just by established researchers in large consortia who have already demonstrated their excellence by acquiring NWO funding, but also for young scientists at the start of their career. She believes that science is about collaboration. ‘A superstar status simply has no place here. I think it is worthwhile developing new criteria, but not if that means that researchers will have to be good at everything in future.’ In brief: whoever wants to realise innovations in the recognition and assessment of research will need to determine what quality really is and consider how science can be organised in such a way that it can provide that quality.­­­­

Give something back in return for taxpayers' money

Frank Miedema, one of the founding fathers of Science in Transition (2013) and currently vice rector Research at Utrecht University, states that scientific quality must be assessed according to the impact that research results have on a certain problem. ‘Scientists are obliged to give back to society in return for the taxpayers’ money that funds their research. And by that I do not mean appearing on a popular talk show like De Wereld Draait Door to shine for three minutes with your research and then count that as a form of valorisation’, he says. ‘Doing science that makes a difference for society is about listening and collaborating: with colleagues, but in particular with interested parties outside of the university. For example, in medicine, you can ask patients and their families which research they need.’­­­­­­­­­­

More reading, less counting

Miedema thinks that it is perfectly possible to measure the societal impact of science and of team science too. Pilots for this are currently being carried out at various international institutes. ‘At UMC Utrecht we are doing that with an inclusive set of indicators. We ask for example: How did you reach your research question? Have you also spoken with patients as well as colleagues in another discipline? What about people outside of academia? Are your data available in open access form? Publications remain important, but we also consider how researchers go about their work and who they work with. For example, if they are doing research into obesity in a neighbourhood, then do they have the right social contacts? Have they spoken to sociologists and economists? Does the research team consist of the right people and do they know how to include questions that are important to external parties?’ In their own portfolio, researchers can subsequently say which team they were a part of, what their responsibility was, which data they helped produce and add the appropriate references. This does not lead to a number or a new index emphasises Miedema. ‘It is about assessing the content, reading as much as possible and not counting. You genuinely need to understand what you are assessing. Somebody can say, for example: together with others, I ensured that very good biological material from a certain patient cohort has become available for other researchers to use. And you can gain credits for that too.’­­­­­­­­­­­­­

Valorisation is not starring in a popular talk show like De Wereld Draait Door for three minutes

An unduly lower status

Such a new form of assessment also has room for "softer" subjects that international top journals do not currently consider. ‘Nevertheless, in medicine, for example, they are incredibly important for patients and their families’, says Miedema. A lot of research is published into the molecular-biological aspects of how strokes occur. But multidisciplinary research into innovations in the rehabilitation programme following a stroke – learning to talk again, learning to deal with the paralysis or reduced capacities – is not published in top journals. ‘That unduly has a far lower status. In other disciplines – educational research, social psychology, economics – you can see something similar.’­­­­­­

Diversity in careers

According to Miedema, in the current system the odds are stacked against people who do good research and are excellent in their subject, but due to the emphasis on more fundamental research fail to count in terms of international publications. ‘That has a one-sided directive effect on the research agenda, as a result of which important societal questions are not considered. Consequently, there is a large movement within and beyond Europe, which calls for assessments based on the content of the research, for diversity in careers, and for distinguishing different forms of excellence. For young scientists it is extremely important that there will now be a stable, new and fair assessment system in place very soon. Such a new system must contain all of these elements and people are working very hard on realising that, also in the Netherlands.’­­­­­­

NWO magazine on science

Onderzoek is NWO's new magazine and the successor to Hypothese. NWO develops science policy in dialogue with the field and we want to show that. In this magazine we discuss the developments, wishes and other voices in the scientific field. Onderzoek is sent free of charge to researchers, policymakers, journalists and other relations of NWO.