From the chair of NWO Domain Science

Column 7: First review of NWO Open Competition Domain Science

A first glimpse at the results of new NWO Open Competition Domain Science.

Dear colleagues,

Let us start the year with a look at the new Open Competition. We realise that 2018 is probably still fresh in your memory as a year of transition – a difficult year with historically low success rates in the final rounds of the various open competition schemes. We promised that things would look up in 2019, with a 30% increase in the total budget for Open Competition and the launch of a new set of instruments. Has this new approach been effective? Time for a first review.

To refresh your memory: we launched the new Open Competition Domain Science in August 2018. Domain-wide, with three building blocks: XS – a superfast competition for small, high-risk proposals; KLEIN – a deadline-free option for projects involving one or two PhD positions or modest investments; and GROOT – a competition for programmes or consortia. This model replaced a wide variety of open competition schemes that ran in separate fields (gebieden – the NWO organisational entities that formed the NWO Domain Science).

While all of the team's experience and expertise went into the design of the new scheme, it was still a step that involved a good deal of uncertainty. Each and every one of us had questions. How would the various disciplines fare in the new system? Would it be difficult for some (sub)disciplines to achieve success? Would the scheme deliver on the promise that multidisciplinary proposals would be given a fair chance? And would monodisciplinary proposals still be viable? Perhaps most importantly, does domain-wide assessment work in practice? Are we – as assessors, as well as proponents – happy with the new scheme? Do we believe that we are granting the best proposals? And – not unimportant – what about the success rate? These are all essential questions. Questions that, from the start, the board has pledged to monitor continuously and evaluate annually, with the proviso that we will let the scheme run for at least a year without making any changes. This is because we do not want to act on (or react to) figures that are too limited in scope.

Niek Lopes Cardozo, chair of the Board of NWO Domain Science.  Photo: Bram Saeys.Niek Lopes Cardozo, chair of the Board of NWO Domain Science. Photo: Bram Saeys.

We will issue a report on the statistics early in 2020, but here is a taste of how things are shaping up.

In KLEIN, the success rate to date has been at least 25% for regular awards and over 30% if preferential treatment proposals are taken into account. The success rate is almost the same for all disciplines, and no discipline has shown a discontinuity in the number of awards compared to the old system.
Importantly, about half of the proposals are multidisciplinary, while the success rate for monodisciplinary and multidisciplinary proposals is nearly the same. No significant difference in success rate has been observed between male and female applicants.

We have now had two full rounds of XS, which was launched as recently as mid-2019. The interest in this scheme has turned out to be high, with just over 100 proposals per round. The good news here is that the mutual assessment scheme – the applicants also act as jurors and rank the applications – appears to work well and interestingly, the applicants are enthusiastic about this approach. The mutual assessment procedure facilitates a quick turnaround: to date, the results have been communicated to the applicants within five weeks of the submission deadline, due in no small part to efficient processing by the NWO Domain Science office.

Meanwhile, the first edition of the two-yearly NWO Open Competition Domain Science - GROOT has reached its final assessment stage. We expect to see a success rate of just over 20%. More details to follow.

So much for the specifics. Generally speaking, the figures are looking good: all initial indications suggest no major discontinuities, a spread over disciplines, a balance between monodisciplinary and multidisciplinary proposals, and no gender bias. Success rates have improved as promised: good for KLEIN and perfectly acceptable for GROOT, while XS also appears to be meeting expectations.

Whether or not all of this means we are satisfied with the domain-wide assessment process is a very different question. The feedback we have been receiving from both jurors and applicants ranges from very negative to very positive, and everything in between. Needless to say, the quality of the assessment is of fundamental importance to the functioning of NWO. An in-depth evaluation is therefore ongoing and you can look forward to an update on this process in spring 2020.

 

Niek Lopes Cardozo, chair of the Board of NWO Domain Science
20 January, 2020

 

Previous columns from the chair