Excellent research within Integrated Programmes just as well relevant

Findings of an external evaluation of the Integrated Programmes

20 February 2018

Excellent scientific findings and relevant societal insights often go hand-in-hand. The Integrated Programmes were valued highly both from within the academic community and by local non-scientific stakeholders. The research results were both relevant and effective in addressing the needs of national policies in developing countries. Furthermore, the projects succeeded in producing out-of-the-box insights relevant to development stakeholders. These were the main conclusions from an external evaluation of the NWO-WOTRO research programme, executed by Syspons.

The Integrated Programmes of NWO-WOTRO Science for Global Development were established based on new and existing networks of researchers and practitioners, making use of synergies and capacities. The open competition scheme gave more flexibility to researchers and had a less strong programmatic approach with predefined themes and approaches.

Involving local stakeholders

Research projects were aimed at both excellent research with developmental relevance and high relevance for local target groups, e.g. policy-makers or development practitioners. By involving local stakeholders – also from outside the projects - in the proposal workshops right from the start of each project, the programme ensured that research questions and methods were appropriate to the needs of development stakeholders in each country. As a result of one of the projects on health, patients now benefit from treatment procedures that are adapted to their needs. To be able to adapt to these needs, flexibility is needed though. For example to adjust the initial research questions.

The Integrated Programmes show mixed performances in terms of motivating non-scientific stakeholders to apply research insights into development practice and policy. Toolkits and support mechanisms for guiding researchers in their work with non-scientific stakeholders were not present at the start of the programme and only developed throughout the programme’s implementation. NWO-WOTRO has already taken up these learnings and increased their support in later research programmes, for example by offering trainings to non-scientific stakeholders on how to apply new tools or technologies.

Though policy-makers at national levels showed an interest in the research, compared to other target groups, they were much less willing to apply research insights in policymaking. In projects where the focus was more on local development organisations, district governments or local communities, application of research into development practice was much more pronounced.

Focus on international collaboration

The cross-disciplinary approach promoted out-of-the-box solutions to development challenges that incorporated a number of different perspectives. The Integrated Programmes’ focus on international collaboration furthermore not only promoted further knowledge exchange and partnerships between Northern and Southern researchers, but also contributed to a higher scientific impact of publications.

Factors that contribute to impact

Especially those researchers who emphasised informal collaboration, relationship-building and frequent communication and interaction with their non-scientific stakeholders were generally more effective in influencing their target groups to apply research insights. Moreover, building and maintaining a close link to decision-makers right from the start of the project and supporting them by developing and implementing practical tools and methods in their work positively contributed to impacts. Also, researchers who were motivated by the desire to implement practical solutions and characterised their work as being of a high-risk high-gain nature were moreover more likely to influence the practices of their respective target groups. Although these behaviours were exemplified by a share of the researchers in the Integrated Programmes, the programme was yet lacking a comprehensive, systematic approach for promoting them more comprehensively.

The Integrated Programmes

In 2017, the last activities emanating from the NWO-WOTRO strategy Science for International Development (2007-2010) were completed. With this strategy plan, NWO-WOTRO turned towards a problem-oriented mission of addressing development issues (away from a geographical focus on tropical countries). This shift in focus also entailed the prioritisation in funding of interdisciplinary research, equal partnerships between Northern and Southern researchers, capacity strengthening, stakeholder participation in all stages of the research process and fostering the use of results. Research funded had to be both scientifically excellent and relevant to society. These components, which are now common in NWO-WOTRO programming, were pioneered ten years ago. Also, strategic networking and support activities were executed to increase mutual appreciation and synergies between researchers and stakeholders.

A number of thematic research programmes were developed under this strategy. Amongst others the ‘Integrated Programmes’, that were aimed at  ‘supporting excellent problem-oriented interdisciplinary research to the benefit of development and societal issues in the South’. The Integrated Programmes were intended to deliver challenging, out of the box insights in the complex problems related to development issues in general, and the MDGs in particular, and possible new directions for solutions while also deepening scientific knowledge to the benefit of informed decision-making and new development policies as well as building research capacities in the contexts of developing countries.

The forty Integrated Programmes were selected on their scientific excellence, development relevance and international collaboration and also on interdisciplinary approaches, capacity building and involvement of non-scientific stakeholders. Each Integrated Programme consisted of multiple individual projects, of which at least one researcher had to come from a Low- and Middle Income Country (LMIC).

Evaluating research

Between July 2017 and December 2017, the Integrated Programmes have been independently evaluated by Syspons. The evaluation team conducted an in-depth analysis of all relevant documents and data, in-depth interviews with programme stakeholders, qualitative interviews with researchers and the external stakeholders from ten projects and a survey of all researchers. Based on the evaluation’s findings, the evaluation team developed recommendations for NWO-WOTRO’s future research funding programmes.

Besides this programme evaluation, a bibliometric study and a programme evaluation of ‘Global Health Policy and Health Systems’, were executed. Conclusions drawn from all three evaluations are foreseen in the beginning of 2018.

Please read the FULL REPORT


Strategic recommendations

  1. NWO-WOTRO should continue to fund international collaboration of researchers with interdisciplinary approaches.
  2. In future research funding programmes NWO-WOTRO should finance projects which are
    a. of a high risk / high gain nature;
    b. emphasise intense informal collaborations and intense relationship-building with non-scientific stakeholders;
    c. build and maintain close links to decision-makers from the start until the end of the project and;
    d. develop and implement practical tools and methods in their projects.
  3. NWO-WOTRO should address target groups with stronger willingness to apply research insights into development practice, for example by formulating selection criteria.
  4. In future research funding programmes that involve PhDs, NWO-WOTRO should plan ahead with a stronger need for capacity building for PhD candidates.
  5. NWO-WOTRO and the Netherlands’ Ministry of Foreign Affairs should continue emphasising research that caters to the development contexts of countries where research takes place.
  6. NWO-WOTRO together with the Ministry of Foreign Affairs should decide whether they want to capitalise on the achieved results of the Integrated Programmes by funding successful projects to further their impact.

Operative recommendations

  1. NWO-WOTRO should continue to demand an intense involvement of target groups and non-scientific stakeholders in the proposal phase of the bidding process. Moreover NWO-WOTRO should further intensify the involvement of non-scientific partners throughout the research.
  2. Future research programmes should provide more support but also guidance and some control mechanisms to researchers on dissemination activities and achieving impacts, i.e. target groups actually implementing the results.

Source: NWO