Security and Rule of Law review to improve funding mechanisms

Presentation of the final evaluation of the Security and Rule of Law (SRoL) research programme

24 June 2020

With the ending of the Security and Rule of Law (SRoL) research programme, NWO-WOTRO Science for Global Development commissioned Ecorys to conduct an external final evaluation to assess its relevance, effectiveness and efficiency of the programme. Strong points were found to be the involvement of practitioner organisations and co-creating activities. Improvements could have be made in the collaborative set up of the programme and the research uptake activities.

EVALUATION REPORT

The review placed a particular focus on the functioning of the key assumptions underpinning the programme design and looked into the working mechanisms and experimental set up of the programme. Overall, the evaluation finds that the projects conducted under the programme have been perceived as effective by grantees, with the impact on personal capacity development within the grantee organisations described as a clear positive contribution.

Project collaboration

Transnational collaboration between northern and southern partners was found to be a key factor contributing to the SRoL Programme effectiveness. This often created a ‘two-way mentorship’, which generated new (research) skills, knowledge, and networks for members of the teams. However, evaluation findings also show that northern organisations are more frequently in the lead and play larger roles compared to southern partners, leading to often unbalanced collaborations.

Activities aimed at co-creation emerged as a strenght of the programme

The requirement to form transdisciplinary teams was considered relevant by grantees, however not at the forefront of day-to-day collaboration. The involvement of practitioner organisations in the research, emerged as a strong element contributing to the effectiveness of the research. Such organisations were considered to bring better understanding of the local context and access to local networks which was key to unlocking knowledge. Co-creation activities also emerged as a strong point of the programme.

A high trend of underspending is noticeable across the full spectrum of projects, with the majority of the underspent funds originally allocated to knowledge dissemination. The underspending trend clashes with the information gathered during this evaluation, which shows that organisations (in the majority of cases southern partners) often report spending unpaid time on the project. In this sense, the trend could be a reflection of the imbalanced relationship between northern and southern partners, where the northern lead is responsible for budget allocations.

The contribution of NWO-WOTRO

With regard to the role played by NWO-WOTRO, the evaluation finds that overall, grantees were positive with regard to its support and assistance, in particular with monitoring and evaluation (M&E) activities and the design and implementation of impact pathways. The majority of grantees found M&E activities (highly) relevant to keep their project on track and achieving objectives. Impact pathways were regarded as a useful mechanism, but the local partners in target countries seem to encounter more difficulties in designing and applying them.

Impact pathways were regarded as a useful mechanism

The selection procedure designed and implemented by NWO-WOTRO was found to be transparent and fair. However, the length and complexity of the process raises questions related to the appropriateness of the process in relation to the needs of the donor organisation for fast, applicable research. Furthermore, the majority of the cases projects are applied for and lead by a northern organisation. A deeper look into the strategy for the dissemination of the calls and the selection criteria indicates that more could be done in this sense to ensure a more balanced pool of applicants and grantees.

In terms of efficiency, this evaluation concludes that the changes to the scope of the programme activities introduced in 2015 implied a severely increased workload on the limited staff available at NWO-WOTRO, which affected the cost-efficiency of the programme, as costs increased without a parallel increase of the overhead budget.

Working together on the programme

The SRoL research programme started in 2013, and is managed by NWO-WOTRO. The programme is part of the Knowledge for Development Policy of the Netherlands Ministry of Foreign Affairs (MFA). The Knowledge Platform Security and Rule of Law (KPSRL) formulated the research agenda for the SRoL research programme. The research programme used a combined approach of knowledge development and application thereof by funding both strategic and applied research.

The evaluation finds that the initial working arrangements between NWO-WOTRO, the KPRSL and the MFA, while justified when the programme was launched, were not effective. This can be attributed to the lack of formal contractual arrangements between NWO-WOTRO and the KPSRL and the resulting absence of a clear division of roles and responsibilities between the two. This affected in particular knowledge uptake and dissemination of research results, also due to the lack of a dedicated pool of funds within one of the two organisations. The evaluators confirm that significant steps forward have been taken to improve this tripartite relationship throughout the lifespan of the SRoL programme, although some obstacles – in particular with regard to research uptake – remain.

Recommendations

Not less than seventeen recommendations have been put forward:

  1. Continue emphasising the importance of the research process in order to engage local decision makers from the start.
  2. Ensure ownership of the MFA and/or embassies at the start of the project, as well as ownership of respective local public institutions.
  3. Concentrate the resources in one ‘overarching’ fund and specify per call what the exact objectives of the projects should be.
  4. Extend the evaluation process with the inclusion of an ‘impact assessment’ one or two years after their completion.
  5. Address the northern and southern relationship consistently as a two-way mentorship process.
  6. Revisit the assessment of impact that projects generate by also paying attention to more qualitative aspects such as fostering dialogue and relationship building.
  7. Actively encourage consortia to work with practitioner organisations.
  8. Reflect on how to better institutionalise M&E in the research programming, irrespective of the duration of the project.
  9. Simplify the selection procedure, without compromising on the quality of the research selected.
  10. Enhance the ‘openness’ of the calls and draw in applications from different types of organisations across the world.
  11. Institutionalise the relationship between NWO-WOTRO and the KPSRL in the context of the SRoL Programme.
  12. Relax requirements related to the secrecy of proposals in favour of better cooperation with the KPSRL and increased exposure of the research through their network.
  13. Set up a dedicated budget for knowledge dissemination activities.
  14. Request applicants to be explicit about risks and their mitigation measures in the proposal phase.
  15. Provide clearer guidelines as to how budgets should be submitted in proposals.
  16. Pool unspent project resources for knowledge dissemination into an uptake fund.
  17. Consider an overhead budget for NWO-WOTRO that is in line with the expected amount of work.

Read further


Source: NWO