Models of implementation continue to evolve for many NGOs. One big trend that particularly interests me is the shift from direct implementation of projects and programmes by international NGOs (INGOs) towards working with partners to support and build their capacity. Networks that represent particular groups are often a key part of this model. They may represent groups like young people, older people or those living with or affected by HIV. As structures they can provide legitimacy for participatory monitoring and joint advocacy efforts.
The shift from direct implementation towards technical assistance and capacity building support has implications for resource mobilisation, advocacy, programme innovation and of course monitoring and evaluation (M&E).
I’ll talk first of M&E and come back to the other points later.
The shift towards working with partners and networks brings with it challenges for monitoring and evaluation. Instead of collecting data for one organisation, we must now shift to collecting, sharing and using data in networks.
There are of course many excellent M&E tools and approaches that work well in this context, helping to measure the effectiveness of capacity building or the attribution of shared outcomes. See more on outcome mapping for one example.
The challenges I see relate more to the structure of M&E frameworks and systems, which are typically developed in the context of an organisation. Both BOND and NPC have done some interesting work on shared measurement frameworks. However, I am not aware of work looking at shared M&E systems. Please let me know in the comments if you know of some good examples.
The organisational focus of M&E systems can lead to tensions and difficulties in a network context. Almost all INGOs have their own frameworks, tools and systems. These have been developed to enable them to be accountable to donors and their broader stakeholders. When INGOs start working with partners and networks the default assumption is often that the partners should also use these systems. This is especially the case where the INGO acts as an intermediary, channelling funding from other donors.
Pity the NGO partner working with multiple INGOs. I’ve seen examples where they are encouraged to use different systems for each INGO they work with. Each working differently and based on different M&E frameworks.
What are the alternatives?
I’d like to share some early experiences from our work with government and civil society in this area.
Our work with groups like HelpAge International, the Citizen Engagement Programme and the South African government takes a more decentralised and emergent approach to this problem. We start from the assumption that each partner in a network should be able to have only one M&E system that is designed to meet their requirements. We focus on how we can enable different systems to easily share data (should they wish to do so). Back-stopping this, we also focus on how we can provide simple and low cost M&E systems for those partners in the network that don’t already have one.
The starting point with this is ensuring that all partners have a way to collect the same kind of data in the same way. To explain how this works take a look at the diagram below.
As the diagram illustrates it is based on two shared databases (or hubs as we call them) that the network partners co-manage. One contains forms (or data definitions it you want to get technical). The other contains the data collected by these forms.
The starting point for a network is to share forms and other data collection tools that they are already using (or would like to use). The Form Hub provides a way to re-create digital versions of these forms that:
(1) Share common metadata like:
Who created the form and when
Who last saved it and when
Geographical coordinates (if applicable)
Tags to code the form by common categories
(2) Enable different forms to be analysed together
What does this mean?
NGOs in a network now have a way to co-create and share data collection forms. If they agree to use the same forms (or versions with differences that do not affect the integrity of the data) then they can also collect data that is both comparable and possible to aggregate. This means that data collected by different organisations in different systems can be analysed together. More on this later. First, how would this work.
Organisations with their own M&E system
Existing systems would need two one-off modifications to work. First, integration with the Form Hub and Data Hub application programming interfaces (APIs). APIs enable different databases to communicate. In this case the one-off integration is needed to:
- Allow an existing M&E system to view forms on the Form Hub and download ones it needs
- Send data collected using these forms to the Data Hub
- Request data from the Data Hub (which could include data from other partners too)
Second, the system will need the ability to save data it collects in the same format as the forms on the Form Hub. Specifically that means transforming it into JSON, which is becoming a common format for data interchange.
Otherwise the M&E system continues to work as before. The changes above mean that it can use forms from the Form Hub and share data with the Data Hub.
Organisations with no M&E system
For those without an M&E system we can offer an even simpler option. BetterData (our M&E platform) offers a web and mobile version that is ready to use with Form Hub forms. It will also communicate directly with Data Hub. For partners using an M&E system for the first time it is as simple as downloading the forms that they need to use.
Implications for networks
This approach offers one way of balancing the tension of using existing systems (focused around an organisation’s needs) while sharing data with network partners when they choose to do so. This means that each M&E system can collect other data, not related to network activities and can continue to be managed independently of network partners.
We’re working to extend the analysis tools in the Data Hub. These currently enable you to generate reports the filter the data by criteria like time frame, geographic area or tags. Reports can be assembled, including specific fields from specific forms that aggregate data across all partners. I see implications in several areas:
Innovation - partners that innovate and test new approaches have a way to quickly share and scale these approaches with others. Once a new implementation approach is tested, the forms to manage and monitor implementation can be easily shared with others.
Planning - several of the partners we work with collect data on planned activities. This covers where they plan to work and when. Sharing this with a network would make it possible to identify synergies or overlaps before implementation begins. Joint planning of this type could ensure that limited resources are best targeted where they are most needed.
Benchmarking and value for money - we’ve also explored ways in which partners can use this approach to share data on activity cost and delivery. This opens the door for cost effectiveness benchmarking, looking at questions like how much each activity costs, how long it took to deliver and what kind of outputs it produced.
Joint learning and advocacy - this is one of the highest priority areas for our partners. By working in partnership to both collect data and advocate for change they can have a far greater impact. This can be particularly powerful when working with networks that represent marginalised groups to both collect evidence from these groups and support them to advocate for change.
No more reports - finally my favourite, no more reports. This one may require some advocacy to donors before it becomes a reality. However, imagine if you could give your donors access to specific data you and your partners have collected in the Data Hub? If you’re M&E system is well designed this could include financial data, reflections on challenges and lessons learned, as well as outputs and outcomes. Using the reporting tools in the Data Hub you (or they) can generate any report that they need.
We’re beginning pilots to implement this approach in two different contexts. First, across different levels of Government in South Africa. Second, with networks of NGOs working together. If you’d like to know more then get in touch. Otherwise I’ll post again when we have some results to share.