We all talk about using data to make decisions in our work. From data driven management to adaptive management, to learning organisations there is lot of discussion about what this means and why it is important. of the importance of using data to make decisions.
However, putting these ideas into practice is not always easy. Put this in the context of a large, decentralised development programme and it starts getting very complex. This blog post looks at the challenges of using data for decision-making in your development programmes. It discusses how to ensure you have the data you need to make decisions in your programme when you need it.
Collecting data is time-consuming. Unless you plan in advance to collect the data you need to make decisions, it's unlikely that it will be available in time for you to make decisions when you need to make them.
Programmes create monitoring and evaluation plans or frameworks. These set out in advance what data they will collect, how, when and who will collect it. However, in my experience their focus is on performance data. It's less common to see them cover operational data needed for operational decisions.
Experienced programme managers recognise this and address it. However, they often end up with a parallel set of systems to those used for monitoring and evaluation. The ones I've seen consist of several complex spreadsheets that track activities, payments made or received and implementation status of programme sites.
There is plenty of guidance on creating monitoring and evaluation plans. However, I've struggled to find much guidance to help programme managers who need to plan what data they need to collect to make operational decisions for their programme.
This post shares some learning from an approach that Kwantu has developed called Process Mapping. It also links to a new guide that we've developed to help document your programme processes in a standardized way.
What data should we collect for decision-making?
The first thing to consider is what data you need to collect to be able to make decisions. This obviously depends on what your programme does. Let's use an imaginary example to explore this: - the Civil Society Capacity Support programme (or CSCS for short).
The objective of the CSCS is to build the capacity of civil society within the education sector. (A real programme would probably have more specific objectives than this of course).
Some of the questions the programme managers might need to answer during implementation are:
- Which civil society organisations should we support?
- How much funding should we provide?
- What kind of capacity assistance should be offer?
- How do we determine when a civil society organisation 'graduates' from the programme?
How do we ensure we have the data needed to make operational decisions?
OK, we've identified some key operational questions that need data to answer them, how do we get that data? In my experience it helps to think of your programme activities as a process with several steps. Each step relates to a point in time where you need to make a decision. To make that decision you need to collect data.
The process of the CSCS programme might look like this:
- Launch call for civil society organisation (CSO) applications to join programme
- Select CSO participants
- Assess CSO capacity
- Develop capacity building plans
- CSO grants awarded
- CSO grant contracts agreed
- Quarterly report due
- Quarterly report received
- Quarterly report approved
- Quarterly payment processed
- Capacity building provided
- Assess CSO capacity
This is actually one high level process with a sub-process linked to one step. The sub-process covers a cyclical process of sub-granting to CSOs, which repeats every quarter as reports are due, received, approved and then the next payment processed.
Now let's turn back to the example questions above. A simple process map like this can help programme managers to define the key points in a programme where they need data to make these decisions. For example:
Which civil society organisations should we support?
This decision is clearly tied to the 'Select CSO participants' step shown above.
The programme can only select CSO participants once it has agreed on selection criteria and assessed which CSOs best meet these criteria. This could be done via an open call for proposals or via actively reaching out to CSOs. Either way, the programme managers need data on both the selection criteria and on how each CSO meets these criteria. No decision can be taken until this data is available and processed to the stage where it can be used to make a decision. This might mean a ranked list of CSO according to ten different criteria with their own weightings.
How much funding should we award?
This decision is clearly tied to the 'CSO grants awarded' step.
Before the programme can award grants to CSOs it might need to understand both what activities they plan to use the funds for (and of course the results they aim to achieve) and which areas they need assistance with to build their capacity. This in turn means completing a capacity assessment and developing capacity building plans for each CSO first. Once this data is available it can again be processed to determine what funds are needed to support both the activities the CSO plans to implement and any areas where funds are needed to enhance capacity.
The same thinking applies to the rest of the questions above. Each can be tied to a step in the programme's process. From this we can reflect on what information is needed to make this decision.
Creating a process manual
These are simplistic examples of course. Your reality is no doubt a lot more complex and likely will include many more than four questions.
The point is to illustrate how you can map your operational processes to define what data you need at each step where operational decisions must be taken.
As with a monitoring and evaluation plan, it helps you to think systematically about what data you need to collect, when it must be collected and by who. Clearly the next step would be to document your management data needs in the same way as a monitoring and evaluation plan documents the M&E needs of a programme.
We call this a process manual.
It aims to document your programme processes in a standardised way. In addition to the areas above it would also typically cover:
- Detailed documentation on what activities happen at each step, which data collections tools should be completed and who is responsible
- Integrated data collections tools (that serve the needs of both management and M&E)
- Common taxonomies of keywords that are used to cross-reference data for easy analysis and aggregation
- Clear definition of all role players in the programme, along with their responsibilities
If you feel this approach could help you then take a look at our new guide to developing a process manual. This draws on lessons from over twenty years working on both government and NGO programmes.
We look forward to your feedback!
Click on the image below to download our new publication Creating a Process Manual for Your Development Programme.