Monitoring and evaluation (M&E) is a critical part of best practice CSI. It is the way in which we answer that fundamental question about our work and about the projects we support: “What difference does it make?”
As we have discussed, monitoring refers to the measurables or indicators in a project that help us to ascertain if we are moving towards our goal or desired destination in a satisfactory manner. These things should be agreed upfront between donor and project even before funding begins. These measurables will tend to be found in a project’s application for funding and will have informed the decision to support that project in the first place. Monitoring these is a process that should take place throughout the CSI funding relationship.
We now move on to the sometimes more difficult process of evaluation.
The evaluation part of M&E is about measuring outcomes and impact wherever we reasonably can, so as to evaluate if the route or strategy we have taken has been effective as well as to identify areas for improvement.
The necessary practice of “impact assessment” can be a difficult one, since it is usually exceptionally difficult for any donor to draw anything like a straight line between moneys made available and the results of a project over time.
Partly this is because these results tend to take place over long periods, sometimes frustratingly moving at a thing known as “community pace” and possibly the result of work and support from times well before current investment in the project.
Mostly, projects claim various supporters and types of support – and just which of these can claim credit (or should that be part-credit?) for success is often impossible to work out with any real confidence.
There are many related problems with an over-insistence on impact measurement, not least being that of deciding whose success one is talking about – the organisation making success happen on-the-ground, or the donor, however closely involved in the project?
- We need to measure impact against something, and usually this means that we’ve pre-set specific measurable indicators in a baseline study before the start of the project.
- The first thing to measure is output, a thing of short-term practical results (in a maths project in a school, this would count the number of times the project was active).
- Then we look at outcomes, where medium-term results are assessed (the maths project has become better managed over time).
- Lastly, we look at impact (young people have achieved greater independence through increased knowledge in maths).
As we can see, measuring “impact” is the most difficult of all as this usually refers to a concept and should therefore have pre-set indicators to help us to assess effectiveness.
For example, if the proposed impact of a project is to “improve the health of the nation”, then we need to have agreed upfront how this is to be practically measured. Would it be through the number of health care facilities that exist, or through their state of equipment, or by the number of people using such facilities, or even through the number of people who do not need to use such facilities?
Whatever the indicators are, these need to be agreed before the project is carried out and then a baseline for this criteria needs to be established.
Four questions of evaluation:
The following four questions help us to evaluate project effectiveness:
- How relevant? (This refers to the project meeting the needs of all its identified stakeholders.)
- How effective? (Here we look at the project’s effectiveness in solving the practical challenge it is tackling.)
- How efficient? (This primarily refers to project systems – how relevant and useful are the project’s ways of working to solving the societal challenge it is set up to deal with?)
- How sustainable? (Here we look at sustainability in its broader sense: this may refer to a project’s operations being run efficiently; to its financial management; and to the project being focussed on actions relevant to its mandate.)