Betting on the right horse: using behaviours to predict impact

The Just Cause Excellence Framework assesses non-profits through a series of lenses, looking at the visible results (which are measured in several ways) as well as the underlying character of an organisation. This approach is based on our fundamental conviction that it is not possible to compare organisations based only on simple measures of impact or return on investment.

This post explains why we believe that quantified impact measures are only part of the picture.

Existing efforts to monitor and benchmark non-profit organisations often focus on searching for a quantifiable measure to summarise impact. For example, where non-profits are applying for institutional funding, this measure is likely to be a small set of pre-agreed, quantitative Key Performance Indicators (KPIs), such as the number of beneficiaries served within a specified time period. Another increasingly popular measure is an assessment of the value for money or return on investment of the programme, whereby social or environmental outcomes are translated into a dollar value and set against the cost of running the programme.

If conducted rigorously, such quantitative assessments can be extremely useful both for donors and the organisation’s own management team. They can act as basis for tracking performance and continuous improvement. It is indeed tempting for many donors and others to look to benchmark non-profits using such metrics.

However, Just Cause believes that neither KPIs nor value for money assessments can work as a standalone measure of performance.

There are three main problems with this simple quantified approach:

1. Rigour

In reality, it is extremely difficult to establish rigorous, quantified measures of impact. Simply measuring raw outcome indicators (such as the ‘number of lives improved’) quickly becomes mired by data collection issues and the problem of ascertaining ‘attribution’ – in other words, what proportion of the outcome can be reasonably credited to the intervention at hand, as opposed to other factors.

Value for money approaches attempt to build one step further on this house of cards by translating social outcomes into equivalent dollar value terms. They do this by identifying proxy measures in various ways. However, the decision of what proxy measures to use and how, is fraught with subjective decision-making that can deeply undermine the rigour of the approach.

We believe that quantified impact measures are useful as an indicative measure of performance, but that it can be risky and misleading to attach too much weight to such measures.

2. Trust and empowerment

Regulators, commissioners and donors across the world are understandably strongly concerned that their money should be used efficiently when allocated to non-profits. As such, they regularly employ quantified impact measures to monitor that performance is ‘on track’.

However, the risk of relying overly on strict performance monitoring that it can neutralise the non-profit organisation at its core. By this, we mean to draw attention back to the one of the basic reasons why non-profit organisations exist in the first place. They are mission driven: established by a group of people acting on the motive of not seeking personal profit, but of achieving a social goal.

This means that even if the donor cannot accurately measure performance (which is likely), the non-profit would still be expected to ‘do the right thing’ and not to take advantage of the situation. This is because the goals of the non-profit and the goals of the donor are aligned. In reality, the non-profit may in fact perform badly – for example because of inefficiency or corruption – and the donor must monitor this. But we believe that over-emphasis on top-down tracking of results can disempower and demotivate a non-profit by undermining the basic essence of its constitution.

3. Creativity and future uncertainty

There is a rich literature and body of experience to suggest that social change rarely emerges in predictable and linear ways. An individual’s well-being will improve because a combination of foreseen and unforeseen factors aligns to reach a tipping point.

As such, programmes that aim to bring about social change face a fundamental challenge when it comes to planning. Although it is possible (and indeed desirable) to set clear goals at the start of a programme, it is not possible to make plans that are precisely defined, and organisations must always be ready to learn from and adapt to unforeseen developments.

For this reason, we believe that the future success of an organisation can be predicated less by its results in the past and more by its character: Does it work closely with others? Does it constantly seek to learn and to iterate its approach? Does it really listen to its customers? Does it empower staff to develop and test out new ideas?

Multiple lenses on excellence

Our approach measures the current and future health of an organisation through multiple lenses, assessing both results and character. As standalone measures, none of the lenses that we adopt would be sufficiently rigorous or telling, but by combining multiple measures, we are able to build up a picture of an organisation that we believe gives a stronger (even if not entirely robust) assessment of ‘excellence’.

If you would like to get involved in the development of our framework, or to share your own opinions on ‘what good looks like’ and ‘how to measure it’ then please don’t hesitate to contact us. We would love to hear from you!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s