Is measuring economic impact right for your charity?

 Is measuring economic impact right for your charity?

ray_explores on Flickr under a CC license

by Jon Franklin, Chief Economist at Pro Bono Economics

Funders, philanthropists and trustees increasingly expect charities to provide evidence of their wider impact on society. But analysing the economic impact of your charity’s services is often easier said than done. Here is Pro Bono Economics’ overview of how to look realistically at what is feasible for your organisation, before investing in data collection and impact consultants.

What do we mean by wider economic impact?

There are a whole host of different ways to measure a charity’s impact. This can include:

  • case studies that capture the experiences of individual beneficiaries
  • qualitative focus groups that help to identify why and how an intervention works
  • or quantitative approaches that attempt to determine the impact of charities through changes in outcome measures.

Economic impact measurement is typically used to summarise the effect that a charitable intervention has in terms of pounds and pence. This is not because it is intrinsically a good thing to do – often organisations, including Pro Bono Economics, can feel deeply uncomfortable about reducing multifaceted changes in people’s lives to monetary values. But it acts as a common currency that people intuitively understand, and makes comparisons to costs simpler. For example, the ability to say that “every £1 invested in our services yields £4 in benefits to society” can help to influence some of the stakeholders in government and business that are hardest to convince.

All of these different approaches to measuring impact have an important part to play in articulating the value of a charity. But, faced with limited budgets, how should your charity choose which one they should invest in? When is it right to talk about your economic impact?

Start with the “why” and the “who”

The starting place for those in charge of charities should be to ask “why do we want to estimate our impact? Who are we trying to influence and what outcomes will those stakeholders be interested in? Will they be motivated by case studies, technical studies, or are they interested in savings expressed in pounds and pence?”

You can then start thinking about the outcomes your organisation delivers and whether these can be measured and explained in a way that will work for those stakeholders. For example, a charity focused on improving mental health through counselling support has several potential avenues for describing their impact. They can tell stories of people whose lives have been turned around. They can express how improvements in mental health reduces NHS usage, or explain the value of improvements in people’s quality of life from their services.

It may sound obvious, but if your key stakeholders aren’t that interested in economic impacts, then it is probably not worth investing the time and effort required to measure them!

What’s going to be feasible?

If you do want to measure your economic impact then there are two key questions that affect what’s possible:

  1. Can you identify what would have happened to your beneficiaries in the absence of your charity’s support?
  2. Is it possible to put a monetary value on the outcomes your charity delivers?

If the answer to both of these questions is “yes” then you’re likely to be in a great position to assess your economic impact. But if you are not sure, or the answer to either of these questions is “no”, then it is likely that, no matter how much time and effort you spend gathering data from your beneficiaries, you will not be able to make strong claims about your economic impact.

The first question typically requires a comparison to some kind of control group – a similar group of individuals to the ones you have worked with but that did not receive your support. This can be difficult for many charities due to barriers related to cost, logistics, and ethics. However, as more and more administrative data is made accessible to researchers, the options for finding some kind of control group using analytical methods are improving all the time. A good example of this is the Ministry of Justice Data Lab. This resource can help charities working with ex-offenders to compare re-offending rates between those that they have worked with and a separate group of individuals, with similar backgrounds and characteristics, who they have not engaged with.

The second question generally relies on evidence already available that can link your outcomes to benefits that can be expressed in monetary terms. This is straightforward for certain types of outcomes, such as improvements in academic attainment or supporting someone to enter employment. However, while new guidance from the Treasury on the valuation of “softer” socio-emotional outcomes via wellbeing measures offers great potential for many charities, there will always be some that will find this process difficult.

Conclusion

At Pro Bono Economics, we believe that providing evidence of economic impact is incredibly helpful for charities if they are well positioned to do so. A good example of this is our recent analysis for Magic Breakfast which had invested in the evidence required to make strong claims about its long-term economic impact.

However, we would also be the first people to flag that measuring economic impact is not right for every charity. Before setting off down the path of data collection and spending money on impact consultants, it is worth pausing to consider if an understanding of your economic impact really matters  to your stakeholders, and if the outcomes your organisation influences lend themselves well to measuring your economic impact.

If you’d like to get in touch with Pro Bono Economics, you can email them or visit their website. And read their new report on the need to improve data in the sector, published as part of the Law Family Commission on Civil Society.

You can also watch Jon’s talk from the Data4Good Festival below.

To join the Data4Good Festival mailing list and get updates about more blogs and news, sign up here.