Impact Measurement – The Perennial Challenge
Impact measurement is one of those topics that everyone in the charity sector agrees is important, yet no one has quite cracked. It’s complex, time-consuming, and let’s be honest often frustrating. So, at the Tech for Good Summit 2024, a panel of experts came together to unpack the challenges and explore potential solutions.
Measuring What Matters
Professor Daniel King (Nottingham Trent University) led the conversation, setting the stage with a fundamental question: Are we measuring the right things? Too often, charities are required to report on predefined metrics that don’t fully capture the true impact of their work. And, as Tris Lumley (New Philanthropy Capital) pointed out, there’s a tension between proving impact to funders and actually using data to improve services.
This isn’t just an academic debate. Alex Hutchison (Data for Children Collaborative) shared how her team helps organisations build meaningful measurement frameworks, ones that go beyond ticking boxes to tell real stories of change. Meanwhile, Fiona Stewart-Knight (Glasgow Caledonian University) highlighted how a lack of resources and skills can leave many third-sector organisations struggling to even start the process.
Data Overload, or Data Drought?
One major challenge is the sheer volume of data charities are expected to collect. The panel agreed that asking charities to submit reports to yet another system is unlikely to work. Instead, the focus should be on harvesting existing data with less overhead. Tris suggested looking at how sectors like criminal justice have created shared data labs, where insights can be drawn from centralised sources rather than requiring every organisation to start from scratch.
But even when charities have data, making sense of it is another challenge. Alex described how her organisation uses the theory of change models to structure impact measurement, breaking it down into categories like behaviour change, organisational change, and systemic change. This helps organisations understand not just whether they’re making a difference, but how and why.
The Competitive Edge of Collaboration
Fiona raised a critical issue: charities often compete for the same funding, which discourages data sharing. But in the tech sector, open-source collaboration is a norm, so why can’t the third sector adopt a similar approach? The panel explored ways to create shared measurement frameworks that allow charities to work together rather than duplicating efforts.
This led to an interesting discussion on whether impact measurement should be seen as a burden or an opportunity. When done well, it can drive better decision-making, unlock funding, and demonstrate value. But when done badly, when it’s just another bureaucratic hoop to jump through, it drains already stretched resources.
Where Do We Go from Here?
So, what’s the solution? The panel didn’t pretend there was a silver bullet, but they agreed on some key steps:
Simplify impact measurement. Funders should work with charities to develop realistic reporting expectations that balance rigour with practicality.
Invest in shared learning. Community-led data hubs or knowledge-sharing platforms could help reduce duplication and improve sector-wide understanding.
Leverage technology wisely. The rise of AI and automation presents an opportunity to reduce the burden of data collection and analysis.
Focus on what really matters. Charities should push back against measuring for the sake of it and focus on data that genuinely informs and improves their work.
As Daniel King wrapped up the session, he left the audience with a challenge: How can we make impact measurement work for everyone—the charities, the funders, and most importantly, the people they serve? It’s a question that will continue to shape conversations in the sector, but one thing is clear—change is needed, and collaboration is key.
Stay tuned as the Scottish Tech Army and the Tech for Good Alliance continue to explore ways to make impact measurement meaningful, practical, and accessible to all.
