Measuring Helpline Outcomes and Demonstrating Impact
There is a growing pressure for charities to effectively measure and evaluate their outcomes – not only to engage funders, but also to improve their operations and motivate staff.
The most popular seminar session at this year’s Annual Conference was our own seminar, hosted by Mega Arumugam, which explored how helplines can effectively measure the outcomes of their service and demonstrate its impact.
Why focus on helpline impact?
The impact you have is the reason that your organisation exists and should be prioritised above everything else. Impact practice is not an afterthought – it is integral to the delivery of the service. It is all the things that you do to put impact into practice.
As a helpline, you should be thinking about the long-term change ‘impact’ that you want your service or programme to have and the shorter-term changes ‘outcomes’ that are needed to create that longer-term change.
Know and understand your helpline data
Different measures call for different types of data whether it’s quantitative, such as the number of calls, or qualitative like how people report they are feeling at the end of a call.
Mega shared different types of helpline data that could help your organisation to measure the outcomes and demonstrate the impact of your service.
- User data - The characteristics of the people you are reaching; gender, location, ethnicity
- Engagement data - How service users are using your service, and the extent to which they use it
- Feedback data - What people think about the service
- Outcomes data - The short-term changes, benefits or assets people have got from the service
- Impact data -The long-term difference that has resulted from the service
It is important to put people at the heart of your impact practice.
Create opportunities for users to influence practice across the entire process of planning, delivering and evaluating a charities’ activities.
Providing demand information is not enough. Funders want to see the impact of the service, for example has your helpline stopped a problem getting worse? Are you supporting someone on a journey that may end up with them having better outcomes?
Data collection challenges for helplines
One of the big challenges for helplines is around collecting feedback, outcomes and impact data.
There was a lot of discussion during the session from the helplines taking part, about the barriers they face which included:
- One-off nature of calls
- Difficulty knowing what happened to the caller (lack of on-going feedback)
- Callers that disengage (for e.g. mental health erratic behaviour)
- Difficulty asking for feedback or a survey after a distressed call
- It feels needy from a helpline point of view to ask for feedback
- Lone worker shift
- Not safe to call back
- Time limits (how long do you monitor the data)
- There can be multiple outcomes so how do you judge if the call was a success?
- For a service that deals with emotional callers, compliments are the only way of collecting feedback
Key questions raised during the seminar were, given these challenges, how do we measure distance travelled by the service-user? How do we know it was our intervention that made the difference? And, how do we develop effective measuring tools?
Helplines Partnership’s Masterclass training provides an insight into mapping the journey and how that essentially provides helplines with a means to accessing some of these difficult outcomes and possibly impact.
Plan and focus
To be able to effectively, and usefully, capture data that will allow you to demonstrate your helpline’s impacts and outcomes you need to think about:
- Your helpline, call and caller data
- What outcomes you may be achieving now
- What outcomes you might like to achieve in the future
- How you are going to measure impact
- What research techniques will allow you to show others the difference that you are making
Mega stressed the importance of asking yourself, “how do we understand if we have made a difference and use that as your basis for measuring impact? Once you know that then focus on how you are going to demonstrate it well and robustly.
"We acknowledge that helplines function in a complex way and it is unlikely that you will strictly follow a step-by-step process. The Code of Good Impact Practice identifies four main areas of activities that you should undertake at some point to focus on your impact:
1. Plan the impact you want to have
2. Collect the appropriate forms of data on your impact
3. Assess your impact
4. Review your findings and learn from it
"It’s much better to measure one thing and do it well than have a whole range of measures that you do badly.”
If you would like to learn more about impact practice and measuring outcomes for helplines our CPD accredited Masterclass explores how to evaluate and measure outcomes in different ways within the context of a helpline service.
Using case studies, theoretical frameworks and examining current practices, participants will gain a greater understanding of the potential scope and impact effective outcome monitoring has.