Our clients work with people to address an important issue in their life and therefore have to engage with sensitive issues in their beneficiaries lives, such as their financial situations, the strength of their relationships with friends and family, and how they have been feeling generally. If this describes your organisation and you want to measure your impact, then it’s a given that you are going to have to ask questions (in say a questionnaire) that may seem prying and beyond your role. We often develop a theory of change with clients that they are happy with, and they believe we have identified the right outcomes to measure, but when we present how to measure these outcomes they can become a little uncomfortable at the change social impact measurement represents. They might be concerned that staff or volunteers will feel uncomfortable asking these questions and not do it correctly or asking such questions will interfere in their relationships with their clients.
These are important concerns and I wanted to share some of my experience on this issue so you can measure your impact confidently.
- Are they sensitive?
Often, we perceive some survey questions as sensitive and therefore difficult to ask, but I generally find responders are not as worried. I ask myself “would I mind being asked this question?” and about 90% of the time I find I am OK with it. I could be less sensitive than the next person, but I see no reason why I am terribly different, and it is an important self-check that I perform. It’s always important to pilot any questionnaire and ask for feedback on any of the potentially sensitive questions and how they can be best handled. By being concerned about perceived sensitivity, we can actually exclude the voices of the people we want to help.
- Involvement in the development
It might not always be possible to involve staff and volunteers in the development of questions – I haven’t been able to as much as I would like to during the pandemic – but their involvement can allay any fears about questions to be asked. Staff and volunteers can help put the right tone on the questions, so they fit with your organisation’s culture and how it engages with beneficiaries. Their involvement also means they can understand why certain questions are asked and their value.
- Staff and volunteers already do it
It might not be in the obvious at first, but staff and volunteers are probably talking to beneficiaries about these sensitive issues already. The reason you need to measure them is that you want to help, and the evidence base suggests you can successfully. If staff and volunteers were not discussing these sensitive issues, then you cannot expect to improve them. Obviously, staff and volunteers have learnt to raise these issues tactfully and this experience means staff and volunteers can help get the tone right.
- Train hearts as well as minds
Staff should be trained on how to administer the questionnaire, but they should also be primed on why these questions are being asked, and why they are important and of value to the organisation.
- Administration is important
How a questionnaire is administered can reduce any fears about asking a question and prevent respondents from feeling pried upon. Using self-completion questionnaires can reduce any anxiety associated with a question, especially if the questionnaire design prevents staff from reviewing a response in front of your clients. The questionnaire can also be clear to a responder that they do not have to answer every question and incorporating a don’t know response in every question is good practice.
- Use tried and tested questions
There are many psychometric scales and national surveys that have high-quality questions, that have been rigorously tested and verified, and are considered ethically sound. Using these questions means you can be assured that you are not causing offence.
- Show the value
People are keener to administer and complete questionnaires when they understand their value. That’s why I always recommend that you regularly communicate your results are to staff, volunteers and beneficiaries.
Overall, we must all remember that going from not measuring your impact to doing so is a moment of important change and becoming comfortable with asking new questions and using new measures can take a little time. Once the culture change happens, then organisations can start to use evidence confidently to improve their social impact.