Involving the Community in Evaluation
Unexpected opportunities can give us a chance to brush up on our skills, and even learn some new ones. Alan Mackie reflects on how saying “yes” to a last-minute request gave him the opportunity to think about community engagement, and what it takes to create a successful project.
Last November I had the opportunity to get involved with a community engagement exercise being run by a community foundation here in Atlanta.
The initial request seemed very doable: Could you help to set up some community focus groups and a questionnaire for a community survey?
I often find it difficult to say no, and knowing the CEO, I quickly agreed.
Then came the hard part – the timescale: Could you do complete it in two weeks? Over the Thanksgiving weekend?
We looked each other in the eye. She smiled broadly, and I blinked first: No problem! Consider it done!
In truth, despite the fact that the work had to be completed over the Thanksgiving weekend, I was glad for an opportunity to get back in the field, and soon we were setting up the focus groups and creating the questionnaire.
Although it was a relatively small piece of work, I only had a short time in which to do it. So I was grateful for the support I received from Kasra and my London-based colleagues.
As well as the satisfaction I got from meeting the challenge and completing the work, I also gained some valuable insights into community-based research.
Initial Steps
Setting up the questionnaire survey was relatively straightforward. My colleagues and I began by reviewing open sources of validated questionnaires and found questions relevant to what we wanted to find out. Once the client had approved the questions Kasra uploaded them to our online survey.
We wanted as many responses as possible, to get a good sample of the residents’ views, and we used several methods to collect responses. Making it as easy as possible to respond (or, reducing what the marketers call “friction” – the obstacles to a transaction) would hopefully increase the number of responses to the survey, so we created a QR code that allowed participants to complete the questionnaire on their smartphones just by scanning the code.
The foundation’s staff also canvassed the community directly: door-hangers printed with the QR code were placed on homes in the community, to invite households to participate. To ensure the survey was inclusive paper versions were available, allowing people without internet access to participate.
With Kasra working on the survey the next step was to organise the focus groups. Working with our client I drew up the focus group topics. Topic guides help to prompt and guide the discussion, ensuring that participants can participate fully.
We decided that the focus groups would be moderated by the foundation’s staff, as they know the community intimately and are trusted by the residents. I offered the moderators initial training on good practice in focus group conduct, especially the importance of listening and resisting the temptation to break any silence.
Getting Results
The first two focus groups were held in person and additional focus groups were held online to ensure the broadest participation. I attended one in-person group and an online group, helping to troubleshoot any potential problems and to audit the impartiality of the process. I didn’t have anything to worry about – the staff members were true facilitators, allowing participants to give their views without hindrance.
The focus groups and the survey together generated an excellent sample of the views of the community, and once they were completed Kasra and I helped the client to compile the quantitative and qualitative data and identify the main story.
Lessons Learned
Reflecting on the project I can see several lessons to take from it:
- Community-based work requires the trust of residents. This can take time to achieve, but using the foundation’s staff meant we could capitalize on their trusted relationships without compromising objectivity or impartiality.
- Community-based work takes time. I was impressed by the time and patience the staff took when moderating the focus groups. No one was working against the clock; every participant had the opportunity to contribute, and their contributions were valued.
- Meeting the challenges of the work requires a clear understanding of the client’s and contractor’s respective roles and responsibilities. Even though it was a fairly small project it was made much easier by having a clear understanding of what the client required.
- But flexibility is also important. The ability to extemporize, and being prepared to always do more to ensure the work is done, means that you can always make changes on the fly and adapt to changing circumstances or unforeseen obstacles.
Although it was a short commission, I believe that everyone involved in the project felt that it was successful. We got a good understanding of the residents’ views and the foundation had the evidence it needed to make the appropriate responses.
Even after nearly thirty years of working on data collection and data analysis this short project gave me an opportunity to learn something new.
We don’t always want to say “yes” to extra work or unexpected requests, but new opportunities don’t always announce themselves as such and there’s always something we can learn from a new challenge. There’s always more to learn.
What Do You Want to Find Out?
If you need to undertake a rapid consultation with your stakeholders, or if you want to explore ways of canvassing opinions around a subject, then please contact us. Get the Data can help with collecting data and opinions through a variety of options. We can set you up with a questionnaire survey, and find ways of utilizing your resources to complete focus groups or interviews.
