Loneliness evaluation: Design and data gathering

Members of the Tackling Loneliness Hub recently came together for a workshop on loneliness evaluation design and data gathering, the first in a two-part workshop series looking at evaluation. Here we capture the key insights and learning from what we discussed.

Evaluation design and methods

During the workshop, we discussed the process of gathering data for qualitative, quantitative and mixed method approaches. There are various measurement tools which can be used when evaluating your services, all of which are referenced in A brief guide to measuring loneliness. These include the UCLA Loneliness Scale, The De Jong Gierveld Loneliness Scale, and the Campaign to End Loneliness Measurement Tool.

We also discussed some ethical and practical considerations when undertaking your own evaluation:

• Gain consent from participants from the outset.
  Consent should be informed, meaning participants are fully aware of the nature of the research, what it involves, any potential risks, and their right to withdraw at any point.
• Anonymise personal data and quotes.
  This means removing or altering any information that could be used directly or indirectly to identify an individual.
• Establish a sampling framework to capture a range of individual experiences.
  This involves strategically selecting participants to ensure that the sample reflects the broader population’s demographics and experiences relevant to the research question.
• Plan how you will ask questions sensitively and anticipate what to do if someone becomes upset.
  Phrase questions in a non-leading, non-judgmental way and create a safe, respectful environment for participants to share their experiences.

A case study from Time to Shine

We also heard from Professor Andrea Wigfield (Sheffield Hallam University) and Jessica Duffy (Leeds Older People’s Forum) on the ‘test and learn’ evaluation approach for the Time to Shine Leeds project. You can view a recording of Professor Andrea Wigfield’s presentation in the Loneliness Evaluation Group.

Their evaluation process consisted of questionnaires, interviews and focus groups, case studies, and volunteer listeners capturing the stories of older people. Key factors for the overall success of the project, as well as some challenges when undertaking the evaluation, were also shared. Some of the challenges included:

• The appropriateness of national evaluation measures for the local community.
• The completion of evaluation paperwork acting as a barrier to engagement.
• Identifying individuals for interviews and focus groups.
• Standardised validated measures of loneliness not providing a full picture, as only in talking to someone might you discover that they are feeling lonely.

Practical tools to support your evaluation

Following our breakout group activity, the discussion continued on the Hub and lots of useful resources and guidance were shared:

• Developing a Theory of Change in 10 Steps from Think NPC.
• Using open questions in your evaluation to help you capture qualitative data in your surveys.
• Time to Shine local evaluation reports which map out the projects’ evaluation process and methods.
• A range of evaluation case studies including:
  • Sound Creators (UK Youth)
  • Community Webs (Bristol Ageing Better)
  • Community Connectors (British Red Cross)