Loneliness evaluation: design and data gathering

Members of the Tackling Loneliness Hub recently came together for a workshop on loneliness evaluation design and data gathering, the first in a two-part workshop series looking at evaluation. This blog provides a brief overview of what we covered.

Getting started with evaluation

The following resources were shared as useful starting points for developing a theory of change and planning your own evaluation: 

• A brief guide to measuring loneliness (What Works Centre for Wellbeing) 
• Measuring your impact on loneliness in later life (Campaign to End Loneliness)
• Planning your evaluation (What Works Centre for Wellbeing)
• Wellbeing Measures Bank (What Works Centre for Wellbeing)
• Identify the difference you want to make (National Council for Voluntary Organisations) 
• Theory of change in ten steps (Think NPC)

Evaluation design and methods 

Helen MacIntyre, Head of Evidence at the Campaign to End Loneliness, talked us through the process of gathering data for qualitative, quantitative and mixed method approaches. 

Helen also outlined various measurement tools which can be used when evaluating your services, all of which are referenced in A brief guide to measuring loneliness: 

• UCLA Loneliness Scale
• The De Jong Gierveld Loneliness Scale 
• The Campaign to End Loneliness Measurement Tool

We also discussed some ethical and practical considerations when undertaking your own evaluation, notably: 

• Gain consent from participants from the outset 
• Anonymise personal data and quotes 
• Establish a sampling framework to capture a range of individual experiences 
• Plan how you will ask questions sensitively and anticipate what to do if someone becomes upset 

If you’re a member of the Hub, visit the Loneliness Evaluation group to view a recording of Helen’s presentation. 

A case study from Time to Shine

During the workshop, we also heard from Professor Andrea Wigfield (Sheffield Hallam University) and Jessica Duffy (Leeds Older People’s Forum) on the ‘test and learn’ evaluation approach for the Time to Shine Leeds project. 

The evaluation process consisted of questionnaires, interviews and focus groups, case studies, and volunteer listeners capturing the stories of older people. Key factors for the overall success of the project, as well as some challenges when undertaking the evaluation, were also shared. Some of the challenges included: 

• The appropriateness of national evaluation measures for the local community
• The completion of evaluation paperwork acting as a barrier to engagement
• Identifying individuals for interviews and focus groups
• Standardised validated measures of loneliness not providing a full picture, as only in talking to someone might you discover that they are feeling lonely

If you’re a member of the Tackling Loneliness Hub, visit the Loneliness Evaluation group to view a recording of Professor Andrea Wigfield’s presentation. 

Practical tools to support your evaluation 

Following our breakout group activity, the discussion continued on the Hub and lots of useful resources and tools were shared: 

• Time to Shine local evaluation reports – maps out the evaluation process and methods for Time to Shine
• Time to Shine case study template – includes questions to capture participants’ experiences and outcomes
• Time to Shine test and learn template – to help capture learnings from evaluation 
• A range of evaluation case studies including: 
   – Sound Creators (UK Youth) 
   – Community Webs (Bristol Ageing Better)
   – Community Connectors (British Red Cross)