Comparative Judgement: How do young people, parents and social workers assess the quality of social work practice?

Overview

The study produced evidence about what parents, young people and social workers think good practice is and how this compares to an established rating system. It involved asking people with lived experience of social services to listen to examples of real practice and share their views. It is the first study in the world to put the views of those receiving social work services at the heart of our understanding of good practice in this way. The findings will be widely shared and will feed into policy and practice developments in Wales and beyond, for instance shaping social work education, practice frameworks and future research.

Research Questions 

1. Do the groups identify similar or different factors in making decisions? 
2. How consistent are parents, young people and workers (within groups) in their judgements on the quality of social work practice? 
3. What is the level of agreement between groups and with expert coding for quality of practice? 
4. What does each group think is important in defining quality of practice? 

Activities and Methods

We gathered recordings of 17 family visits from three Welsh local authorities. The research team anonymised the recordings by removing names of people and places and changing the pitch of the recording to protect the identity of the participants.

We conducted two types of analysis: we applied an existing coding scheme known as SWIM (Social Work and Interviewing Motivationally) and used the comparative judgment method. The research team applied the SWIM coding scheme to evaluate the quality of social work in the recordings from an academic perspective.

Comparative judgment is a method that has been used extensively in the field of education. We used the comparative judgement method to gather views from groups of parents, care-experienced young people and social work practitioners – the assessors. We recruited 40 assessors in total. The assessors listened to pairs of recordings, telling us which recording was a better example of social work practice and why. Each assessor made multiple comparisons, until they listened to all recordings at least once. In total, the assessors made 573 comparisons and listened to 22,000 minutes of audio.

We have compared the rankings of the three groups with each other as well as SWIM through statistical analysis. We are also conducting qualitative analysis on the contextual information gathered for each recording and the assessor rationales. We ran collaborative sense-making activities with each group of assessors and our Local Authority partners to explore and refine our initial findings.

Findings

We are currently working on finalising our findings. We will update this page with publications as they become available.

You can read more about the progress of this project in our recent article.