What we’ve learnt about the outcomes of our programmes

In 2019-20 we conducted a year long study in conjunction with the University of Sussex evaluating the extent to which the Theory of Change behind our ‘Face It’ programme highlighted relevant outcomes and whether our programme was contributing to positive changes in targeted outcomes.

On both counts the findings of the study indicated that our Theory of Change highlighted the relevant outcomes while both quantitative and qualitative insights pointed to positive changes in each of the foundational and core outcomes we seek to achieve.

Nearly 70% of our participants report improved levels of social and emotional wellbeing post-programme while elsewhere 76% report using the positive coping skills learned on our programmes to help them better self-regulate their behaviour. The full report is available on our website, here. In this blog, we share key highlights from the report and what we have learnt about the outcomes of our programme has on the young people we work with.

Our Face It programme

Many of the young people we support have experienced 4 or more adverse childhood experiences, making them 32 times more likely to develop social, emotional or behavioural issues.  Without the right support, these issues can lead to disruptive behaviour in schools, putting them at risk of school exclusion which very often leads to a lifetime of unfulfilled potential (only 1% of excluded children achieve 5 GCSEs, they are 4 times more likely to be imprisoned as adults and 99 times more likely to become homeless). Developing their social and emotional skills not only buffers against these negative effects, it equips them with the skills relevant for immediate and future success. This is especially critical for the most disadvantaged young people who are at a skills-deficit to begin with.

Our ‘Face It’ programme is an intensive therapeutic programme developed to support the wellbeing and social and emotional skills of young people with complex needs. We work with disadvantaged young people (aged 11-18) who have been excluded from school or are involved with the criminal justice system. The programme is delivered experientially using a mixture of group activities & 1-to-1 reflection sessions, covering 10 core modules to first raise self-awareness & then build wellbeing interactively through creative techniques like storytelling, art, debating & role-play.

While the primary objective of our programme is the development of social and emotional skills & wellbeing, we specifically target positive changes in the following four areas: emotional self-regulation; coping skills, resilience, social and emotional wellbeing.

 

Our quantitative impact

Social and emotional wellbeing: 

68.2 percent of participants report increases in wellbeing post programme. To determine the extent to which these changes were a result of our work with young people, we ran a paired t-test to compare our pre and post-programme results. The t-test results indicated that the changes to our participants’ wellbeing were statistically significant – in other words, the changes did not occur by chance. Overall, participant wellbeing levels shifted from 3 points below national average levels pre programme, to just under 1 point below post programme. 

Qualitatively, participants also reported changes in relationships, confidence, optimism, feelings of calm, ability to function and think clearly – all aspects of social and emotional wellbeing.

Emotional self-regulation:

55 percent of participants reported an increase in their use of positive regulation strategies (by 7 percent overall), and some participants (40 percent) reported a decrease in negative regulation. 

Qualitative findings reinforced the relevance and importance of this outcome for participants: 52 percent of respondents reported positive changes in this area. This outcome appears to be highly relevant to the theory of change.

Coping skills:

Under half of participants (43 percent) reported positive changes in this area according to quantitative survey results. Elsewhere 76% of our participants reported using coping skills learned on our programmes once they finished working with us. 

The theme of coping skills came through strongly in qualitative interviews, with 36 percent of respondents giving unprompted examples of using new coping skills. 

Resilience:

Quantitative survey findings suggest that overall, most participants are seeing improvements in this area (62 percent). 

Findings specifically highlight the power of the social element of Face It, including feelings of belonging, learning how to relate to others, empathy, tolerance and communication. These themes come through strongly in qualitative data. This was unprompted with as many as 55 percent of participants highlighting some sort of social skill as key learning.

Key learnings

Both research and our own experience have shown us that young people have a strong appetite for support with their social and emotional skills and wellbeing. 

According to research, during secondary school, pupils become less satisfied with the provision of life skills education as they progress, while simultaneously placing a greater value on those skills. Our experience also seems to support this. Participants engaged well with the programme and appreciated being part of it. This is highlighted by the proportion of participants who asked for a longer programme and who expressed the (unprompted)  desire for a reunion (20%). 

Feedback from qualitative interviews was also overwhelmingly positive. While this could be a product of social desirability bias, the combination of survey results and interview findings indicates that, overall, the Face It programme is successful in bringing about positive changes among many participants. 

Group-work:

As above, this seems to be a powerful element of the programme. In qualitative interviews, most participants (86%) reported having positive experiences of sharing as part of a group and many (55%) reported improved social skills as a key benefit of the programme. 

Games, movement and interactivity:

This was mentioned repeatedly as a part of the programme that is enjoyed the most by participants. 

Teacher involvement:

A small number of participants commented on the benefits they gained from having their teachers as part of the programme. Young people reported feeling better connected with their teachers. Participants on our programmes are given the opportunity to tell their story, be heard and listened to as they ‘verbalise ‘what happened’ to bring them to this point. This helps them see the impact their behaviour has on the ‘other’ both in the group and more widely with their families and communities. Exploring their own story with their peers and their teachers allows them to think about alternative patterns of behaviour in pursuit of healing themselves and their relationships and often builds social connection and feelings of belonging. 

There are additional known benefits from having teachers participate in programmes (such as the ability of the teacher to sustain strategies learned through Face It for continued support). 

We plan to build on the lessons learnt from this evaluation by working with ImpactEd, who are experts in evaluation in the education sector and helped us develop our Theory of Change, as an external evaluator. This external evaluation will give us a robust and externally validated view on the efficacy of our intervention by providing a control group to compare to and supporting us to monitor school-level data on attendance and attainment.

Leave a reply

Your email address will not be published.
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>