Programme Evaluation Partnership
The University of Essex (UofE) Research and Evaluation team have submitted the 2020/2021 Quarter 4 Project Evaluation reports to ABSS. Following the Partnership Board’s decision to offer participants an incentive for participating in qualitative evaluation interviews, there was a marked improvement in the response rate of participants.

The University team are commissioned to recruit and train a team of Peer Researchers to support programme evaluation and the team have worked closely with both the ABSS Work Skills project and SAVS team to recruit parents to the Peer Researcher training programme. The first taster sessions took place in May and June 2021 and proved to be very popular with participants. We are now exploring how we can develop this programme further, including whether we can offer an Open College Network (OCN) qualification in Peer Research.

The University have worked with Southend Borough Council on the outcomes and outputs dashboards and researchers have approached all local Delivery Partners to offer ‘in-house’ data collection and evaluation tools. A number of Delivery Partners are already using the survey software to conduct their evaluations, which are used as secondary data by the University research team. As a result of these conversations, the Specialist Home Visiting for Perinatal Mental Health team have agreed with the University to transfer their evaluations online. It is hoped that, with direct access to participant data, the University research teams will be able to conduct more rapid analysis of data.

Independent Programme-wide Summative Evaluation
Planning for the independent Summative Evaluation continues for RSM and their partners, the University of Essex. A panel of parents, Delivery Partners, ABSS core staff and other community organisations and individuals joined a Co-Design Group to help form the evaluation design and key questioning, providing RSM with some valuable insights and feedback on their ideas. Information about ABSS, its projects and Partners is being collated, with systems now in place to securely share data and potentially commercially sensitive documentation. RSM was given virtual tours of the ABSS Data Dashboards and the SmartSouthend website to introduce them to a range of data available for their use.

National Evaluation
A number of meetings has taken place between the new national evaluation team and representatives from the five ABS sites. The national evaluation team is led by NatCen Social Research alongside the University of Sussex, National Children’s Bureau, RSM, and Research in Practice. The NatCen team presented their plans for the evaluation at an introductory workshop before holding a series of workshops with each of the sites to begin the process of collecting information and refining their plans.

Outcomes Framework
The ABSS Insight and Analysis Group has been supporting the development of project-level outcomes dashboards, which will present data on progress by projects against their project-level outcomes. The group agreed criteria for which projects would receive these dashboards, based on the contract value, strategic importance and novel nature of the project design, and discussed common elements that could be compared across different projects. A test version of a Project Outcomes Dashboard is being developed for the Family Nurse Partnership (FNP) project and will be used as a model for future dashboards.

The group has also been reviewing the programme-wide Outcomes Framework. The measures listed against each outcome were reviewed and identified measures with missing data, particularly the EYSFP data which was not collected by schools in 2020 and will not be collected in 2021 due to the pandemic. The group are exploring using large population-level datasets developed for other studies, such as the Understanding Society study (led by the Economic and Social Research Council, and The Institute for Social and Economic Research (ISER) at the University of Essex) as this has been running for ten years and includes a module on child development. The team are now exploring whether this dataset has a large enough sample to provide useful insight at a local level. Other opportunities included adding perception-based questions to Southend Borough Council’s annual survey and adding a small number of specific questions to the evaluation surveys being planned by RSM for the Summative Evaluation. The group also discussed the need to develop defined outcomes and measures for Systems Change and Community Resilience, and this work is ongoing.

The ABSS Outcomes Framework is a fixed document that requires some degree of experience to navigate. There is an interactive Outcomes Framework, which charts progress against all measures, but again, this requires some time and experience to use. In order to present outcomes and measures in a more accessible way, the team are developing Stories of Impact for the ABSS Programme. These combine outcomes data with project data and participant experiences gathered from Formative Evaluation reports, accompanied by a narrative text summarising the insights identified, to provide a more rounded picture of impact at project, thematic or geographic levels.

Workforce Development
Clare Littleford has completed her conversations with ABSS staff, Delivery Partners, Parent Champions and other relevant individuals to map out current workforce development provision and to identify future priorities and opportunities. The next step is to synthesise the findings from these conversations and to develop a document outlining initial proposals for a refreshed Workforce Development Strategy. This will be produced in the coming weeks.

ABSS Research Bulletin
Following conversations with a number of Delivery Partners, the Research and Evaluation team is redeveloping the Research Bulletin into an online presence within the ABSS website. The exact content to be included is still being explored, but in addition to the summaries of research contained in the previous incarnation of the Research Bulletin, it may also include features focusing more on practice, or blog-style entries about topics of interest to Delivery Partners and Parent Champions. A pilot version of the type of content envisaged will be shared for feedback with Delivery Partners and Parent Champions in the Autumn of 2021.

Data Development and Analytics
The main focus of work by the Southend Borough Council Data Team during April 2021 was production of the Quarter 4 and annual TNLCF returns, including the socio-demographic and outcome measures for 2020-2021. The final circulation of PDF project dashboards was completed in May 2021 and access to the new online project dashboards has been enabled for all former recipients of the emailed documents. A demonstration of the new dashboards was undertaken over Teams with RSM to help with the Summative Evaluation using project data. Outcome measures were agreed for the Family Nurse Partnership project and an Excel mock-up of a future project outcomes dashboard, incorporating dummy FNP data and live ASQ data, was circulated and agreed. This will be the basis for a future online dashboard as soon as FNP are in a position to provide ‘real’ FNP data for the 2020-2021 year.

ABSS Research Methodology
A revised Theory of Change for the overall ABSS Programme has been developed and underwent an extensive process of review, including with the ABSS Project Management team, the University of Essex research Partners, with a group of Parent Champions, and through the Insight and Analysis Group. The final version was approved by the Partnership Board in June 2021. This Theory of Change sets out the underpinning theories and assumptions behind the ABSS Programme and will feed into work finalising outcomes around Systems Change and Community Resilience.

Work is also underway to revise and develop Theories of Change and Logic Models for individual projects. A contractor has been brought in to support this work, developing Theories of Change and Logic Models for six individual projects and through that developing a model that can be applied to future projects. This work will strengthen the theoretical approach taken to developing ABSS projects and services, and so will contribute to the evidence base that the ABSS Programme as a whole is developing.

The programme-level and project-level Theories of Change will be reviewed regularly to ensure that they remain live and fit with the current and future needs of the ABSS Programme and its legacy.

Leave a Reply

Your email address will not be published. Required fields are marked *