The small-scale pilot found that Resilient Families Plus did not have an additional impact on students’ academic self-concept in Maths and English and in academic resilience. Evidence suggests that students who initially reported lower levels of academic self-concept and academic resilience improved slightly after being involved in the program. These findings need to be treated with caution due to the small number of schools involved, inconsistent and incomplete program delivery, the use of other well-being programs that schools were running concurrently (including a Department-sponsored well-being program with similar features), and the lack of a control group to compare gains made over time. 

Program fidelity was low as schools truncated the student curriculum and only implemented two or three sessions (estimated to be 15 – 20%) of the whole program and adapted the program (e.g., changing the sequence of activities and completion dates) based on their needs, making it difficult to determine if all program elements were delivered or effective. Two of the parent components of the program were not delivered by the program developer and the two new components of Resilient Families Plus were only completed after students sat the post-intervention survey. Schools reported difficulties delivering the full program as they found it time and resource intensive and were already implementing alternative curricula. Parental participation in the parent activities was very low (8% of participating students or 2% across Year 8 enrolment numbers from the two participating schools) as teachers reported challenges in organising these activities and securing parents’ time and involvement. 

For these reasons, the program in its current form is not yet feasible for implementation in schools. The program developer identified barriers such as family stress, work commitments, English as an additional language and schools’ own challenges with parent engagement that may inhibit parents’ participation. To further improve the impact of this component, it would be valuable to ensure that schools and teachers are supported in securing greater parental participation to engage in student learning, especially in responding to issues that these communities may face.

Although the program is affordable, it would not be suitable for a randomised controlled trial (RCT) measuring its impact on student outcomes. There are two main reasons for this. First, the intervention needs to address its adherence to the prescribed model and barriers to faithful implementation, including schools’ interest using the program which makes it difficult to design an effective RCT. Second, the intervention allows schools to adapt its curriculum and activities to their needs and existing curriculum. This flexibility and the way it operates needs to be further assessed prior to a trial.

Key conclusions

1. Impact of the intervention: Students reporting at baseline low levels of academic self-concept and academic resilience may benefit from Resilience Families Plus more than students who initially reported higher levels on these academic precursor measures. This is because students with lower baseline measures of academic self-concept and academic resilience showed small increases in these measures after being involved in the program. However, these increases were not statistically significant so need to be treated with caution. The small number of schools involved in the project and students who completed both the pre- and post-surveys (2 schools, 34 students, 7 parents), incomplete program delivery at the time students sat their post-intervention survey, other well-being programs that schools were running concurrently and the lack of a control group to compare gains made over time. 
2. Pilot challenges: There were significant challenges recruiting schools to participate in this evaluation. Neither the Victorian Education Department or research funders provided assistance in recruiting schools and other well-being programs that schools were already running (e.g. one of the two schools was implementing Respectful Relationships¹, a Department-sponsored well-being program with similar features), may have impacted schools’ willingness to adopt Resilient Families Plus. A parallel research study undertaken by the program developer² at the same time meant schools had to disseminate information and gather student consent forms for two separate projects may have delayed return rates, resulting in lower student participation numbers. The program developer’s research study was prioritised over those of the evaluation. For example, students had to complete the program developer’s survey questions before the evaluation’s in the same sitting, which might explain the lower survey completion rates than those of the concurrently run study.
3. Implementation issues: Despite schools and parents acknowledging its value, schools faced challenges implementing the full 10-weeks of the student curriculum by the time of the post-test. For example, one school only selected to use 3 of the 10 sessions and then complemented these with other program content. Schools also reported changing the timing and sequencing of the activities. Additionally, schools did not implement the full complement of the parent components and when they did, this was out of the recommended sequence. Schools valued the flexibility the program allowed given they could select how many sessions to include as well as the timing and sequencing of these, however this resulted in poor fidelity to the prescribed program.
4. Feasibility: Schools were reluctant to implement all the student curriculum components of the program and instead chose to supplement with alternative curriculum as they deemed the student curriculum as resource and time intensive. While acknowledging the program’s school-based parenting sessions, schools questioned its feasibility given challenges in managing these activities, the low interest and involvement from parents, and the time and involvement required from families and schools. The program developers identified the barriers to implementing the parent components were related to family stress, work commitments, English as a second language and schools’ challenges with engaging parents.
   
5. Readiness for trial: Although the program is affordable, lack of adherence to the prescribed model and barriers to faithful implementation need to be addressed prior to any future trial. Schools’ interest in using the program and being involved in a trial would also need to be determined.