Evidence for Learning:&npbs;Thinking Maths

Thinking Maths

South Australian Department for Education
Implementation cost
Evidence strength
Impact (months)
+1
months
Project info

Independent Evaluator

Australian Council for Education Research (ACER)

A three-term professional learning program for Years 6 – 9 maths teachers to engage middle school students’ maths learning.

Students: 7068 Schools: 158 Grant: $195,000 (evaluation)
Key Stage: 0 
Completed September 2018

Overview

Evidence for Learning tested Thinking Maths, a professional learning program which builds teachers’ capabilities to engage middle school students’ maths learning in order to improve their maths achievement. The South Australia Department for Education (the Department) designed the program in response to a drop in maths performance between Year 7 and 9. In order to inform the Department’s understanding of the effect and cost-effectiveness of the program, a rigorous and independently-funded trial is useful. As a decline in maths performance is also evident in Australian students’ results more widely, the findings from this trial may be of interest to other Australian practitioners and policy makers.

The program involves 30 hours of face-to-face professional learning aimed at building teachers’ capabilities to make maths learning deeper and more engaging. Teachers collaborated to design quality maths tasks, using teaching strategies to encourage students’ metacognition and growth mindsets as they progress in maths learning, and made a commitment to apply program ideas in their maths lessons in-between professional learning. Teachers also actively discussed research-informed strategies that were demonstrated in the professional learning sessions.

Years 6 – 9 teachers participate in five professional learning days at 4 – 5 week intervals over three school terms in an eight month intervention period delivered and led by two facilitators from the Department. Teachers are expected to make a commitment to implement the strategies they learn after each professional learning day back in their classrooms to improve student engagement and achievement. 

Australian Council for Educational Research (ACER) conducted this evaluation. This trial is one of the largest randomised controlled trials (RCTs) in education in Australia.

E4L summary

Our evaluation found that students whose teachers received Thinking Maths made, on average, one month’s additional progress in maths, however there were critical differences between year levels. Primary students made an additional two months’ progress in maths, which is promising, but there were two fewer months’ progress for Secondary students. The program had a substantial impact on teachers’ knowledge, but this improvement was not fully translated into impact on students’ outcomes. This result has a high security rating and is very low cost to implement.

Thinking Maths

  • Subject area: Numeracy

Evaluation logic model

The primary research focus was to test the impact of Thinking Maths on student mathematics achievement, due to teachers’ eight month involvement in the Thinking Maths program, compared to teachers in ​‘business-as-usual’ classrooms.

The logic model below reflects the key elements evaluated. It reflects the anticipated change in teachers’ professional identity and self-efficacy, pedagogical content knowledge and beliefs about mathematics teaching that result in changes in teaching practice from Thinking Maths. These in-turn influence students’ mathematics self-efficacy, cognitive engagement in learning, and metacognitive strategies, with the outcome of improved learning.

Figure 5 Outcome evaluation effect size results

Research results

Research result table

* Refer to Evaluation Report Appendix A, used to translate effect size into estimated months progress.

+ Refer to Appendix B, for E4L independent assessment of the security rating.

Evidence for Learning has developed a plain English commentary on statistical significance to support readers in interpreting statistical results in our reports.

The program and schools involved

The project involved 158 schools in South Australia, most of which were located in the metropolitan (63%) and rural (30%) areas. There was an equal distribution of schools in low, mid and high social economic areas.

Program Schools involved

How much will it cost?

The cost of the Thinking Maths program is estimated at $149 AUD per student per year. This estimate includes training and materials ($1070 per teacher or $43 per student), and the significant cost of five Temporary Relief Teacher (TRT) days replacement ($2650 per teacher or $106 per student). Estimates are based on training being delivered to a group of 35 teachers with an average class size of 25 students, reaching 875 students.

Evaluation conclusions

Key conclusions

  1. The Thinking Maths program had a small positive effect, equivalent to one month of additional learning progress on Years 5 – 10 students’ performance in the PATMaths achievement test, when compared to business-as-usual mathematics classes. However, these findings were not statistically significant, meaning we need to treat them with some caution.
  2. Thinking Maths had a statistically significant impact equivalent to two months’ learning gain in Primary students’ achievement on the PATMaths test. However, for Secondary students, there were two fewer months of learning progress.
  3. Among a sub-sample of School Card holders, the students (both Primary and Secondary) of Thinking Maths teachers had two months additional progress in performance on the PATMaths test, however this finding was not statistically significant.
  4. Thinking Maths had the largest statistically significant effect on mathematics teachers’ pedagogical content knowledge, as well as their professional identity and self-efficacy. The intervention also showed a small positive impact on teaching practices overall, with students reporting that Thinking Maths teachers were more likely to give extra help when needed, ask questions to check understanding and challenge their thinking. Findings showed similar gains on students’ cognitive engagement, but no additional gains in metacognition. These results on student outcomes were not statistically significant. A small and statistically significant increase in students’ mathematics anxiety was also found.
  5. Teachers reported a number of benefits of this professional learning program including hands-on activities, expert modelling of metacognition strategies and teaching resources that supported teachers to directly transfer ideas to their classrooms. The process evaluation indicated that timetabled lessons, common tests, set text-books, and lack of time to plan were barriers to effective implementation in Secondary schools. Schools and program development should consider differences in learning contexts to better accommodate and support teachers to optimise implementation.

E4L commentary

Evidence for Learning has provided its own plain English commentary on implications based on the evaluation findings and considerations for schools and systems. 

Reports

The following ​‘practitioner-friendly’ reports are free to access and download.

This evaluation report and supporting materials are licensed under a Creative Commons licence as outlined below. Permission may be granted for derivatives, please contact Evidence for Learning for more information.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Licence.