Evidence for Learning: Using the Toolkits

Using the Toolkits

Guide to using our Teaching and Learning Toolkit and Early Childhood Education Toolkit.

The Toolkits are designed to support educators and educational leaders who are making decisions about how to improve learning outcomes.

Our Teaching and Learning Toolkit (the T&L Toolkit) and the Early Childhood Education Toolkit (ECE Toolkit) are accessible summaries of education research. The two Toolkits are designed to support educators and educational leaders in making evidence-informed decisions about how to improve learning outcomes, particularly for children and young people experiencing disadvantage.

The Toolkits do not make definitive claims as to what will work to improve outcomes in a given setting. Rather they provide high quality information about what is likely to be beneficial based on existing evidence. 

The Toolkits aim to: 

  • support evidence-informed decision making in Australian schools and early childhood education settings;
  • provide guidance for educators and educational leaders on how to use their resources to improve educational outcomes for their students, particularly those experiencing disadvantage;
  • act as an introduction to educational research.

The Toolkits synthesise international and Australian research, and present a wide range of educational interventions, summarised in terms of:

  • the average months’ worth of learning progress; 
  • the strength of the evidence;
  • the cost.

The Toolkits are live resources updated regularly as new studies are published in Australia and internationally.

The [Teaching & Learning] Toolkit doesn’t tell us what to think, but provides an insight into research so we can make better judgments of our own.

~ Professor Martin Westwell in his former role of Director, Flinders Centre for Science Education in the 21st Century
Evidence for Learning: Introduction to the Teaching & Learning Toolkit.

We believe that the Toolkits should be used as part of a four-step decision-making process.

Step 1: Consider your context and what you want to achieve

It is crucial to consider the needs of your students, what you want to achieve, and any barriers you might face before adopting an approach.

Step 2: Look behind the headlines and think about what is behind the average’

While the Toolkits do not make definitive claims as to what will work to improve outcomes in a given setting, they provide a starting point on what might be valuable, helping to identify best bets’. It is important to explore the detail in the Toolkits’ pages, including the impact of approaches for different ages or subjects.

Step 3: Think about cost and evidence as well as impact

Some approaches may be effective but not cost effective. Other approaches may have a lower overall impact but may have a more well-established evidence base.

Step 4: Draw on your professional expertise and consider implementation

The Toolkits can tell you whether an approach has a good track record, but your professional expertise, other resources and ongoing evaluation are also important in making well-informed decisions on what is best to support children’s learning. Adopting a new approach requires careful planning, implementation and monitoring.

In my role, I spend a great deal of time collecting, reading and unpacking research around teaching practice and approaches with our teachers. The [Teaching & Learning] Toolkit is neat, concise and easy to use. It will save me countless hours in the way it lights the path directly to the most relevant and reliable research.

~ Frances Roberts, Head of Curriculum, Bounty Boulevard State School

Months’ impact

Months’ impact is estimated in terms of the additional months’ progress you can likely expect children and young people to make as a result of implementing an approach, compared to similar children and young people who did not receive the approach. The months’ impact takes the average progress over a year as a benchmark.

For example, the feedback’ strand in the Teaching & Learning Toolkit shows that improving the quality of feedback provided to students has an average impact of six months. This means that students in a class where high quality feedback is provided can be expected to make, on average, six months more progress over the course of a year compared to another class of students who are performing at the same level at the start of the year.

Effect sizes

These impact estimations are based on effect sizes’ reported in comparative data (see table below). Effect sizes are quantitative measures of the impact of different approaches on learning. 

Effect sizes describe the size of the difference between two groups in a standard and comparable way. However, it can be difficult to understand what a given effect size actually means for the progress of children and young people. That is why the Toolkits translate effect sizes into the months’ progress measure, as shown in the table below.

Months impactEffective size from …… toDescription
0

-0.05

0.05Very low or no effect
10.060.09Low
20.100.18Low
30.190.26Moderate
40.270.35Moderate
50.360.44High
60.450.52Very high
70.530.61Very high
80.620.69Very high
90.700.78Very high
100.790.87Very high
110.880.95Very high
120.96>1.0Very high

Evidence strength

Evidence strength padlock’ ratings provide an overall estimate of the robustness of the evidence. Topics are awarded padlocks based on the number of studies that meet the Toolkit inclusion criteria. Additional padlocks are then lost due to a number of potential threats to evidence security.

Studies included in the Teaching & Learning Toolkit all take place in schools rather than lab-studies, involve a comparison group (rather than simply testing students before and after intervention), and meet the definition of each topic.

Additional padlocks can be lost because:

  • only a small percentage of the studies have taken place recently;
  • a large percentage of the studies included are not randomised controlled trials*;
  • a large percentage of the studies took place in conditions distinct from regular practice of educators;
  • a large percentage of the studies were not independently evaluated;
  • there is a large amount of unexplained variation between the results included in the topic.

* While other study designs still give important information about effectiveness of approaches, there is a risk that results are influenced by unknown factors that are not part of the intervention.

PadlocksRatingDescription
0 padlocksN/AFewer than 10 studies that meet the inclusion criteria, and months’ progress are not communicated.
1 padlockVery limitedAt least 10 studies that meet the Toolkit inclusion criteria.
2 padlocksLimitedBetween 11 and 24 studies that meet Toolkit inclusion criteria.
3 padlocksModerateBetween 25 and 44 studies that meet Toolkit inclusion criteria.
4 padlocksExtensiveBetween 45 and 69 studies that meet Toolkit inclusion criteria.
5 padlocksVery extensive70 or more studies that meet Toolkit inclusion criteria.

Cost

The cost estimates in the Toolkits are based on the average cost of delivering the intervention. Each strand also discusses the range of potential costs— for example, many interventions are more expensive when professional development costs are added to ensure high quality delivery.

Cost estimations in the Toolkits are based on the approximate cost of implementing an approach in a class or group of 25 students or young children. Cost estimates commonly include the cost of additional resources, the cost of training or professional development or the cost of activities for students or young children.

CostDescription
$Very low: up to about $4,000 per year per class or group of 25 children, or less than $160 per child per year.
$$Low: $4,001 to $8,000 per year per class or group of 25 children, or up to about $320 per child per year.
$$$Moderate: $8,001 to $30,000 per year per class or group of 25 children, or up to about $1,200 per child per year.
$$$$High: $30,001 to $50,000 per year per class or group of 25 children, or up to $2,000 per child per year.
$$$$$Very high: over $50,000 per year per class or group of 25 children, or over $2,000 per child per year.

The Teaching & Learning Toolkit was developed by two charities based in the UK, the Education Endowment Foundation (EEF) and the Sutton Trust, in collaboration with academics at Durham University. In 2015, the first version of the Australian Teaching & Learning Toolkit was launched, with support from the Department of Education and Training in Victoria, Social Ventures Australia (SVA) and a number of other organisations in Australia. The Australasian Research Summaries for approaches in the Teaching & Learning Toolkit were produced in collaboration with Melbourne Graduate School of Education and launched in 2016.

The Early Childhood Education Toolkit was produced by the EEF in collaboration with a team of academics at Durham University, led by Professor Steve Higgins. The Australasian Research Summaries for the 12 approaches in the Early Childhood Education Toolkit were produced in collaboration with Telethon Kids. In 2019, the Australian Early Childhood Education Toolkit was launched with support from The Bryan Family Foundation.

In July 2022, an updated version of the Australian Teaching & Learning Toolkit was launched. In early 2023, an updated version of the Australian Early Childhood Education Toolkit will be launched.

The full references for the Toolkits are:

The Teaching & Learning Toolkit


Education Endowment Foundation. (2022). Evidence for Learning Teaching & Learning Toolkit: Education Endowment Foundation. Retrieved from https://evidenceforlearning.org.au/education-evidence/teaching-learning-toolkit

Evidence for Learning in collaboration with Melbourne Graduate School of Education. (2016). Australasian Research Summaries. Retrieved from https://evidenceforlearning.org.au/education-evidence/teaching-learning-toolkit

The Early Childhood Education Toolkit


Education Endowment Foundation. (2019). Evidence for Learning Early Childhood Education Toolkit: Education Endowment Foundation. Retrieved from https://evidenceforlearning.org.au/education-evidence/early-childhood-education-toolkit

Evidence for Learning in collaboration with Telethon Kids. (2019). Australasian Research Summaries. Retrieved from https://evidenceforlearning.org.au/education-evidence/early-childhood-education-toolkit