John Hattie’s ‘Effect Sizes’

John Hattie says ‘effect sizes’ are the best way of answering the question ‘what has the greatest influence on student learning?’.  In effect, Hattie’s in depth research provides evidence to suggest the most successful strategies to utilise in the classroom to increase student achievement.  The information below will take you through his findings.

An effect-size of 1.0 is typically associated with:

• advancing learners’ achievement by one year, or improving the rate of learning by 50%

• A TWO grade leap in GCSE, e.g. from a C to an A grade

An effect size of 1.0 is clearly enormous! (It is defined as an increase of one standard deviation)

Below is Hattie’s table of effect sizes.

Influence

Effect Size

Source of Influence

Feedback

1.13

Teacher

Student’s prior cognitive ability

1.04

Student

Instructional quality

1.00

Teacher

Direct instruction

.82

Teacher

Acceleration

.72

Student

Remediation/feedback

.65

Teacher

Student’s disposition to learn

.61

Student

Class environment

.56

Teacher

Challenge of Goals

.52

Teacher

Peer tutoring

.50

Teacher

Mastery learning

.50

Teacher

Homework

.43

Teacher

Teacher Style

.42

Teacher

Questioning

.41

Teacher

Peer effects

.38

Peers

Advance organisers

.37

Teacher

Simulation & games

.34

Teacher

Computer-assisted instruction

.31

Teacher

Testing

.30

Teacher

Instructional media

.30

Teacher

Affective attributes of students

.24

Student

Physical attributes of students

.21

Student

Programmed instruction

.18

Teacher

Audio-visual aids

.16

Teacher

Individualisation

.14

Teacher

Finances/money

.12 School

Behavioural objectives

.12

Teacher

Team teaching

.06

Teacher

Physical attributes (e.g., class   size)

-.05

School

Terms used in the table (Interpreted by Geoff Petty)

An effect size of 0.5 is equivalent to a one grade leap at GCSE

• An effect size of 1.0 is equivalent to a two grade leap at GCSE

• ‘Number of effects is the number of effect sizes from well designed studies that have been averaged to produce the average effect size.

An effect size above 0.4 is above average for educational research

The effect sizes are averaged, and are a synthesis of research studies thought to be well designed and implemented by research reviewers. Hence they are the best guess we have about what has the greatest effect on student achievement.

Some effect sizes are ‘Russian Dolls’ containing more than one strategy e.g. ‘Direct instruction’ is a strategy that includes active learning, structured reviews after one hour, five hours and 20 hours study. There is also immediate feedback for the learners, and some corrective work if this is necessary.

Hattie does not define most of the terms in his table. My understanding of them is:

Feedback Hattie has made clear that ‘feedback’ includes telling students what they have done well (positive reinforcement), and what they need to do to improve (corrective work, targets etc), but it also includes clarifying goals. This means that giving students assessment criteria for example would be included in ‘feedback’. This may seem odd, but high quality feedback is always given against explicit criteria, and so these would be included in ‘feedback’ experiments.

As well as feedback on the task Hattie believes that students can get feedback on the processes they have used to complete the task, and on their ability to self-regulate their own learning. All these have the capacity to increase achievement. Feedback on the ‘self’ such as ‘well done you are good at this’ is not helpful. The feedback must be informative rather than evaluative. See the feedback page on my website or Teaching Today chapters 6 and 43.

Students prior cognitive ability: This is IQ and similar measures

Instructional quality: This is the student’s view of the teaching quality; the research was done mainly in HE institutions and colleges.

Instructional quantity: How many hours the student is taught for. Direct instruction: Active learning in class, student’s work is marked in class and they may do corrective work. There are reviews after one hour, five hours, and 20 hours study. See the separate handout.

Acceleration I think this is very bright students being put forward a year in schools

Home factors Issues such as social class, help with homework, extent to which the learner’s education is thought important

Remediation/feedback Diagnosing what students find difficult, and getting students to fix it.

Student’s disposition to learn Student motivation

Challenge of Goals Students being given challenging but at least partially achievable goals

Peer tutoring students teaching each other, peer-explaining, peer-checking, peer-assessing etc

Mastery learning A system of tests and retests of easy material with a high pass mark, if a student does not pass they must do extra work and then take a retest on the material they were weak at. See Teaching Today by Geoffrey Petty.

Questioning Students being questioned. The most effective questions are high order ‘why?’ ‘how?” and ‘which is best?’ questions that really make students think . They need to be given time to think too, and can do better if they work in pairs than work alone.

Effect sizes Below 0.4 now follow. Some of these add a lot of value in a short time so don’t ignore them completely

Advance organizers A summary of the material in advance that puts some sort of structure to it. This can take a matter of moments and is best referred back to often.

Computer-assisted instruction Effect sizes for this are gradually rising as the instruction becomes more interactive, more engaging and generally better designed.

Instructional media Using state of the art visuals, videos, etc

Testing Testing by itself is not as effective as remediation/feedback where the test is used to find what the student needs to improve and they then do corrective work.

Affective attributes of students The attitudes, beliefs and feelings of students

Programmed instruction A form of instruction that involves students being taught by a computer or set of workbooks, by doing a series of prescribed tasks. If the student gets an answer wrong they are directed back to correct their misunderstanding. Devised by Skinner in the 1960s, but not much used now.

Individualisation Students working on an individualised programme of learning. This may work better if students are not working in a solitary way.

Finances/money Funny ….. this seems to have a larger effect when paid to me…

Behavioural objectives Having and using objectives in the form: “The students should be able to…” immediately followed by an observable verb. For example ‘explain’ is okay because you can listen to, or read the student’s explanation. However ‘understand’ isn’t behavioural because you can’t see or read the understanding.

Retention Students who do not do well enough in one school year, being kept back to do the year again.

Beware Over-interpretation!

  • Surface learning (e.g. rote remembering without understanding) could produce high effect sizes short term for low cognitive skills such as remembering. For example the use of mnemonics has an effect size of about 1.1 (There is more to learning than passing memory tests.)

For a more user friendly interpreatation of John Hattie’s results please refer to Geoff Petty’s book Evidence Based Teaching.  It explains Hattie’s effect sizes along with providing succesful strategies to implement in the classroom.  We have a copy in the LRC.  Nicola Mason will point you in the right direction!

Ps.  Here’s what it looks like…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s