2008-08-24

Assessing Teachers Needs Research

Now the summer exam results circus is over &mdash and the usual suspects have made their usual criticisms of the examination system &mdash I would like to suggest to OFSTED (and school managers) what they should be looking at when they assess teaching.

Currently, most schools operate formal lesson observations where a manager sits in on a lesson and fills in a proforma. This sheet has key observations to make and a four level grading system, based on OFSTED's procedures so that the school can defend it when inspectors arrive. (The grades range from 1 = very good to 4 = poor, with 3 = satisfactory, the new poor). Teachers are, of course, carefully trained in the system, so that formally observed lessons fill one tick-box after another.

Particularly important to the watched are those lesson features deemed good practice. They can be set as hurdles, limiting the grades otherwise good lessons if the box is not ticked. For example, 'are the lesson aims written on the board?' Not 'do the students understand what they are learning?' or 'were the students swept along?'. Another example: 'was ICT used in the lesson?', even if that use was no better that the non-computer alternative, since there is a government target on ICT use in the classroom.

So what would be better? I suggest that there is plenty of research evidence as to what techniques work in classrooms. Rather than writing off a teacher on their annual observation because they did not use ICT that lesson and the class exam average was below the 'benchmark', or because the teacher was idiosyncratic, the observers should be checking to see if the teacher was doing what objectively works.

Most educational interventions have some positive effect on students achievements, so what is needed is a list of the most effective interventions, since we all have only limited time and energy. There are several literature reviews summarising the evidence for interventions. For example here, and here.

Effect sizes can show quickly which interventions are worth expending time and effort on and which can safely be given lower priority. A list from the first link shows these effect sizes:
Feedback / 1.13
Prior Ability / 1.04
Instructional Quality / 1.00
Direct instruction / 0.82
Remediation feedback / 0.65
Student disposition / 0.61
Class environment (culture) / 0.56
Challenge/goals / 0.52
Peer tutoring / 0.50
Mastery learning / 0.50
Team teaching / 0.06
Behavioural objectives / 0.12
Finances/money / 0.12
Individualisation / 0.14
Audio visual aids / 0.16
Ability grouping / 0.18
Effect sizes do not tell you what is good, but they do indicate what actually improves student outcomes. An effect size of 1.0 is well worth achieving, and is approx. equivalent to one year of advancement. 0.5 is well worth a try. Requiring your teachers to include interventions with lower effect sizes may be counter-productive, indeed some of your better teachers may start to quietly rebel.

Lesson observations have the power to force teachers to do what the Principle or Head Teacher wants them to do. It is essential that these demands are informed by the best educational research, and not by political or bureaucratic considerations.

No comments: