Foul Claimed on New School League Tables

Headteachers are demeaning themselves in their rush to criticise the new English Baccalaureat figures included for the first time in this year's league tables, and thereby excuse their own schools' poor rankings.

The English Bac, or EBac, is awarded if a student gains GCSE grade C or above in each of English, Maths, Double Science, a humanities subject and a language.

Complaints have grouped into three main lines:

1. No time to properly game the system.

The main complaint is that the figures are retrospective, with the rules of the game only published after the exam results were out. 'How can we be expected to do well without the time to change our curriculum policies?' chant the headteachers.

This exposes the key moral weakness of modern schools, which is that directly manipulating the key indicators to make the school look good is preferred to actually improving the pupils' education.

Gaming the system, then, is the main occupation of school managers.

2. The EBacc is a return to Academic Snobbery.

Why not allow vocational courses as well?

Schools has a choice to make. Enter children on to the course with the best educational aims (say, French or Science GCSEs) when many will achieve grades D to G passes and so not count in the laegue tables. Or, enter them for vocational courses such as the Btec, that guarantee the 'equivalent' of four GCSE grade Cs to any pupil still conscious at the end of the course.

The choice is really this stark, and sadly most schools go for option B with varying degrees of enthusiasm. Those under the cosh from Ofsted especially realise this is the only way out or 'special measures'. Look at the tables and you can spot such schools: they will have improved their 5 A to C figures at improbably fast rates, have high CVA (value added) scores from the additional 'equivalent' courses and LOW EBacc rates.

The 'most improved' school in the country, Perry Beaches in Birmingham, has moved its 5 A to C figures from 21% to 74% in four years. CVA is also high, but only 3% got the EBacc. Since it takes four or five years to progress through the school as a pupil, the changes must have been instantaneous to have fed through this quickly.

Certificates of GCSE equivalent passes shouldn't count if everyone passes them. It misleads prospective parents into thinking the school is academic and improving, when it is only the figures that are going up. The quality of the education may actually be declining in these schools as they move from GCSE to Btec and other similar courses.

3. Independent schools are unfairly penalised.

Independent schools are not restricted to only offer courses approved by the politically directed Qualification Curriculum Authority which only approved courses with sufficient levels of coursework in their assessment schemes.

Independent schools don't approve of coursework, so many offer alternative courses, such as Classical Civilisation, which don't count towards the EBacc, damaging their figures.

Now, this is a fair complaint. But Independent schools are not compelled to enter the League Tables manipulation game, or even publish figures at all. They are free to create their own tables if they wish so they can compete on their own manicured level playing fields with their own rules.

Unfair, perhaps, but they can take their ball and play elsewhere if they don't like it.

Less Gaming, Please.

The arrival of the EBacc has embarrassed lots of schools. They complain of the pressures of league tables and the focus on A to C grade passes which excludes the varying efforts of anyone not near the grade boundaries. But they should welcome anything that makes gaming harder and so less attractive. Less gaming should herald a move back towards professional judgements in schools instead of political ones, where the children come first.

I won't be holding my breath though. Heads have been manipulating their table positions for a long time, and will be looking for ways to continue the game. It is all many of them know.
Read more!

Ed Balls Shocked by My Question

Since the Schools Secretary has failed to answer any questions I've addressed to him in posts, I submitted a question to Sarah Ebner's blog over at the Times. She had managed to get Balls to join a live Q&A session and had asked for questions to ask his. Needless to say, the slimy minister read all the questions and proceeded to answer the ones he had prepared earlier, in true Blue Peter style.

The full discussion is here , with my question down at times 14.02 to 14.05. The accompanying photo, captioned http://www.timesonline.co.uk/tol/comment/live_debate/article7012253.ece is reproduced here — does he look shocked at the quality of the question?

The edited sequence goes like this:

14.02 Sarah Ebner:
This is another point which comes up often on the blog. Glen asks: why persist with judgements of schools based on raw percentages of students achieving 5 grade C GCSEs? It penalises schools serving deprived areas - schools which need a hand up, not a kicking. It also pressurises schools to focus on grade D students, and encourages entries into easier ICT online courses and such like.

14.05 Ed Balls:
Sarah, you are completely right and so is Glen, as parents we all have to look at the current league table and try to work out what they really mean on the basis of what we know about the school itself, the catchment area etc. And league tables can sometimes suggest schools are 'high achieving' when they actually do a poor job at raising standards and supporting progression. Our new Report Card is designed to give parents much more information - about raw results but also whether all children make progress, discipline, parent satisfaction etc. I think it will be much fairer and more informative - it will be in all schools over the next 2 years.

So, I had asked him about the huge pressure that the government and Ofsted puts, often unfairly, on schools in difficult circumstances, distorting their priorities, and he answers an imaginary question about league tables!

OK, he did that to all of the questions, but then why did he bother to travel to the Times offices, just to act as if he were in the House of Commons (not) answering questions put to him there?
Read more!

Mass Produced Target Grades

Sixth form students will by now have dragged themselves through the January exam series. They can relax until scores are released in March, when most will be judged according to their college 'target grades'. And it is likely to be a miserable experience for most.

I used to talk to my students and get to know their individual strengths and weaknesses. I would encourage those who I perceived were studying hard, and chide those who were just attending class without the necessary intellectual engagement. Reports to parents and managers were based on my professional opinion of each child. But not any more.

Teachers still get to know each of their charges, but their professional judgements are now routinely tempered by the knowledge that performance against their grade target trumps all other information.

Value Added

Target grades are now the ubiquitous tool of comparative assessment in schools: Key Stage 3 results are used to predict GCSE grades, while GCSE grade averages are used to compute the most likely grade a student might achieve at A Level. This is a very good process for working out if the school is doing a good job, since if a year group cohort gains a mean score above the mean predicted grade, then the group has learned more than could reasonable been expected. The school thus has recorded some value added, in the language of education.

Using the same data for individual teachers is only likely to be reliable over a period of several years, since the sample sizes from individual classes are much smaller, leading to more variation from year to year.

Blinded by Numbers

The big problem stems from applying these statistics to individual students. It is very easy to calculate an expected grade from a single child's previous achievement, but with a sample size of just one, the precision is poor. The reliability stemming from a cohort in the hundreds is lost, and the prediction is routinely in error by a whole grade or so. (See my post Physics Exams Too Easy, Says Ofqual

Now, this would not be a problem if these figures were just another piece of the puzzle to be understood by the teacher, but OFSTED, the government overlord of teaching standards, thinks students should know these rough predictions, and be challenged to achieve them. And leaned on if they don't come up to scratch.

Once upon a time, I got to know my own students, and made judgements as to their individual abilities and potentials, and assessed their effort accordingly. Not perfect, but at least both teacher and student were in the loop.

Forget the Child – Press the Button and Set That Target

Now, each student is given a grade to achieve by the end of a two year course, during which they will mature and develop. If they are very lucky, they will get several target grades which take into account the historical difficulties of each subject they are studying. If not, as is happening more commonly now, they will get a single grade to span the range from Photography and Media to Chemistry and Maths. And to make the target aspirational, a grade will be added to ensure that only a quarter of students will be able to meet their targets, with poor reports and disciplinary procedures for those souls unlucky enough to keep missing impossible targets.

Advantages

• Simple and cheap to operate.
• Keeps OFSTED happy.

Disadvantages

• No educational merit.
• Can turn keen students into serial target-missers.

An open and shut case for school managers. Shame about the children.
Read more!

Government Buries 'Good' News

Even though the new Academies have done so well in last summer's exam period, the government can still not bring themselves to publish usable comparative data to prove the case.

The recent publication of the GCSE and Equivalent Results in England was accompanied by a government press release which declared:

“Even more encouraging is the clear fact that children who have faced the most challenging circumstances over the past 12 years have not been cast adrift and left at the bottom, but thanks to the Academies, National Challenge and City Challenge programmes have actually seen a more rapid improvement in results than those in the least deprived areas.”

Concerns

The biggest concern with the steady and dramatic increases in the figures, after grade inflation, is the widespread tactic of entering weaker pupils for the laughably easy vocational courses that have 'GCSE equivalence'. For example, half of all schools enter almost everyone into the vocational ICT course instead of the more challenging GCSE ICT, as it can be completed in one year and can be worth anything up to four GCSEs at grade C or above. The Telegraph reports:

A report last year by Ofsted, the education watchdog, found that qualifications such as the one run by OCR were “less demanding” than other mainstream exams. It said pupils were able to pass “whether or not they had understood what they had done”.

It is true that the number of students nationally that gain 5 A*-C grades is only 3% less if you exclude the 'equivalent', vocational courses, but there are categories of schools which have a lot to lose if their pass rated don't improve sharply.

Schools Minister Vernon Coaker's press-release said that the new Academies A*-C grades had increased at double the national rate, so I looked at the accompanying data.

Sneaky Tricks

The data looks comprehensive, and even breaks down results for different types of school and with/without the vocational courses. The time series do indeed show Academies improving faster then any other major category of schools, but the figures for the Academies without the vocational courses are missing!

Vernon's minions emphasised the Academy raw percentage increases, but rolled their key data in with that for comprehensive schools, making it impossible to see what effect the dash for vocational courses has had.

Since it is just these schools that are most desperate to bump up their figures quickly to avoid the Wrath of Balls, it is very suspicious that the data, which was released with some fanfare, should have been airbrushed like this. What is the government afraid of? Given the obsession of the government with media manipulation, one can only assume that Academies have achieved their politically essential successes through a choice of courses that benefited the school more than their unfortunate pupils.
Read more!

NUT SATs Ballot

The National Union of Teachers (NUT) has voted to scupper the SAT exams for eleven-year-olds this year, but it won't produce the renaissance in teaching they expect.

It is the teaching union silly season, and time for their AGMs. There is normally a flurry of embarrassing quotes from representatives that are quickly ignored, but this week a substantial motion has been passed by the largest union, the NUT. They have decided to ballot members for industrial action to disrupt the national assessment of seven- and eleven-year-olds (Key Stage 1 and 2 SATs), taking advantage of the recent collapse of the Key Stage 3 assessments and the dithering of the government minister Ed Balls.

Teachers generally dislike these assessments as they are unreliable and used for annual teacher appraisals, and now seems like the best chance in years to force a weak government to abandon them.

The massive expansion of national exams over the last decade or so, fed by the movement to modular exams that can be retaken an unlimited number of times at GCSE and A Level, along with the SATs (the National Curriculum Tests at ages 7, 11 and 14), has overloaded the exam boards' marking systems. There are simply not enough markers in the country to process all the papers. This caused the collapse of the Key Stage 3 exams last year, and has caused this year's results to be posted later than ever before, and appeals are expected to flood in shortly afterwards due to quality control problems, especially for the English assessments.

I have posted before that these tests have become too ‘high stakes’ to be useful and national standards should be assessed in other ways, but the NUT is not keen on developing a decent assessment system. They just want to be rid of the SATs.

Pressures

Their real problem, so they say, is that the pressure placed on pupils in the run-up to the tests is too great. Primary school children spend much of Year 6 preparing, practising the tests and taking test questions away to do for homework. Every child is given targets couched in the assessment language ('I'm now at level 3B for Writing, and I am aiming for a level 4C' the pupils will repeat) and the school inspectors check that pupils are aware of them.

True, the pressures are ridiculously high, and teaching to the test is so endemic that to suggest anything different to any teacher younger than 40 will get you a confused look. It is also true that the government has set up the system to be like this, despite their protestations of innocence, but the Union is picking on the wrong target deliberately, hoping that no-one will question their members' complicity in the whole affair.

Act Professionally

The teachers have a great deal of freedom in how they manage their classrooms day-to-day. If they do not want to pressure their charges then they should stop talking up the exams all the time. They should stop teaching to the tests and setting questions from previous years' papers, and they should certainly stop running the government funded 'Booster Sessions' - additional revision work for those poor souls deemed to be close enough to the pass level that they can be artificially pushed over the line with some special attention and pressure.

Instead of declaring the tests 'harmful' to pupils, the NUT ought to just tell their members to stop squeezing the last drop of exam performance from their classes simply to gain better pass figures for their annual appraisals and a higher league table position for their school. If the teaching unions want teaching to be treated as a profession, then professional behaviour must be encouraged. All the while teachers put ratings ahead of education their motives will be suspect.

Teaching Renaissance,

If teaching to the test stopped, whether by scrapping the tests or by teachers taking control of their classrooms, then, the unions say, the curriculum will become broader and children will have a better experience. But they are mistaken. Even if the test went, the skills of teachers to plan their own programmes of study have so withered that most teachers would not know what to do with the extra term of teaching. What would they do with no exam to prepare for? How would they know what skills to develop and knowledge to learn if it is not written down in great detail by the government?

'It's not in the test!'

When the SATs were first introduced, teachers had some idea of what schooling was for, some philosophy of education. But over the years those teachers have retired and the younger ones have only known teaching for national assessments. Those few who dare to go beyond the minimum entitlement laid down in the National Curriculum, following their pupil interest or their own enthusiasms, have been slapped down by managers with the immortal lines:

'What are you teaching that for - it's not in the test!'

The English teaching 'profession' has become so de-skilled that if the tests disappeared nothing would really change. If the prison doors were flung open tomorrow, most teachers would be too frightened of the freedom to go out into the daylight, doomed to pace around the same familiar cell.
Read more!

Jim Knight wants more 'Flash and Bang'

The just released 2007 Trends in International Mathematics and Science Study (TIMSS) has seen England rise to fifth position after the four yearly study looked again at the quality of the science education of fourteen-year-olds around the world (BBC report: England's Pupils in Global Top 10).

English students are beaten only by those from Singapore, Taiwan, Japan and Korea, with all of Europe trailing in their wake. Jim Knight is clearly pleased at this validation of Labour policies on the world stage, but he can't bring himself to ease the political pressure off over-burdened schools, even if they have done all he asked of them.

Party Pooper

Knight has looked and looked, and he managed to find some bad news in the report. That's right — science teachers up and down the country can stop partying, under the impression that all was well in their subject and a pat on the back was due.

Children are enjoying science less than they used to! There has been a 21% drop in 'positive attitudes' reported by the pupils, and Jim is not happy.

Teachers, go and sit on the naughty step.

Must Do Better

Being the best in Europe and the industrialised West is not good enough if a few far-eastern nations with fantastically well drilled children are better.

Knight says in the press release:

This shows we are on the way to being world class but as we move towards this goal we need to make sure every child has fun in the classroom as well as achieving good results.
…
I am determined to make maths and science more exciting subjects to teach and learn, and I want every school to have access to the most innovative and effective teaching methods. I want more action in the classroom and more problem solving and ‘flash and bang’ to enthuse our pupils.

A new OFSTED target, perhaps? Inspectors could report:Your lesson on nuclear power was well taught and the children learned well, but there wasn't enough 'flash and bang' for the lesson to be rated any good.'

Squeezing the Pips

Other countries to suffer from reduced student positivity included Singapore and Hong Kong — both in the top ten alongside England. Jim Knight seems to think that league table rankings and pupil enjoyment are independent of each other, but teachers have complained for years about the curriculum and targets straitjacket that they have to operate in, and the effect on the enjoyment that classes are able to have.

The government has squeezed children hard so that they achieve their potential, but the pips are squeaking now. If he was serious about restoring awe and wonder to school science lessons, then Knight and Balls would be cutting the testing and accountability burden.

Freeing teachers to impart some of their love for their subjects, though, would risk a slip in the rankings. And that would never do, would it?
Read more!

'Coasting Schools' Attacked in the Latest Ministerial Balls-Up

The Government is tilting at windmills again with a ministerial attack on imagined weak schools. Ed Balls, the Schools Secretary, has decided it is time to tackle good schools for not being good enough. The press release says

Ministers are taking action after analysis shows that one in seven pupils do not progress a whole attainment level in English between the ages of 11 and 14. Until now ‘coasting’ schools have often missed out on focused attention and have been hard for parents to identify because of apparently satisfactory results.

This is appallingly wrong-headed for two reasons.

The Limits of Testing

First, it is in the nature of the tests that some pupils are awarded the wrong level. If everyone advanced by a whole level in their understanding, then the limited reliability of the tests would mean that a third will get the wrong level (or, about one in six would appear to have stood still for three years.)

A year ago The Primary Review, an independent two-year long enquiry into primary education in England, reported in its research survey Assessment Alternatives for Primary Education that

Regardless of the consistency of individual test items, the fact that a test has to be limited to a small sample of possible items means that the test as a whole is a rather poor measure for any individual pupil. This is because a different selection of items would produce a different result. Wiliam (2001) estimated the difference that this would make for the end of Key Stage tests in England. With a test of overall reliability of 0.80, this source of error would result in 32 per cent of pupils being given the wrong level.

This made a big splash at the time, for example here in the Guardian.

'Value Added' Scores

Second, the coasting schools will be identified if they meet any of a list of criteria given in the same press release, including if

• The school’s Contextual Value Added (CVA) score is significantly below average;
• There has been little or no improvement in the school’s progression rates over several years;

So a school that is consistently doing well needs a kicking if pass rates don't go up year on year, or if that school's particular challenges don't feature in the CVA ranking model. OFSTED, England's schools inspectorate, themselves say that categorising schools on the basis of CVA scores is "meaningless", as described by the BBC news item last August:

… in an example OFSTED gives it may appear that a school with a CVA of 1,009 is doing better than another with a CVA of 992.

"However, that would be incorrect," [OFSTED] says in new guidance to schools about the use of data. "In both cases, the range between the upper and lower confidence limits includes 1,000, so both schools are achieving average outcomes; their performance is about as expected."

The guidance adds: "No meaning can be attached to an absolute CVA value, and any ranking of schools by their CVA values is meaningless."

Homework for Ed Balls

Ed Balls needs to visit schools, not to "target" them (a rather aggressive term for what should be a supportive programme), but to study some maths. It is a shame that Mr Balls learned enough arithmetic at school to manipulate figures, but did not make a sufficient study of how to handle anything but the simplest data.
Read more!

Ofsted: Maths Exam Progress Is Not Evidence for Better Education

In their report, Mathematics: Understanding the score, Ofsted bemoans the amount of teaching to the test that goes on. It seems that, after many years of squeezing ever increasing exam grades out of children, they have finally seen some of the damage they have caused.

Their press release says that many schools...

...are not teaching mathematics well enough because they place too much emphasis on routine exercises and on ‘teaching to the test’. While this style of teaching prepares pupils to pass examinations, and gain necessary qualifications, it is less effective in promoting the required understanding to apply mathematics to new situations, solve problems and communicate solutions.

So, after years of forcing school results to climb improbably, under the threat that the satisfactory is not good enough mantra will blight their reports, are Ofsted going to back off from damning teachers who are devoted enough to their charges to ignore the unprofessional pressure from managers and HM Inspectors?

Like hell they will!

Managers will still be sweating over a tenth of a percentage point drop in the A* to C figure and Heads of Department will still be coercing teachers to keep marking and returning the coursework until it is right. Pupils revealed as just below the threshold grade D or level 5 from the relentless practising past-paper will still be enrolled on Ofsted endorsed Booster Sessions whether they need this additional support or not while needy students in lower grade bands are left to flounder.

Leopards don't so easily change their spots. This report will be, however, another excuse for inspectors and politicians to lean on teachers all the more.
Read more!