Ed Balls Shocked by My Question

Since the Schools Secretary has failed to answer any questions I've addressed to him in posts, I submitted a question to Sarah Ebner's blog over at the Times. She had managed to get Balls to join a live Q&A session and had asked for questions to ask his. Needless to say, the slimy minister read all the questions and proceeded to answer the ones he had prepared earlier, in true Blue Peter style.

The full discussion is here , with my question down at times 14.02 to 14.05. The accompanying photo, captioned http://www.timesonline.co.uk/tol/comment/live_debate/article7012253.ece is reproduced here — does he look shocked at the quality of the question?

The edited sequence goes like this:

14.02 Sarah Ebner:
This is another point which comes up often on the blog. Glen asks: why persist with judgements of schools based on raw percentages of students achieving 5 grade C GCSEs? It penalises schools serving deprived areas - schools which need a hand up, not a kicking. It also pressurises schools to focus on grade D students, and encourages entries into easier ICT online courses and such like.

14.05 Ed Balls:
Sarah, you are completely right and so is Glen, as parents we all have to look at the current league table and try to work out what they really mean on the basis of what we know about the school itself, the catchment area etc. And league tables can sometimes suggest schools are 'high achieving' when they actually do a poor job at raising standards and supporting progression. Our new Report Card is designed to give parents much more information - about raw results but also whether all children make progress, discipline, parent satisfaction etc. I think it will be much fairer and more informative - it will be in all schools over the next 2 years.

So, I had asked him about the huge pressure that the government and Ofsted puts, often unfairly, on schools in difficult circumstances, distorting their priorities, and he answers an imaginary question about league tables!

OK, he did that to all of the questions, but then why did he bother to travel to the Times offices, just to act as if he were in the House of Commons (not) answering questions put to him there?
Read more!

Election Poll News: No Change Really

Looking through last week's papers (our paper delivery boy made it through all the snow disruption last month, but seems to have forgotten today now the weather's fine and the sky is blue) I found a story I had missed.

The small front page story started with

“Gordon Brown has insisted Labour could still win the general election outright as another poll showed the Tory lead narrowing.

Research by ICM for the Sunday Telegraph put David Cameron's party down one point since last month on 39%.”

I know it might indicate something about my personality type, but the story irritated me. Down one point, with a sample size of a measly thousand?

Now, results of course vary from sample to sample in a predictably random way, which puts a limit on the reliability of any judgements made from the data from just one sample. But how much? Trust the data to within 0.1%? Or 10%?

Here's the maths bit — skip to the next paragraph if it's not your thing.

If the poll results over time can be represented approximated by a Poisson distribution (a reasonable assumption), then the variance of the number of people preferring one party is equal to the mean number of people choosing that category. For opinion polls, we don't know this mean, but it is close to the reported figure, i.e 39% of 1002 in the sample. With this variance, we can be about 95% sure that the true mean number of people in repeated samples choosing Tory would be 390 (40%) plus or minus twice the square root of the variance (about 40 votes, or 4%).

And the result is:

So the real result is "The Conservatives polled between 35% and 43%, which is consistent with no change at all." OK, not a good headline, but even newspapers have an obligation to at least try to be right. I'm sure papers used to put this sort of information at the foot or the article (where hardly anyone would see it), even if the article writer ignored such a basic check. But to have every outlet from newspaper to the TV news run a similar story is laziness.

Every newsroom must have someone with enough maths skill to do this right, haven't they?

Read more!

Mass Produced Target Grades

Sixth form students will by now have dragged themselves through the January exam series. They can relax until scores are released in March, when most will be judged according to their college 'target grades'. And it is likely to be a miserable experience for most.

I used to talk to my students and get to know their individual strengths and weaknesses. I would encourage those who I perceived were studying hard, and chide those who were just attending class without the necessary intellectual engagement. Reports to parents and managers were based on my professional opinion of each child. But not any more.

Teachers still get to know each of their charges, but their professional judgements are now routinely tempered by the knowledge that performance against their grade target trumps all other information.

Value Added

Target grades are now the ubiquitous tool of comparative assessment in schools: Key Stage 3 results are used to predict GCSE grades, while GCSE grade averages are used to compute the most likely grade a student might achieve at A Level. This is a very good process for working out if the school is doing a good job, since if a year group cohort gains a mean score above the mean predicted grade, then the group has learned more than could reasonable been expected. The school thus has recorded some value added, in the language of education.

Using the same data for individual teachers is only likely to be reliable over a period of several years, since the sample sizes from individual classes are much smaller, leading to more variation from year to year.

Blinded by Numbers

The big problem stems from applying these statistics to individual students. It is very easy to calculate an expected grade from a single child's previous achievement, but with a sample size of just one, the precision is poor. The reliability stemming from a cohort in the hundreds is lost, and the prediction is routinely in error by a whole grade or so. (See my post Physics Exams Too Easy, Says Ofqual

Now, this would not be a problem if these figures were just another piece of the puzzle to be understood by the teacher, but OFSTED, the government overlord of teaching standards, thinks students should know these rough predictions, and be challenged to achieve them. And leaned on if they don't come up to scratch.

Once upon a time, I got to know my own students, and made judgements as to their individual abilities and potentials, and assessed their effort accordingly. Not perfect, but at least both teacher and student were in the loop.

Forget the Child – Press the Button and Set That Target

Now, each student is given a grade to achieve by the end of a two year course, during which they will mature and develop. If they are very lucky, they will get several target grades which take into account the historical difficulties of each subject they are studying. If not, as is happening more commonly now, they will get a single grade to span the range from Photography and Media to Chemistry and Maths. And to make the target aspirational, a grade will be added to ensure that only a quarter of students will be able to meet their targets, with poor reports and disciplinary procedures for those souls unlucky enough to keep missing impossible targets.

Advantages

• Simple and cheap to operate.
• Keeps OFSTED happy.

Disadvantages

• No educational merit.
• Can turn keen students into serial target-missers.

An open and shut case for school managers. Shame about the children.
Read more!