Bad Stats, not Bad Teaching

I have complained before that many "scientists" unknowingly arrive at erroneous conclusions because they are incompetent at statistical analysis. Bill Gates advisers on education reform are no exception. After spending two billion dollars on research the Gates Foundation has arrived at an explanation for poor student performance -- bad teachers. The key finding:

"Research shows that there is only half as much variation in student achievement between schools as there is among classrooms in the same school. If you want your child to get the best education possible, it is actually more important to get him assigned to a great teacher than to a great school."

Read about it here.

Sounds good, doesn't it? An emphasis on teacher skills leads naturally to a whole slew of policy prescriptions and Gates obviously wants to put them into place, after all he's spent 2 billion dollars coming up with them.

But not so fast. John Hawks, who really does understand statistics, points to an obvious flaw in the Gates analysis.

So let's say a typical elementary school has 20 students in each class and 4 classes in each grade level. What Gates is saying is that the variance in means between samples of 20 students within a school is greater than the variance in means between samples of 80 students among schools.

Uhh...the standard error in a sample of 20 is twice the standard error in a sample of 80. Seems like Gates' point is pretty easily explained by statistics without blaming the teachers.

Read it here.

Nice catch! I guess that the "scientists" who have been spending Bill Gates' money skipped class the day they covered ANOVA in Stat 101, or maybe they just had a bad teacher.