I've mentioned this before in context of drug screening in high school students, but Bayes' theorem holds when analyzing tests that assess teacher performance as well.Suppose your district buys into Edumegacorp's teacher rating system, the Assessment of Holistic Learning Environments (the AssHoLE test). The test is 95% sensitive--it will detect bad teacher 95% of the time. It is also 95% specific--"good" teachers will pass 95% of the time. Sounds like a great tool, no?
Let's further suppose it's used in a district that uses the best teachers in the universe--99% of them are competent. 1000 of them are tested. Hey, a good teacher has nothing to fear, eh?
Do the math. The ten incompetents will be correctly identified--it's a sensitive test. 5% of the remaining 990, however, will also be identified as incompetent. About 50 teachers will fail their assessment. Only 10 of the 60 who fail the AssHoLE test are truly incompetent.
It gets better though. The tests being used to judge teacher performance were not designed to test teacher performance, they were designed to assess student performance in very limited tasks.
This is like biopsying your brain to see how your liver is doing.
Just saying....
The brain biopsy is via Wikimedia commons, with a share alike CC license--
I hope they asked the patient who's brain has been splayed open to the internet universe.
I hope they asked the patient who's brain has been splayed open to the internet universe.
Posts
2 comments:
I once had an administrator compare benchmark tests to the need to "weigh in" as a method for assessing health. I explained to her that, when I was at my healthiest, I had put on some weight as I began to lift weights, run and eat healthier.
I also explained that measuring health was tricky, because it requires a battery of tests to measure a ton of different things (many of which I don't understand) and that the tests are diagnostic tools rather than evaluative measures. Besides, a cognitive process seems more challenging to measure than physical health.
She looked confused, so I summed it all up, "So what you're really saying is that you want healthy kids, but instead of taking the time to get to know their health levels, we're best off adding everyone's weight and then comparing each class and blaming the teacher afterward, not taking into account any pre-existing conditions, height differences, genetic issues or muscle mass. You're right. We definitely need weigh-ins."
What I wanted to say was, "Don't fuck with me. I'm a man of metaphors, yes, but I pick them apart as easy as I create them."
Dear John,
You make me smile every time I read your comments--you are a pistol in the finest sense of the word.
I urge anyone reading these words to go out and buy your book. Years from now folks will be crowing about having a 1st edition John Spencer book (back when books were still printed on paper).
Post a Comment