Saturday, September 17, 2011

Tossing the bathwater, keeping the baby

Robert Marzano may become the Jean-Baptiste Lamarck of education.

Lamarck was the guy before Darwin who thought giraffe babies had longer necks than their parents because their parents strived to reach leaves. This "inner need" drove evolution.

Lamarck gets short shrift now, I think unfairly--he was wrong, true, but he proposed an idea that was testable, and he got folks doing just that.

Marzano, too, has some good ideas, and he's even convinced a lot of people that his ideas are founded on  "research," using meta-analysis to draw up some enticing numbers:

I've read Marzano's explanation for his "percentiles," extracting them from "average effect size," in order to "provide[s] for a dramatic interpretation of the possible benefits of  a given instructional strategy."

Lumping various studies, some of dubious quality, under the umbrella of meta-analysis, and then applying a further manipulation to amplify already questionable methods leads to the graph above.

And now teachers around the country are getting the graph above shoved under our puppy-dog noses like a soiled newspaper, while some high paid folks intone "Do this and our state scores will rise 45 percentile points."

Marzano is either stupid or disingenuous, and I doubt he's stupid. Making a case that a particular strategy will raise student scores 45 percentile points is like saying that a particular diet will add 45 years to your life. A lot of people will try the diet, and most will even be better for it. But very few are going to add many years to their lives.

So, yeah, "identifying differences and similarities" can improve student learning outcomes. But anyone who teaches kids already knows this. Really.

All the snake oil in the world is not going to get us a 100% pass rate on the 2014 NCLB tests, but some folks are going to make a lot of money telling us how to get there. Administrators are under tremendous pressure to do what's literally impossible.

Come on, PLC, make the case for Marzano's numbers. I'm calling bull crap.
Quotes and graph taken from Marzano'/Pickering/Pollock's Classroom Instruction That Works: Research-Based Strategies for Increasing Student Achievement 

Cult of personality seems to run deep in education. 
Don't even get me started on Howard Gardner


John T. Spencer said...

Marzano is the world's highest-paid collage artist. Nothing original. Ever.

I've started asking people not to use his name when they quote "his" strategy. For example, "We do Marzano's vocabulary." Really? So you have the kids draw a picture, write it in their own words, do an example, a non-example? That's been around for years. I did that as a kid. Marzano doesn't own it. Really.

Anonymous said...

I agree 100% with Mr. Spencer. I remember doing those types of activities for vocabulary all through school.

I am concerned about how Marzano compiled all that data. I have seen his books plastered all over my school's adminstration offices and our instructional coach hands them out as presents to new teachers. But reading his books always leaves me a little wanting. As a science teacher (and as someone who worked in the veterinary field for 4 years before joining education), I need to see the actual studies and conditions of those studies. How can we know that the studies were reliable?

David Rudel said...

Mr. Doyle, I wonder if you think there is some value in Marzano's "results" in that they might at least suggest which strategies have more potential than other strategies, or do you think that the methodology is so flawed that even this qualitative deduction is unsound?

doyle said...

Dear John,

It's interesting, isn't it, though to be fair, Marzano doesn't claim ownership of the strategies. (He did trademark a benign phrase--I need to go look it up again.)

Dear Anonymous,

Ayep--my experience in medicine with meta-analytical studies makes me wary,very wary.

Dear David,

The reason Marzano has glommed so much attention is because of his staggering claims.

That some strategies are more useful than others in some circumstances is not surprising, and their relative value makes for interesting lounge chat, but any teacher worth the title will have most of these strategies and many others in his or her bag of tricks.

(The studies he cites for homework hardly support its use in the lower elementary grades, but I bet many admins never look at those studies.)

At any rate, Marzano's slickness presenting "his" data is what makes his work appear valuable. Admins salivate at "45 percentile" jumps, not so much at qualitative rankings.

Thanks all for dropping by!

doyle said...

Dear David,

I have not yet finished the post for the review. Been a nutty week!

In the meantime, though, everyone who pokes around here should click on David Rudel's name above--his books on misconceptions rock!

David Rudel said...

aw shucks! *blush*

Mary Ann Reilly said...

Wonder what longevity is if you feast on a steady diet of Marzano books? Death at an early age?

I have never gotten through an entire book as I drift away, fall asleep, start hearing the teacher's voice from all those Peanut's movies (Blah Blah Blah), and thinking about late night TV ads.

It's all about sales. He's selling myths, dreams, stuff you can never actually have.

It's not about learning (or teaching for that matter).

A Shared Meal said...

I got in a heated debate two weeks ago with an instructional coordinator about needing evidence for some learning theory. She said I see it in the classroom that's evidence enough. Until there is real science in education theory we will continue to have these debates.

Tyson said...

Marzano and Gardner--top two on my list of education charlatans. It's exasperating that even when Marzano's own analysis of the studies suggest little or no effect for a particular strategy (i.e. elementary homework), he still supports them. How is that research-based?

Every time I read your blog I find myself either already in agreement with much of what you have to say, or being forced to reconsider some of my previous conceptions. Thank you.

doyle said...

Dear Tyson,

Your last line is about as good a reason to blog as any.

Thanks for the kind words.