Rotherham spins TFA stats

Andy Rotherham likes to portray himself as a voice of moderation and reason in the rough & tumble world of education debate.  But he is anything but.

Here he complains about a recent study by the Great Lakes Center which finds, among other things, that Teach for America (TFA) participants don't stay in the teaching profession for long.

In fact, in a study that delineated the leaving issue more effectively, a 2008 study by Harvard’s Project on the Next Generation of Teachers, found that 61 percent of Teach For America corps members stay in teaching beyond the two-year commitment. Teach For America surveys its alumni regularly and the most recent survey found that 65 percent of Teacher For America’s 20,000 alumni remain in education, with 32 percent continuing as teachers.

Let's take the second source of opposing evidence first.  TFA's own reported percentages of almuni who are still teaching is not exactly a reliable source.  But leaving sceptism aside, it's not clear what "32 percent continuing as teachers" really means.  The report suggests, and Rotherham interprets it to mean, that 32 percent of all alumni are teaching now, that is, currently.  If so, the following little note found at the very end of the report would be pertinent:

Percentages that reflect current data—as opposed to cumulative data—are drawn from our 2008 alumni survey, which received a 57 percent response rate [my emphasis] and went out to our alumni from corps years 1990-2006.

So, 32 percent of the 57 percent we know about, or 18 percent, are still teaching.  Maybe some of the other 43 percent are, too, but it's probably less than 32 percent since the non-respondents are probably less likely than respondents to still be teaching -- or to still be in education for that matter.  If a TFA alum is still teaching, it's a feel-good thing to respond to the survey.  But if she's in business consulting?  Well, maybe not so much.   In any event, a survey with a 57 percent response rate is highly unreliable, and Rotherham wouldn't treat its results as worth much if he were the stickler for hard facts that he presents himself to be.

As for the Harvard Study, his second source, this, too, had a response rate problem: with only 62 pecent of TFA alums responding. Which way does that bias the results? As with TFA's own survey, the most plausible story is that non-respondents tended to be earlier leavers, since they wouldn’t be quite so proud, so this would bias persistence rates upwards.

Besides failing to note the potential bias, Rotherham cleverly cited only the 61 percent of TFAers that the study found continued teaching more than two years.  He failed to mention subsequent attrition.  The study found that only 35 percent of the sample of TFAs continued after 4 years, and only 25 percent were teaching after 6.  Given the likely bias, that 25 percent is probably an upper bound estimate.

It seems that either Rotherham is not such an astute and critical consumer of research, or he is not such an honest broker, after all.