Belly of the Beast

Where have I been for the last 6 months?

Refreshing Nate Silver’s fivethirtyeight blog ten times a day, like a rat hitting the treadle for morphine. Silver’s last electoral map projections are still up on his site: 50 for 50. He’s left it up like a war trophy—a scalp. And the scalped head in question is most likely Gallup’s. Silver has been beating Gallup like a rented mule all election season, and with good reason: it blew the last two election cycles badly and was a major Romney-leaning outlier all the way to the end this time.

“It was one of the best-known polling firms, however, that had among the worst results. In late October, Gallup consistently showed Mr. Romney ahead by about six percentage points among likely voters, far different from the average of other surveys. Gallup’s final poll of the election, which had Mr. Romney up by one point, was slightly better, but still identified the wrong winner in the election. Gallup has now had three poor elections in a row. In 2008, their polls overestimated Mr. Obama’s performance, while in 2010, they overestimated how well Republicans would do in the race for the United States House.”

Why has Gallup been getting it wrong? Like Mr. Romney and his party, it’s a bit stuck in the 50s, depending on land-line phones for its polling.  “Research by polling firms and academic groups suggests that polls that fail to call cellphones may underestimate the performance of Democratic candidates.

The roughly one-third of Americans who rely exclusively on cellphones tend to be younger, more urban, worse off financially and more likely to be black or Hispanic than the broader group of voters, all characteristics that correlate with Democratic voting.”

Well, duh.

But there are laws restricting access to cell numbers and the ability to make unsolicited calls. So what is a polling outfit to do? The big winners  this year were the research firms

who depended partly or entirely on online polling, which is where most Americans do their fact-finding and opining.

The stunning obviousness of this methodological flaw is

matched only by the glacial slowness with which traditional research firms are changing to adapt.

All of which got me to thinking about advertising research, which in 2012 is still rounding up the usual underemployed or retired suspects, sticking them in conference rooms with 2-way mirrors and showing them animatics or “adlobs.” It is as stuck in the 50s as Gallup, which itself had its roots in advertising research, and the reliability of the results is just as suspect.

If the triumph of Nate Silver teaches us anything, it’s that good data isn’t about the answers people give. It’s about who is giving the answers, and who’s asking the questions.

No one here but us old white people.