The Dread "Outlier" Syndrome

by publius

The new CBS/NYT poll has Obama +14 among likely voters. The McCain camp calls the new poll improbable. At Hot Air, Allahpundit notes that it’s a “hefty outlier” and provides the pro-Dem party ID breakdown to provide some comfort (though admittedly while conceding it’s bad news).

Ah, I remember these games. Whenever a bad Bush/Kerry poll would come out in 2004, we’d learn from someone that the party ID was off, or that “likely” voters didn’t really matter this year. Or whatever. Opium for the massly deluded, that stuff.

Kaus pretty much had the best response. The day after the 2004 election, he wrote:

November 4, 2004. Bush 51, Kerry 48: Pollster Ruy Teixeira demands that these raw numbers be weighted to reflect party I.D.! ... P.S.: And they probably only included "likely voters"!

There’s two explanations for an unfavorable poll — (1) the poll is defective; or (2) your candidate is losing. I clung quite tightly to #1 throughout 2004 — you might even say I developed a problem. Seriously, I can quit drinking relying on party ID at any time. No, I never look at party ID alone . . . with nobody else.

I’m not gloating about any of this. I — like every liberal here — went through this same water torture in the fall of 2004. It sucks. But take it from someone who's been through it, clinging to hopes of outliers and reweighted party IDs is just . . . well, a river in Egypt.

Comments

[Brandon Moore]

Check Nate's comments at fivethirtyeight. Just because people have tried too hard to deny bad news before doesn't mean you have to take every outlier at face value, not when you (Nate) have more data. It's good news and pulls up the best estimate of Obama's lead, but it's likely a bit of an overestimate (+9 is still pretty darn good).

[Warren Terra]

I agree with everything you write, and I remember clinging to the same threads in 2004. And it's really wonderful to have the other guy's backers loudly proclaiming "This poll is crap! We're behind by eight points, tops!" Still, I can't help but wish you had a beter way to illustrate it than with a Mickey Kaus quote. Don't get me wrong, it's a funny quote, and apposite to boot, but we're talking of a guy so loathsome that Bob Wright developed an entire business model around people watching Bob sarcastically mocking Mickey.

[John Spragge]

I followed the link to an article about a PAC that intends to run anti-Obama ads in Nevada, Colorado, and Michigan. I confess that I think that the Republicans have got behind the persuasion curve. In aviation, if you get into a certain flying configuration, the more power you add, the slower you go; we call this getting behind the power curve. If you get behind the persuasion curve, the more vigorously you make your case, the more annoyed the people you need to persuade get.

[publius]

kaus is a struggle for me. I agree with everything you said warren. but i also think he's incredibly smart and -- at times -- extremely insightful in a unique way. again, i basically agree. but that line i quoted above is pretty money.

[Gary Farber]

Presuming you meant to link here, Brandon Moore, Silver writes:

[...] Presently, our best estimate is that Obama has about an 8-point national lead. However, CBS polls have leaned about 3 points more Democratic than the average this year. In other words, our baseline expectation is that a CBS poll should be showing about an 11-point for Obama right now.

You wind up to the Obama side of the +/- 3 point margin of error, and that's how you get to 14 points.

Not that this is good news for McCain exactly -- the balance of polling over the past 48 hours indicates that Obama's true lead is probably more like 8 points (maybe even inching upward toward 9) than 6 or 7. But it's not quite as bad as it will look on the surface.

UPDATE: Yes, the topline number is indeed Obama 53, McCain 39, although the version that we prefer -- with third party candidates included -- gives Obama "only" a 12-point lead.

[Gary Farber]

"but that line i quoted above is pretty money."

Pretty money? What does that mean? (Clearly, something good, but where does this expression come from?)

[brent]

Well in 04, the underlying demographic expectations were a bit difficult to properly estimate. I think, for instance, that Democrats had a lot of reason to believe that we were motivated and would turn out in significantly higher numbers than Republicans. If we had been right in those assumptions than our skepticism toward the polls would have been correct.

All of that is to say that its not necessarily foolish to question the methodology of the polls when they depend as much as they do upon assumptions about likely voter models and party id that can very easily be wrong and shift the results 3-5 points either way.

I suspect Obama is well ahead. 14 points seems a little unlikely to me just because I don't suspect that anyone will ever win a modern election by that amount ever again. Political polarization just seems to much of a factor to me to have anyone win by double digits anymore. But 7-9 points. That I could see.

[Andrew]

Pretty money..."Money quote"...that quote is "money". If not completely "money", then pretty "money" at least.

I guess.

[flyerhawk]

Point of reference. Here is electoral-vote.com's aggregated projections for Oct 14th 2004. They projected Bush to win 284 EVs, he got 286.

They currently project Obama to win 357.

Aggregate poll numbers are very accurate. Now they may change over time as voter's change their minds collectively but they are generally quite accurate.

BTW, the posters over at HotAir are completely unhinged.

[Gary Farber]

"BTW, the posters over at HotAir are completely unhinged."

Thanks for the tip. They're not only unhinged, but eating their own.

Too funny. They're denouncing their own folks as crypto-leftwingers for daring to link to commie outlets like the NYT and CBS.

And counting on "two million to seven million" PUMAs.

Sheesh.

[snarkout]

Pretty money? What does that mean? (Clearly, something good, but where does this expression come from?)

Gary, if you're still reading this thread, I believe it comes from Jon Favreau's (quite enjoyable) first movie, Swingers. It's what Vince Vaughn's character uses throughout as a synonym for awesome: "You're so money, and you don't even know it!"

[snarkout]

(YouTube clip above NSFW due to language. But money!)

[theCoach]

I was drinking the Ruy as well -- I wanted to believe.

However, this year there are some really interesting polling problems in play. Unfortunately for team McCain they almost all point in the direction of Obama (new voters, registration numbers, ground game, reverse bradley, etc.) Nate's road trip might be telling the story of this election -- the grassroots ground game has changed, and it will be interesting to see what comes out of that. It seems far more likely at this point that Obama coattails to 60 in the Senate than McCain makes a comeback -- 59 is probably the over/under.

[ptet]

Thanks for the Hot Air comment tip too. Genuinely hilarious.

"I’m starting to think that the MSM is deliberately trying to set up a race riot or something. When McCain wins, and he will win, Obama’s minions will believe the election was “stolen” based on all the polls. All hell could break loose. The question is why would they want to do that? What would they gain by misrepresenting the polls then losing?"

Forget CERN. The amount of paranoia there could rip a whole in the space-time continuum...

[wonkie]

The poll is high, but what the hell. These are the glory days. i intend to enjoy them.

According to a poster on Swampland Obama is expecting the polls to tighten as we get closer. His campaign strategists think that the white male vote will shift back toward McCAin but that thhe surge in women willhold.

My worry is voter surpression. Obama can win if he wins beyond the levelo foRepubican voter surpression, which is well underway in several swings states, notably Ohio.

[voxpoptart]

Look, it's about time you answered this publicly: why the heck are you so sure the exit polls of 2004 were wrong, and that Kerry did not in fact receive the most votes?

The case for the stolen 2004 election has received several writeups involving a wide variety of specific evidence that looks, prima facie, pretty convincing: I've read Patrick Kennedy's Rolling Stone article and Greg Palast's Armed Madhouse, but there's also Steven Freeman's Was the 2004 Election Stolen? and Mark Crispin Miller's Fooled Again if those are names that seem more trustworthy. I don't KNOW that their cases are true, but standing unchallenged, they look to me very much like truth. I've looked for specific refutations of their cases, but I've never found any: they just get dismissed out of hand.

You and Hilzoy have been among the dismissers. You're generally far better than that. What exactly is wrong with the evidence of theft - and why shouldn't be on the watch for it again?

[ThatLeftTurnInABQ]

Nate's road trip might be telling the story of this election

I echo that sentiment, with the important modifier that credit needs to go to Sean Quinn for being the author of those 538 roadtrip stories.

[ThatLeftTurnInABQ]

One thing that "people watching at home" need to be aware of on Nov 4th is that the Obama campaign has put out an enormous effort to get early votes, which may in turn skew the results of election day exit polling.

If a sizable percentage of Obama voters are not going to the polls on Nov 4th because they already voted, they won't be interviewed by the TV networks and the resulting statistics reported on TV may be biased in favor of McCain. This will lead to a situation opposite to that of 2004, where Dems were expecting a Kerry victory in key states based on exit polling data and the actual margins were influenced the other way due to absentee ballots (which normally skew heavily pro-GOP).

This year the shoe will be on the other foot. But that also means that news stories you hear on election day which are based purely on exit polls may be less pro-Obama than they should be.

I am especially concerned that this will effect the sort of demographic analysis stories the TV networkds love ("Wolfe, Obama is doing poorly amongst left-handed Catholics with hip transplants in Ohio...")

[Anarch]

I echo that sentiment, with the important modifier that credit needs to go to Sean Quinn for being the author of those 538 roadtrip stories.

I rarely mention this, but I have to say that I'm seriously impressed at 538 not just as a site in itself -- which it is -- but as the kind of expert, serious analysis that can't be found in other media. The blogosphere all to often falls short of its potential, but when it reaches it... damn.

[ThatLeftTurnInABQ]

Bwahaaaaa! Republicans head for the lifeboats, this ship is going down:

Midwest focus group: Your ad made us think Obama is a terrorist and we're voting for him anyway!

Yes, the spot worked. Yes, they believed the charges against Obama. Yes, they actually think he's too liberal, consorts with bad people and WON'T BE A GOOD PRESIDENT...but they STILL don't give a f***. They said right out, "He won't do anything better than McCain" but they're STILL voting for Obama.
...
I felt like I was taking crazy pills. I sat on the other side of the glass and realized...this really is the Apocalypse. The Seventh Seal is broken and its time for eight years of pure, delicious crazy....

[Anarch]

Wow. Just... wow.

Note to Democrats: Don't. F***. This. Up.