Poll Mania

The meme suggesting that Bush won largely on the strength of anti-gay backlash seems to be undergoing a rather thorough debunking. See for example Kevin Drum, Andrew Sullivan, Slate, and David Brooks (NYT Nov. 6). Andrew puts it most succinctly:

The percentage of people who said in 2004 that their vote was determined by the issue of "moral values" was 22 percent. In 1992, if you add the issues of abortion and family values together, that percentage was 27 percent. In 1996, it was 49 percent. In 2000, it was 49 percent. So the domestic moral focus halved in 2004. Obviously, the war took precedence, especially if you combine the categories of the Iraq war and the war on terrorism more generally. Again: the Republicans should be wary of over-playing their hand. If they believe the entire country is the religious right, the backlash could begin very soon.

I'm not going to try to wade in further. My initial intuition is that the gay issue may have played a slight role in the election, and that in this election every slight role could be looked at as decisive.

Instead I want to talk about my problems with polls. This may become a quarterly project, but I really want to convey that people put far too much stock in polls. Lots of polls play into our own preconceptions.

For instance look at the poll which asks "Generally speaking, would you say things in this country are heading in the right direction, or are they off on the wrong track?" According to this poll, lots of people, for quite some time, believe that the country is heading on the wrong track. You might look at a poll like that and think: "I believe that the country is on the wrong track, apparently lots of people agree with me." You would almost certainly be wrong if by 'people agree with me' you mean that people have the same reasons for thinking that the country is on the wrong track.

Suppose you think the country is on the wrong track because of our invasion of Iraq. There are some number of people who think we ought to have nuked Iraq. They probably think our invasion put us on the wrong track. Do you agree with them? Some people think that we should never go to war. They think that any war puts us on the wrong track. Do you agree with them. Some people think we should have used quite a bit more force and not worry much about civilian casualties. They probably think we have gone on the wrong track in Iraq. Do you agree with them? My point is that we assume that if people respond to poll questions like that as we would have, that they shared our reasoning. That almost certainly isn't true. My examples show that you could believe a number of wholly contradictory things about Iraq and conclude that our conduct put the US on the 'wrong track'.

Furthermore, you have to realize that huge numbers of people might not agree with Iraq as being an important factor in their 'wrong track' analysis. Maybe lots of people think we are going on the wrong track because states are resisting gay marriage. Maybe even more think that we are on the wrong track because we are even considering gay marriage. A couple of people may think we are on the wrong track because we aren't listening to the messages from Mars carefully enough.

Such problems are enormous in the relatively straight-forward 'wrong track' arena. But ask if 'moral values' are important in an election, and the range of possible reasons for saying "yes" is even more vast. If someone said "yes" were they talking about homosexuality, honesty, steadfastness, resolve, abortion, or some other topic which they personally found important? We don't know. And making assumptions about it because most of the ones who answered "yes" voted for Bush isn't sound analysis.

So when analyzing polls, it is important to remember that matching conclusions might not mean matching reasoning, and that ambiguous terms might not have been interpreted by the answerer in the same way as you would interpret them (or in the same way as the questioner meant for it to be interpreted.

Comments

[mac]

The null hypothesis here is that people vote for the incumbent during a war. I haven't seen anything at all to refute it: Bush won 51 to 48. He did a little better in the cities this year, and a little worse in small towns. This is just as well explained by regression to the mean as any kind of "trend."

In short, I think you might as well study chicken entrails or tea leaves than try to get anything out of the polls, other than the obvious of voting for an incumbent in the middle of a war. The exit polls also validate this, as people also agreed with big chunks of the democratic platform (better to balance the budget than cut taxes, skeptical of social security privatization, etc.)

[von]

The meme suggesting that Bush won largely on the strength of anti-gay backlash seems to be undergoing a rather thorough debunking. See for example Kevin Drum, Andrew Sullivan, Slate, and David Brooks (NYT Nov. 6). Andrew puts it most succinctly:

I dunno if it's a debunking. Sullivan, for instance, uses national data. The issue, though isn't how "social issues" played in California or Florida, but how they played in Ohio and the midwest.

[Gromit]

Buyer's remorse, Slarti?

Face it, you voted for Bush, and you got Dobson's theocratic juggernaut in the package. Whether fundamentalism was the whole story or not, without it we'd be talking about President-Elect Kerry's agenda for the next four years. Fiscal conservatives and intellectual liberals alike are loath to admit it, but a significant part of America was buying what Bush was selling: fear of the Other, Us vs. Them. Until we work together to counter this, instead of simply wishing it away, the theocratic lobby will only grow in power, because their message of inherent moral superiority is very attractive to folks whose dignity is being assaulted by a weak economy and by perceived vulnerability to terrorism (the truly vulnerable areas actually voted overwhelmingly for Kerry). This is turning into an era of picking on scapegoats when the real threats are much harder to address. The sooner we all recognize this, the sooner we can get to work addressing the real problems.

[Gromit]

Whoops, got my "S" posters mixed up. I meant "Sebastian". Sorry!

[Sebastian Holsclaw]

No buyer's remorse (at least not yet). I am intimately acquainted with a large number of people in the 'fundamentalist' movement. I'm not as scared of them as you seem to think I should be. I'm especially not all that freaked out by Dobson.

[Edward]

I am intimately acquainted with a large number of people in the 'fundamentalist' movement. I'm not as scared of them as you seem to think I should be.

Being equally acquainted with a large number of people in the "fundamentalist" movement, I actually am a bit scared of them. Something is changing in the church I grew up in. There's a really frightening blend of victimization and agression emerging. Feeling like they're "under attack" (from whom exactly is never quite clear), they're lashing out.

My father, who attends the same family of churches Ashcroft does, believes there's no way to reconcile the power grab he's seeing being backed by the church with the teaching of the church. Christians are meant to be fair and kind and generous. Nothing about the Bush campaign struck him as any of those things. My brother who is truly one of the kindest souls I've ever met, is becoming increasingly conservative and terribly impatient with the country's ability to reverse Roe v. Wade and criminalize homosexuality. And he's one of the gentle ones.

I ask my family what's up...why the agression, and repeatedly they offer that they feel they're under attack. Again, they can't tell me by whom, which leads me to believe they're being fed this nonsense by their church leaders.