« Bush's Moral Values | Main | Security, social and otherwise »

December 21, 2004

Comments

Sebastian, without commenting otherwise on your post, the "missing" 8% is almost certainly people who said "I prefer not to answer" (in some form or another).

Thanks for blogging about this, Sebastian! I have a dumb question for all of you OW people who seem to understand surveys and polls. A Muslim discussion list that I co-moderate is currently discussing the Cornell survey and a European survey which was the subject of a London Sunday Times article. The article misleadingly claimed that the survey showed that Europeans were increasignly anti-Muslim. Actually the survey's questions asked these Europeans to opine on whether their country had become increasingly anti-Muslim. What is the point of asking people questions on which they could not be expected to have a reliable or verifiable answer? Why don't they just ask them for their own opinions rather than to ask them to project their own fantasies and then report it as if it is scientific? This is once again making me Confused in Cairo. I am hoping OW can help me understand this.

I am really happy that you have deciphered the Cornell survey and that it is not as scary as the articles have been saying. I will point people on my list towards your post.

I note that SH added conditions to his interpretations of the questions that weren't present in the questions themselves. Of course we want bad guys caught and bad organizations watched irrespective of whether they are Islamic or not. But the questions refer to any Muslim or any mosque.

By his projections of guilt upon entities Islamic, I have to assume that SH would himself be part of the 44%.

Does anyone remember those (possibly apocyphal) polls in which people were shown, absent their context, the freedoms outlined in the Bill of Rights and were asked questions about them, and an alarmingly high percentage responded that those things sounded like a recipe for Communism, and were probably a bad idea? This sounds to me a lot like that. Given the opportunity to opine, a large percentage of your fellow citizens would support restricted freedoms for everyone, depending on the freedoms being discussed.

It would be like forcing everyone with a gun to constantly report their whereabouts because someday they might shoot someone. 

Heh. Many people want something not very different from that.

mp, come on, that's a cheap shot. It's always a valid critique of a poll to look at a question and see whether there would be room for confusion. Is it valid to interpret that some might read question D and think it refers to SOME Muslim charities; i.e. this is something we should do if we have probable cause? Yes; the question is ambiguous and you can always count on people to misinterpret signals. But there is no way to avoid the bigotry of answering question A in the affirmative, which Sebastian points out. Nothing in this post supports your attack.

Anna, asking someone "do you feel that your country is increasingly anti-Muslim?" isn't asking them to "project their fantasies"; it's asking them for their perceptions, which is very important from a data-gathering standpoint, especially in conjunction with measured data. To give an example based on the survey mentioned in Sebastian's post, it would have been interesting to see not only "do you think that [whatever] is a good idea?" but "do you think that the rest of the country thinks that [whatever] is a good idea?".

The reporting, on the other hand, sounds questionable. (The Times requires that I pay for the privilege of reading their article, so I decline to do so.) A preponderance of "yes" responses to "Do you feel that anti-Muslim sentiment is rising in your country?" cannot be reliably read either as "Anti-Muslim sentiment is rising in the country" or as "Anti-Muslim sentiment is rising in these people"; the only non-biased reading is "these people feel that anti-Muslim sentiment is rising in the country".

In addition, I'm interested in finding out the context of the surveys in question - how were the interviews taken, and what were the settings involved? In particular, are the interviewers taking into account socially-motivated responses? (As an example, I've worked on studies linking aggressive and "anti-social" behavior to drug use in teenagers in Latin America. The primary investigator and the interview designers discovered that they had to use paper interviews, separate the respondents, and make it very clear both verbally and in writing that the answers were confidential in order to ensure accurate responses... because the social environment, which values aggressive, tough-guy (or tough-girl) behavior, prompted the teens to over-report their behavior "problems" in order to impress their peers or the interviewers, as long as the respondents were in a context where they were either responding directly to a live interviewer or among people they wanted to impress. Removing the social context provided more honest answers, because the teens didn't feel that they had to impress anybody. Of course, then we had to control for under-reporting because some of the respondents felt like the "official" feel of the survey environment meant that if they said what they were really doing, they'd get in trouble...)

Here is the text of the article since the Sunday Times won't let you access it.

Anna

December 19, 2004

Muslims face rising suspicion in Europe

By John Elliott
The Sunday Times - Britain
http://www.timesonline.co.uk/article/0,,2087-1409301,00.html

EUROPE’S reputation as a tolerant haven for people of all religions may be under threat. Muslims are apparently being viewed with high levels of
disapproval, a survey has revealed. Sweden emerges as the most pessimistic west European nation; 75% of Swedes questioned said there is “definitely a lot” or “rather a lot” of disapproval of Muslims.

Britain is relatively tolerant. Just 39% of respondents said they believed that a significant number of people were opposed to Muslims.

The survey was conducted in the wake of the September 11 attacks and the Madrid train bombings to examine religious attitudes. Researchers asked 1,000 people in 19 European countries: “Do you think nowadays there is a lot of, a little of, or no disapproval of Muslims living in European society”.

In Holland, 72% said there was a lot of disapproval of Muslims and in Denmark the figure was 67%. On average, 52% of people interviewed across western Europe believed that there was large-scale unhappiness about
Muslims.

An estimated 13m Muslims live in the 25 countries of the European Union. On top of fears over Al-Quaeda attacks, tensions have risen in countries such as Holland where Theo van Gogh, the radical film-maker, was stabbed to death by an Islamic extremist. This was followed by at least 13 arson attacks on mosques, churches and Islamic schools across the country.

The study also suggests that anti-semitism has increased; 30% of those questioned believed that it had risen in their country over the past five years, 35% thought there had been no change and 14% believed it had fallen. In Britain 34% of those asked believed there was more anti-semitism while 49% felt there had been no change and 17% thought it had declined.

Mark Hofmans of GfK Worldwide, the German market research company that prepared the report with The Wall Street Journal, said he had been surprised by the results.

“Most shocking were the results about anti-semitism and attitudes towards Muslims, which were much more negative than we expected and widespread, too,” he said.

Chowdhury Mueen-Uddin, deputy director of the Islamic Foundation, a Leicester-based centre that promotes understanding between Muslims and non-Muslims, said the study “reflects the general perception now prevalent
in our society”.

He added: “Anyone who has the wellbeing of society uppermost in his mind cannot but feel deeply concerned at these findings. Being a Muslim I am worried that Europe is replacing its anti-semitism with yet another cancer —Islamophobia. There is nothing worse than the feeling that you are not trusted or are viewed with suspicion by your neighbours and fellow citizens.”

He added that Muslims viewed Britain more positively than other countries in Europe: “We feel we are much better treated here than anywhere else — the society as a whole is much fairer than the other European countries.

“In France for example (Muslim girls) are denied wearing the hijab in schools and in Germany, while the Turkish population have been living there for more than 30 years, they are still treated as guest workers and not allowed to be part of mainstream society.”

Azher Basharat, 35, a shop worker and devout Muslim from Forest Gate, east London, said: “Things have got a little worse since September 11, but in the main things are fine. People in Britain are very tolerant.

“What has changed, though, is that some people look at us with greater suspicion. You can see it in their faces — they look a little uneasy. I’ve not been attacked or anything, though — and I can’t remember anyone saying anything racist or Islamophobic to me.”

I wonder how the results would change if instead of 'citizen', they substituted the word 'American", or lead with 'American' in referencing Muslim? I was pretty enthused with the 27%, given that most polls are skewed to achieve the results they imply and that many of those polled get what news they get from MSM. Apparantly the presentation of this poll by the AP continues to prove my point. Thanks, Sebastian for the breakdown.

I had the same reaction as Phil to this. It seems that there are always plenty of nuts who, as a matter of principle, do not support civil liberties.

What is the comfort in knowing that only 25% (as opposed to a higher percentage around 45%) support curtailing civil liberties? I agree that AP got its reporting wrong since the reporter was obviously looking for the hot lead. the underlying report is still disturbing.

As an American Muslim, and even if I weren't one, I do indeed think everyone who's commented agrees that the 20 something figure is indeed scary. It's just not quite as scary as 44%. And the reporting of polls routinely drives me crazy. it seems that you have to have unlimited time and a master's degree in statistics just to decipher what is really going on because journalists routinely distort and misreport it. Not to mention the issue of what is the point of the poll in the first place (like that European one - I understand there might have been a reason for it but this was not conveyed very clearly to the guy who summarized it for the Times article).

I think blogbudsman's point is critical here.

I wonder how the results would change if instead of 'citizen', they substituted the word 'American", or lead with 'American' in referencing Muslim?

Even with my circle of Muslim friends and my Muslim partner, when I'm aware that I'm about to be questioned on "Muslims" in America, the first mental image I get in my head is the photos of the 9/11 hijackers. That is quickly replaced with images of families in and around Detroit, then with professors at universities, and then, somewhere much later with my friends and partner. I consider this conditioning by the press (as well as thinking of my friends as my friends first and remembering they're Muslim somewhat later), but if that's my thought process, I wouldn't be surprised to learn it's the same for a majority of Americans.

If the questions were stated like, for example:

All Americans who define themselves as Muslim should be required to register their whereabouts with the federal government

I suspect the percentage would be lower.

About the European poll: it is interesting to ask people whether they think their country is becoming increasingly X, so long as you bear in mind that the reasons why large groups of people answer 'yes' can include not just (a) that their country is, in fact, becoming more X, but also (b) that they are more sensitive to any suggestion of X than they used to be, or (c) (for bad Xs) that their sense of what level of X is a serious problem has changed, and thus they are more likely to think that they have a real problem with X, and (d) they assume, for no particularly good reason, that other people in their country, whom they've never met, have a problem with X, perhaps because of distorted media coverage. (In this way, a survey showing that Americans believe there are more crimes committed against photogenic pregnant women than there used to be would not obviously be evidence that this belief was true.)

Thus, I spent six months in Sweden in the early 80s, and at that time it was an article of faith, both among Swedes and among outside observers, that Swedes had a problem with drinking. All those long winter nights, people would say; that gloomy country. And there were plenty of surveys that showed that more Swedes than citizens of most other European countries believed that Swedes had a problem with alcohol. But when, on a whim, I looked up the actual alcohol consumption figures, it turned out that Swedes were nowhere near the top of the list of European countries in terms of per capita alcohol consumption. So, for whatever reason, they regarded as a big problem levels of alcohol consumption that, say, the French would have regarded as rather abstemious.

This is not to minimize European anti-Muslim sentiments -- I have no idea how extensive they are -- but just to make a general point about these sorts of surveys.

I think the 27 percent figure is scarier, if anything. Limiting civil liberties is a really vague term, and given the public's measured hostility to their OWN constitutional rights when polled, it wouldn't suprise me. 27 percent want to treat all Muslims like sex offenders in a well-nigh unambiguous question, though, and those people are incredibly frightening.

I don't want to downplay the 27% at all. I generally expect that 5-8% of people will believe almost anything ridiculous, so it is disturbing to me that we have 20% more than all the crazy people thinking that Muslims should register with the government. But that is still a very different world from nearly half.

agree with both carpeithcus and SH.

In general, news reports and editorials that cite polls are awful. So many polls depend so heavily on how the question is phrased, that it is irresponsible to report their results if you do not give the exact wording of the question.

Anyway, almost any poll I can think of is better summarized with a chart than with a narrative news article. It's both quicker to read and more informative.

A sideline, though no less important issue here is the reason why polls like this are reported as they are.

Journalists are depicted as overwhelmingly liberal with a subset (FOX, Wall Street Journal editorial page to the extent that it "reports") depicted as overwhelmingly conservative.

A categorization which makes more sense is (and has been since majoring in journalism years ago) that journalists are a subset of "businessmen" who indeed sell their wares with the hot headline. That headline changes over time according to what journalists perceive as saleable at particular times, thus many stories today "seem" to me to be selling to "my" (general term) increased xenophobia, fear of 9/11, my insulted Christian sensibilities, and "my" outrage at taxes, government, etc.

I am none of these things, of course, but journalists, like car salesmen and saleswomen, think I might be more likely to buy a car today with anti-theft features rather than one with low gas mileage, unlike 25 years ago. And they throw in the expensive useless undercoating just the same.

So, polls and headlines must be deconstructed, as Sebastian does here.

But it's getting worse. Why?

My crackpot theory is that professions which (in mythological times) used to adhere to various oaths (medicene, law, journalist) of responsibility now concentrate only on the bottom line. It's strictly business now.

I suspect that polls used to be reported with press because graphs and charts used to be so expensive to create and print. Now that almost anyone can whip one up in 15 minutes, and now that printing techniques don't require the crazy typesetting of before, I don't think there is a reason to avoid graphs in the reporting of polls.

Of course when that happens, I can start complaining about deceptive scaling issues.

The problem with press reporting of polls is pretty much the same as the problems with science reporting in general -- journalists by and large (a) don't have the background to understand it thoroughly; (b) don't have the time to really dive into it and at least get a solid layman's understanding; and (c) are under pressure to generate a snazzy headline, even if the actual results are more tenuous and vague.

But it's getting worse

Is it? If so, maybe just because there are so many more polls to choose from these days.

The headline writing these days is near criminal. USA today blares
http://www.usatoday.com/money/workplace/2004-12-20-fsa-usat_x.htm/>Workers Lose Health Care Funds when after reading the article, very little is lost by individuals and the situation is improving. It's deceitful and it's intentional and the motive would have to extend beyond mere profits.

One more chance...http://www.usatoday.com/money/workplace/2004-12-20-fsa-usat_x.htm>POW!

Anna,

Pollsters tend to ask such indirect questions when they believe people will hesitate to respond to a direct question with the truth.

For instance, "Do you feel that your country increasingly anti-Muslim?" would often get far a more accurate response than "Are you anti-Muslim?"

"I don't have a problem with them, but a lot of people I know do."

I don't know what the splits are (and it is likely that they don't break out so neatly), but I note that another way to say "27%" is "the majority of a 51% ruling coalition." Given what Bush voters had to overlook on civil rights to pull the lever for him, I think people on this thread are strangely hopeful about the country's willingness to trade civil rights for a misperceived increase in security.

here is how poll results should be reported.

Highlights: on the general job approval question, 27% strongly approve of the job Bush is doing, 21% somewhat approve, 12% somewhat disapprove, 38% strongly disapprove, with 2% expressing no opinion.


Asked whether they approve or disapprove of the job Bush is doing on specific topics, you get this:
The economy: 46% approve, 51% disapprove
The situation in Iraq: 42% approve, 57% disapprove
The US campaign against terrorism: 53% approve, 43% disapprove
Social Security: 38% approve, 52% disapprove
Health care: 37% approve, 56% disapprove

4 thoughts:
1. So the order of a shrill is a plurality. :)
2. damndamndamnIcan'tbelievewelosttothisguy
3. Democrats who defect on social security should be given no quarter
4. If we're well-led, there is a small but real chance to make 2006 their 1994.

"I'm a bit confused about why the people who agreed with zero plus the people who agreed with one plus the people who agreed with two or more equals only 92%, but I'll put that aside."

8% of people said "no opinion", "I don't know", "stuff it" or some variation thereupon.

Personally, I'm in the "none of the above" catagory. The closest to reasonable seems to be the "moniter closely" option, but monitering too often turns to interfering and even entrapment or provoking illegal actions. I'd rather stick with innocent until proven guilty and have the FBI concentrate on montering--and interfering with--groups that actually do or threaten to do something illgeal--whether the group be Islamic, Christian, white supremicist, Weatherman, Basque or any other cause.

I think Sebastian has hit the nail pretty much on the head here, although I, personally, wouldn't have given as much benefit of the doubt to D as he did. But that's merely splitting hairs.

Hilzoy, that is what I thought when I saw the questions. I thought if people are becoming more sensitive to a given issue and therefore see it more, it may actually be improving rather than getting worse. Mario, that explanation also occurred to me but it seemed to me it would have been fairly simple to ask questions less direct such as "do you think Muslims in your country are more loyal to their religion than to the country?" or stuff like that to get a feel for whether people "trust" them or not. To all, thanks for educating me on this; I feel better equipped to read about polls after having had this conversation.

The comments to this entry are closed.

Blog powered by Typepad