Playing The Numbers

by hilzoy

From the Washington Post:

"The U.S. military's claim that violence has decreased sharply in Iraq in recent months has come under scrutiny from many experts within and outside the government, who contend that some of the underlying statistics are questionable and selectively ignore negative trends.

Reductions in violence form the centerpiece of the Bush administration's claim that its war strategy is working. In congressional testimony Monday, Army Gen. David H. Petraeus, the top U.S. commander in Iraq, is expected to cite a 75 percent decrease in sectarian attacks. According to senior U.S. military officials in Baghdad, overall attacks in Iraq were down to 960 a week in August, compared with 1,700 a week in June, and civilian casualties had fallen 17 percent between December 2006 and last month. Unofficial Iraqi figures show a similar decrease.

Others who have looked at the full range of U.S. government statistics on violence, however, accuse the military of cherry-picking positive indicators and caution that the numbers -- most of which are classified -- are often confusing and contradictory. "Let's just say that there are several different sources within the administration on violence, and those sources do not agree," Comptroller General David Walker told Congress on Tuesday in releasing a new Government Accountability Office report on Iraq.

Senior U.S. officers in Baghdad disputed the accuracy and conclusions of the largely negative GAO report, which they said had adopted a flawed counting methodology used by the CIA and the Defense Intelligence Agency. Many of those conclusions were also reflected in last month's pessimistic National Intelligence Estimate on Iraq.

The intelligence community has its own problems with military calculations. Intelligence analysts computing aggregate levels of violence against civilians for the NIE puzzled over how the military designated attacks as combat, sectarian or criminal, according to one senior intelligence official in Washington. "If a bullet went through the back of the head, it's sectarian," the official said. "If it went through the front, it's criminal.""

Here are some of the things we know about these statistics: they don't include Sunni-on-Sunni violence, or Shi'a-on-Shi'a violence. They don't include car bombings. There are unexplained changes in the figures from one report to the next. They don't seem to take seasonality into account.

More discussion below the fold.

The reason for not including car bombings is just ludicrous:

"Car bombs and other explosive devices have killed thousands of Iraqis in the past three years, but the administration doesn't include them in the casualty counts it has been citing as evidence that the surge of additional U.S. forces is beginning to defuse tensions between Shiite and Sunni Muslims.

President Bush explained why in a television interview on Tuesday. "If the standard of success is no car bombings or suicide bombings, we have just handed those who commit suicide bombings a huge victory," he told TV interviewer Charlie Rose."

Gosh. By the same logic, why include any deaths at all among the civilian casualty numbers? After all, people who kill people are often bad guys, and who wants to hand any bad guys a victory?

The reason for not including Sunni-on-Sunni or Shi'a-on-Shi'a violence involves a much deeper mistake.

Cast your minds back a few years. Back then, a lot of people thought that the only people we had to worry about were the insurgents. The growth of Shi'a militias -- well, that wasn't really a problem. (There are some pretty amazing quotes to this effect in this article by Spencer Ackerman, which I provide some excerpts from here.) We characterized the problem too narrowly: as I wrote back then, "The insurgency is the militia (or: group of militias) that's dangerous to our troops. All the militias are dangerous to Iraq." But we focussed only on the insurgency; as a result, we missed the chance to try to deal with the militias when they were much, much weaker.

Now, we're doing it again. The danger we should be worried about, it seems to me, is a civil war. One form that a civil war might take is: Sunnis fighting Shi'a. But that is not the only possibility. It's not even the only actuality. In Basra and large areas of the south, Shi'a are fighting Shi'a for control. The recent fighting in Karbala was Shi'a on Shi'a. If we do not focus on this, we'll be making the same mistake we made before: defining the problem too narrowly, and waking up one morning to realize that the part of the problem we were ignoring has become the main event. That being the case, to define the relevant set of casualty statistics to exclude Sunni-on-Sunni or Shi'a-on-Shi'a violence is to make a profound mistake.

There are various lists of articles trying to measure the actual casualty figures, or questioning the government figures. See, for instance, here, here, and here. But there is an easy way to resolve these issues. If the government were to release its figures, and explain the methodology behind them, then it would be clear whether they had been cherry-picked or not. The National Security Network has written to request this:

"We write to respectfully suggest inquiry and attention into the exact nature and methodology that is being used to track the security situation in Iraq and specifically the assertions that sectarian violence is down. Not only is accurate reporting the key to sound policy, it is also the responsibility of government to those who have lost loved ones to this horrific conflict.

Furthermore, in order to make good policy going forward, it is imperative to American policy makers, Iraqi government officials, and the American people to have an accurate understanding of the impact of the President’s “surge” on Iraq’s civil war and the civilian population. Accurate portrayal of the scope of sectarian violence in Iraq is critical to making a complete assessment of the President’s escalation strategy, designed to provide space for political resolution to sectarian disputes. It is imperative that these statistics are accurately reported—and reported soon—as the Congress evaluates the White House’s September 15th report and the future of the US engagement in Iraq.

We owe it to the Iraqis, and to ourselves, to have as full accounting as possible."

They're right. Explaining the methodology behind their numbers is obviously the best way for the administration either to put these sorts of doubts to rest once and for all, or else to prove that they are well-founded. And yet, strange to say, I am not holding my breath.

Comments

[McMartin]

Wait, by this logic, doesn't this mean that the best reaction to 9/11 would have been to deny that it ever happened at all?

[novakant]

There is a related article called The Myth of AQI in the new Washington Monthly which will make your jaw drop. It turns out that AQI is only responsible for somewhere between 1 and 8.5 % of all the Iraqi violence, consists of only around 850 people and that consequently most of the Anbar-Sheikhs-switching-sides-to-fight-AQI phenomenon is basically a PR stunt.

[Ugh]

If the standard of success is no car bombings or suicide bombings, we have just handed those who commit suicide bombings a huge victory.

You don't need to see his identification.
These aren't the droids you're looking for.
He can go about his business.
Move along.

[Lurker]

McMartin,

It might well have been. After all, the financial impact of the attack was rather minor in terms of actual property losses. The damage was done by the psychological reaction. The military answer has been rather too large and has probably crippled US economy worse than accepting a few 9/11's a year in a stoical manner would have done.

[Bill]

Or again, the Times of the nineteenth of December had published the official forecasts of the output of various classes of consumption goods in the fourth quarter of 1983, which was also the sixth quarter of the Ninth Three-Year Plan. Today's issue contained a statement of the actual output, from which it appeared that the forecasts were in every instance grossly wrong. Winston's job was to rectify the original figures by making them agree with the later ones.