My Photo

« Just For Giggles | Main | Civility »

October 11, 2006

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d834515c2369e200d834bb516a53ef

Listed below are links to weblogs that reference The Lancet Study:

Comments

Les Roberts was on Democracy Now this morning. He said reporters could easily verify or refute the Lancet study. Just go to the gravediggers in 4 or 5 villages or towns and ask them to compare burial rates in 2002 with the more recent past. The Lancet paper says death rates have quadrupled, so one doesn't need a subtle statistical analysis to see an effect like that.

If professional gravediggers are a bottleneck. I don't know that they are or aren't, but it isn't obvious to me that, for example, people aren't burying their own dead.

Right, Lizardbreath, but maybe you'd see a doubling rather than a quadrupling. Or maybe you wouldn't. It would not be a conclusive test, though I'll point out that Les Roberts is one of the authors and the one most visibly defending this study and the previous one in public, so he's not trying to discredit himself. He's offering a good faith way for people to do an independent check.

Which is what all of us American and British citizens should be demanding of our government. I get the impression (mostly from the Lancet critics) that people don't want an independent investigation that would settle this issue, yet to me this is a moral and ethical no-brainer--as invaders of Iraq we have a moral duty to know how many deaths our actions have caused, directly and indirectly.

Not that I want our government to actually conduct the study--just support independent investigators in every way possible.

"Just go to the gravediggers in 4 or 5 villages or towns and ask them to compare burial rates in 2002 with the more recent past."

But, look: after you did that, you'd have to extrapolate to the whole country to get the answer, right?

And to justify the extrapolation, you'd need to know that the 4 or 5 villages were actually representative.

So in other words, you have reproduced the study-design of the Lancet study (i.e. selecting random samples and scaling up) except this time doing it *really badly*, with far too small of a sample size.

Anyone pretending there are problems with the Lancet study would just raise the same charges against the grave-digger study: "those villages were unrepresentative!" "The grave-diggers lied!" etc.

What *would* be interesting--though you'll never hear it--would be to have one of these holocaust-denialists tell you exactly how *they* would design a study to produce the most reliable estimate of excess deaths. Instead of just criticizing all of the time, why don't they come up with some positive proposals of their own?

Given the constraints of time, barriers to communication, danger in the field, breakdown of centralized reporting, and so on, I think the Lancet people have come up with about as good of a number as we can hope to get for a while. Claims to the contrary are just more negativity from the party of no ideas.

Sigh. Kid bitzer, did you post that not knowing that Les Roberts is one of the Lancet authors?

I thought it was to his credit that he's trying to get journalists to do their own investigation, half-assed though it would be. Of course it wouldn't be conclusive, but if several journalists did this and found that, yeah, gravediggers had noticed that burial rates had skyrocketed it would be strong supporting evidence. If they found no effect then it wouldn't disprove the study, but it would lower the chances of it being correct. In other fields people don't necessarily accept one study as conclusive, nor do they dismiss it because they "know" better.

The issue is extremely important and there's no reason (yes, I know Iraq is extremely dangerous) why we shouldn't be demanding our government sponsor (not control) a similar study on a much larger scale. And if they refuse that'll tell us something. I think I know what they'll do.

my response was not intended as a criticism of Les Roberts or of you. My point was simply that this would be extremely weak confirmatory evidence, open to all of the complaints that are being raised against the study itself--plus some more, based on the trivially small sample size.

Sure, it's to Roberts' credit that he says "good look for yourselves". It's as good an answer as any. And I am happy to join you in asking our government to keep track of its effects.

Except that there would kind of be no point in asking, because, you know, Rumsfeld doesn't do body-counts and all.

I wasn't trying to harsh on you, just pointing out the extreme limitations of this method of follow-up. E.g. this:

"If they found no effect then it wouldn't disprove the study, but it would lower the chances of it being correct."

strikes me as wrong, exactly because it would be so easy for a study-skeptic to cherry-pick a few villages that have escaped relatively unscathed and claim that they have debunked the study. Whereas in fact it would only show that those villages were unrepresentative.

As far as I can make out, the idea of going to war with a country so you can turn it into a different kind of country, one better suited to your interests, is nuts. Nuts, as in willfully ignorant of history and of human nature. Close your eyes, stick your fingers in your ears, chant "la la la" as loudly as you can, and assume the world will look like you want it to look when you stop. That kind of nuts.

And that appears to be our foreign policy.

Exactly, which was one of the prime reasons for opposing the war from the start (and my biggest reason, based on my own knowledge of the histoy of warfare). This central premise in favor of war was nuts.

And whatever belief people had in this possibility should be irrevocably dashed by now, with so much post-war incompetence and unending carnage.

Now, the current debate concerning Iraq policy has more to do with dealing with the irrational beliefs of the war supporters (who cannot face the consequences of the worst American strategic blunder since when?) than actual policy on what to do next.

I suspect the Lancet study is correct in estimating that total war related Iraqi deaths is between 400,000 to 800,000. Let your own personal bias select the number in that range.

So two points. If you just refuse to believe the Lancet study, what is the actual number, and based on what methodology? Anyone who rejects the study but will not engage in meaningful discussion of this question is simply being willfully blind. And that is the current policy of the Bush administration -- refusing to even address the question.

Two, if rational thought requires accepting these big numbers, how can any future prediction of a semi-decent outcome for the war be based in reality? Carnage on this scale unleashes its own terrible monster. There are no mulligans in fighting wars.

"If you just refuse to believe the Lancet study, what is the actual number, and based on what methodology? Anyone who rejects the study but will not engage in meaningful discussion of this question is simply being willfully blind."

Well as far as "excess deaths" goes, a strong methodological objection is that you shouldn't trust the pre-war Iraqi death rates. As it stands, the reported death rate was dramatically below the US, UK, Germany, France, Brazil, and Mexico. (8.1,10.3,10.3,9.0,9.3,10.1 per 1000) {I used the easiest to find rates over the past four years for each country, which isn't strictly sound but considering the lack of huge disasters in any of the reference countries I suspect isn't the cause of an error}. The reported Iraq number appears to be around 6.4.

When I was looking at the numbers originally I suspected it might be a function of age, which is why I added two relatively younger countries--Mexico and Brazil.

So under the sanctions regime (which was allegedly killing thousands) Saddam had apparently a drastically better death rate than the US, UK, Germany, France, Brazil or Mexico. That is drastically as in 20% better than the US and 35% better than the UK and Germany. (Am I doing that calculation right? Normally I'm math-confident but that seems ridiculous.)

Well as far as "excess deaths" goes, a strong methodological objection is that you shouldn't trust the pre-war Iraqi death rates.

But the study compares post-war Iraqi deaths as reported by surveyed households with pre-war Iraqi deaths as reported by the same surveyed households. It doesn't depend at all on officially reported pre-war Iraqi death rates.

Sebastian: So under the sanctions regime (which was allegedly killing thousands) Saddam had apparently a drastically better death rate than the US, UK, Germany, France, Brazil or Mexico.

The sanctions were most lethal to infants and small children (as NGOs were pointing out from 1991 onwards) - dysentery, for example, which an adult may well survive, can easily be lethal to a child under five. As the authors of the Lancet study note (in, I think, the first report) infants and very young children were the age group least likely to have death certificates and most likely for their deaths to be underreported, especially if several years had passed. I have no idea if the deaths of children under the sanctions could by itself account for the difference you point out: but a filmed report by John Pilger released in 2000 said that 4000 infants and children were dying each month as a result of the sanctions.

Seb: a very good point, you make, I think. My only comment: weren't most of the so-called extra deaths attributable to violence? Whatever you might think of health policy in pre-war Iraq, it seems unlikely that what was underreported was an awful lot of tribal and political violence.

The terrifying implication of the study, which people ae not really fessing up to, is that, if true, even those people who are following this as intently as possible have no idea what's really going on over there.

My point was simply that this would be extremely weak confirmatory evidence, open to all of the complaints that are being raised against the study itself--plus some more, based on the trivially small sample size.

Complaints by whom? The methodology is textbook in epidemiological circles.

One thing I think both left and right should take from this is that we have no idea whatsoever what the consequences of our policies often are, whether they be sanctions or the invasion. That uncertainty ought to be figured into the deliberations much more than it is as of the moment. Instead, we con ourselves with false assurances of certainty and lots of armchair reasoning. Elsewhere, people may or may not be dying because of it.

Just that uncertainty itself is morally objectionable.

You are right, but when I read the report again the numbers they use are even more silly:

A sample size
of 12 000 was calculated to be adequate to identify a
doubling of an estimated pre-invasion crude mortality
rate of 5·0 per 1000 people per year with 95% confidence
and a power of 80%, and was chosen to balance the need
for robust data with the level of risk acceptable to field
teams.

Their estimate allegedly independently finds that the pre-invasion Iraqi death rate was 1/2 of the German, French, Mexican and Brazilian rates and 2/3 of the US death rate.

And that should be "Lizardbreath, you are right, but..."

sven--you'll find no argument from me on this point. The complaints are spurious and meretricious, and if we had not just been reminded to watch our manners, I would say they are intellectually dishonest.

Nevertheless, any of the people who complain about the original Lancet study (thereby showing their ignorance of standard epidemiological methods, as you point out) will also complain about the proposed grave-digger check--and will have *slightly* more reason for complaint, since the proposed number of checks really would be too low to show much.

But given that the point of the study is to measure the change in death rates, does the fact that the method used systematically underestimates the actual death rate undercut its usefulness for that purpose? Unless you've got some reason for assuming that the survey method underestimated deaths in the pre-war period and then stopped underestimating deaths in the post-war period, I don't see that it does.

"A sample size of 12 000 was calculated to be adequate to identify a doubling of an estimated pre-invasion crude mortality rate of 5·0 per 1000 people per year with 95% confidence and a power of 80%,"

context shows that they were using the 5/1000 number for a power calculation, i.e. a preliminary estimate of how large a sample will be needed in order to come up with meaningful results.

The numbers you use in power calculations are not the same as the numbers you use in final calculations, and may be intentionally low-balled or high-balled in order to create a greater margin of error.

(I.e., the thought is "let's ask enough people so that *even if* the pre-war mortality were as low as 5/1000, our findings would still be statistically robust").

Power calculations are not actual measurements; they're more like preliminary estimates to make sure that your measuring stick is calibrated correctly, and that you are using the right scale.

Don't forget the doublespeak that spews from the Prez:

I am, you know, amazed that this is a society which so wants to be free that they're willing to -- you know, that there's a level of violence that they tolerate.

Reading through the thread I find the discussions about the study's methodology interesting and potentially useful. But what I really find fascinating is the prevelant (both in this thread and elsewhere) gut-level reaction that the numbers are just too high. (Note that my initial reaction was similar, that 600k excess deaths by violence seemed very high). But, as I asked way upthread, is the headline number really so high on a gut level?

To revisit my crude arithmentic, the study's lower bound of about 400k excess deaths by violence implies an average of 350 Iraqis being killed per day. The mid-level number of 600k brings that up to 500 per day.

Recent news reports had the Iraq government reporting a daily average of 90 civilians being killed by violence in Baghdad alone in September 2006. No definition of civilian but an obvious inference is that, if the current Iraqi army casualties (plus police perhaps? plus insurgents perhaps?) are included that number will be higher. Add in the rest of the country, some parts of which are reportedly less violent, some parts more and that lower bound number of 350/day suddenly appears to be not that high at all and the 500/day figure at least within reason on a gut level as well.

Obviously there are people with a very strong incentive to dismiss this study out of hand (e.g. George Bush and his strongest supporters feel it makes their decisions and actions look terrible and runs counter to the "things aren't so bad and are improving" theme). But I think there are also many more people (and especially Americans) who just don't want to accept that so many people are getting killed. It's ugly in Iraq and that attention grabbing number brings the ugliness home.

SH,

So are you suggesting that the methodology under-estimates the pre-invasion mortality rate but over-estimates the post-invasion mortality rate?

"But given that the point of the study is to measure the change in death rates, does the fact that the method used systematically underestimates the actual death rate undercut its usefulness for that purpose?"

Sure it does. First, finding such an obviously wrong result has to call the methodology into question. Second, there is no reason whatsoever to assume that the error is evenly distributed over time. A well known problem with retrospective surveys which ask for information from many years before is that they underreport things from the beginning of the period and overreport the end of the period.

This well known methodological problem is magnified when you try to measure 'excess' against a baseline which is being determined by retrospective oral interview.

While I can see it for retrospective surveys generally, that seems like an unlikely critique of a retrospective survey of something as memorable as deaths within a household. Do you really think that the odds of recalling that a member of your household died are substantially different for the period 1996-2001 than for 2002-the present? I'd think that accuracy of recall for events of such magnitude would approach perfection over both periods. Does anyone forget the deaths of a household member at all, much less in the past decade?

One thing to remember here: the methodology in this study is NOT ad hoc (or at least not as ad hoc as some people would have you to believe). It's based on methodology used in other epidemiological studies. The estimates tend to run high, but not overly so, and tend to be more accurate than other methods.

Some of the criticism seems to be in a vacuum, and not connected to its real-world usage, where there have been confirmations.

"Does anyone forget the deaths of a household member at all, much less in the past decade?"

The exact year is rather important if you are trying to estimate an annual death rate.

What does that have to do with systematic underreporting (again, something which seems very unlikely to me in the context of deaths within a household)?

"What does that have to do with systematic underreporting"

You have a cut-off and have to locate the deaths in a particular year if you want to calculate a yearly death rate.

*sigh*. Can we rename this thread? "Arguing about complex statistics between people who know nothing of complex statistics"?

I don't know much about statistics -- just the bare minimum I need for my job. However, I'm familiar enough with statistics methodology to realize that some objections here are rather silly. I mean, they sound good to those who don't know statistics (IE: Most people), but in terms of actual "people using statistics" they're ridiculous. It's like first-year algebra students -- fresh from learning about these crazy 'function' things -- expressing grave doubt that one can easily find the tangent line at any point along that function, or reliably express the area under the curve with a simple procedure.

It sounds magical and crazy to them, but it's such a solid and unremarkable part of calculus that no math paper is going to spend much time talking about how basic integration works.

Look, laymen have problems grasping the actual true probabilities of the Monty Hall problem. I've had people argue it voraciously, confidant that their "gut instinct" was correct. They were wrong -- and could be proven wrong with elementary probability math.

A lot of the "objections" to the study are like Joe Average proclaiming the Monty Hall odds are 50/50. He's utterly, totally wrong -- but he doesn't have the math background to understand how badly wrong he is.

Okay, I think this is going to be repeating something already said upstream, but it doesn't seem to have sunk in.

Suppose that the prewar rate is being grossly underestimated (because--choose your paranoia: the Lancet people want to exaggerate the effect; Saddam was lying about how many were dying; the villagers asked by the Lancet people just forgot about a lot of deaths before 2003; whatever).

How would that affect the study? Well, the 650,000 is arrived at by subtracting pre-war totals from current totals. So if the pre-war totals were higher, it would make the difference smaller; it would mean that the 650,000 people who are being claimed as *excess* mortality are not really excess at all. Many of them would have died anyway, even if the old pre-war rate had continued unchanged.

But the problem with that proposal is simple: the vast majority of these people (like roughly 12/13) died by violence, "the most common cause being gunfire".

These are not people who would have just died anyway had the war never happened. These are people who were shot. Some of them by our own troops, some of them by death-squads and militias.

So now if you want to claim that the estimated excess is being inflated by including deaths that would have occurred even in pre-war conditions, you have to claim not only that a few hundred thousand of those people would have died anyway, but that a few hundred thousand of those people would have been shot anyway (or died some comparably violent death).

This just seems to descend deeper and deeper into special pleading. Go there if you like.

"A lot of the "objections" to the study are like Joe Average proclaiming the Monty Hall odds are 50/50. He's utterly, totally wrong -- but he doesn't have the math background to understand how badly wrong he is."

Maybe so, but I do have the math background to understand the Monty Hall problem--enough that I never really understood why so many people thought it was a problem.

LizardBreath: Does anyone forget the deaths of a household member at all, much less in the past decade?

Well, LB, put yourself in the position of a member of an extended household. You and your husband and his brothers and their wives and his parents and a couple of cousins and their spouses. During the past decade, every single woman in the household has borne children and the children have often died within a year or two of their birth. Can you honestly say you think you would remember, just on being asked, every infant who died in the household over the past ten years?

I wanted to add: That doesn't mean the study is necessarily right. It just means no one arguing it's wrong (at least here) seems to have the relevant background to make an actual case. They're relying on "There's two doors! 50/50 odds!" arguments rather than -- say -- whipping out Bayes and plugging in some numbers.

I freely admit -- I don't have sufficient skills to do anything more than the crudest methodolgy check, and even then I'd probably screw it up.

But I'm going to lean towards the experts and let peer review handle it. Since this confirms the Lancet study -- while taking into account the valid critiques (IE: From statisticians and such, not people on a blog) on Lancet -- I tend to think they're on the money.

I suspect people's "Guts" don't agree because Americans have no instinctive understanding of chaos or war -- civil or otherwise. We blow up other countries. It doesn't happen to us.

Sure. But your initial assertion was that events in earlier years are systematically underreported compared to later years because people forget the earlier events, or at least that was how I read:

A well known problem with retrospective surveys which ask for information from many years before is that they underreport things from the beginning of the period and overreport the end of the period.

And that's generically plausible -- people do forget things -- but not really in the context of deaths within a household. But in a less memorable context, it's the sort of claim I'd buy without support because it does make sense, people forget stuff.

Now, if I understand you correctly, you're saying that there's going to be a systematic tendency for people to get the year of death of members of their households wrong in a fashion that makes it look as though there were fewer pre-war deaths than there actually were (I suppose if they reported post-war deaths accurately, but had a systematic tendency to report pre-war deaths as having occurred before the study period, when they actually occurred within the study period.) And that doesn't make sense to me -- I'll (in general) buy a tendency to forget things without support, but I'm not familiar with any general human tendency to think things happened longer ago than they did. Do you have any support for this, or am I misunderstanding what you're saying?

Maybe so, but I do have the math background to understand the Monty Hall problem--enough that I never really understood why so many people thought it was a problem.
Because they saw two doors left, and the possibility to choose between them. Most people viewed that as a coin flip.

The most reliable way to explain it to laymen -- without math, that is -- is to simply expand the problem. One million doors, one car. You choose a door, Monty throws open 999,998 doors (leaving one closed) and asks you if you want to switch.

Anyone who can't grasp the benefits of switching at that point -- and who has been informed that the car doesn't change doors -- is someone who is NEVER going to understand.

Kid Bitzer, the actual reported death to violence under the study is 55%. An interesting note is that of all the pre-war deaths, only two were reported as violent, one in an explosion. That is a rather shockingly low number in itself.

"I suspect people's "Guts" don't agree because Americans have no instinctive understanding of chaos or war -- civil or otherwise."

That is why Jane Galt compared it to other war-torn countries.

What Ara said at 1:10.

But as long as the subject is on the table, thank you Sebastian for making the observation that I was about to make -- that your fundamental objection is that "finding such an obviously wrong result has to call the methodology into question."

Yes, your second objection is perfectly reasonable, but it's just a Wild-Assed Guess, ultimately based on your first objection. You seem to be suggesting that this particular study is skewed in a way which takes it past the threshold at which the conclusions are no longer useful. But what is that threshold? How did you determine it? Not based on any empirical data, AFAICT.

Let's grant that pre-war mortality was underreported and post-war mortality overreported (I personally think it must have been, though I'm open to counterarguments).

In order to do that we are postulating a formula f(n) which describes the likelihood that a death which took place n units of time ago would not be reported. Your implicit solution to this (known) methodological problem would be to re-weight the raw data according to f(n), and then recalculate the excess deaths estimate. But you haven't offered any evidence that f(n) is statistically significant in this instance, other than the "obvious" wrongness of the end result. Science in all its glory, eh?

I don't have the time to chase down reasonable values for f(n), or the knowledge of statistics necessary to apply them to the raw data, but I don't see any reason why it couldn't be done. In the meantime I'm going to accept an argument from authority and defer to epidemiologists who have lots of real world experience with what f(n) ordinarily looks like and whether it has an appreciable impact on a study like this.

"And that's generically plausible -- people do forget things -- but not really in the context of deaths within a household. But in a less memorable context, it's the sort of claim I'd buy without support because it does make sense, people forget stuff."

See Jesurgislac above, you're treating it like a US household.

"I'll (in general) buy a tendency to forget things without support, but I'm not familiar with any general human tendency to think things happened longer ago than they did."

I'm pretty sure you are wrong about this too, but I sold back my statistics books in college so I won't immediately be able to help you out. But in my experience, bad events in the past either feel very immediate or well in the past.

"We estimate that as of July, 2006, there have been 654 965 (392 979–942 636) excess Iraqi deaths....Of post-invasion deaths, 601 027 (426 369–793 663) were due to violence."

601/654 = 12/13.

"the actual reported death to violence under the study is 55%"

Can you explain the discrepancy here?

"In the meantime I'm going to accept an argument from authority and defer to epidemiologists who have lots of real world experience with what f(n) ordinarily looks like and whether it has an appreciable impact on a study like this."

It seems to me that the epiemiological experts are saying things like:

"Given the conditions (in Iraq), it's actually quite a remarkable effort," said Steve Heeringa, director of the statistical design group at the Institute for Social Research at the University of Michigan."

"This is the most practical and appropriate methodology for sampling that we have in humanitarian conflict zones," said Brennan, whose group has conducted similar projects in Kosovo, Uganda and Congo."

They seem to be saying "this is the best we can do" not "this is really good data".

See Jesurgislac above, you're treating it like a US household.

That wasn't sarcasm? Seriously, are you sitting here saying that "Eh, what are the odds Iraqis are going to be able to accurately remember how many family members died in the last ten years, including all the infants? Who keeps track of dead babies?" Jeez, two hundred years ago here dead babies got names, and tombstones, and written into the family bible.

I'm pretty sure you are wrong about this too, but I sold back my statistics books in college so I won't immediately be able to help you out. But in my experience, bad events in the past either feel very immediate or well in the past.

Mmmm. If you come up with any sourcing for this as a source of systematic error, do point me to it. I'd be interested.

Oooh! Math wankery!

enough that I never really understood why so many people thought it was a problem.

In my case, it's because the Monty Hall problem turns on a very subtle phrasing which the person who originally told it to me botched. Specifically, it's required that Monty know where everything is in order for you to want to switch; if he simply opens another door "at random" and it happens to show a goat, the odds remain even and there's no point in switching.

"Can you explain the discrepancy here?"

Nope, but that is what their table 2 says, they must be doing something weird with the "excess" calculation.

"Eh, what are the odds Iraqis are going to be able to accurately remember how many family members died in the last ten years, including all the infants?"

See you did it again. You turned "household" into "family".

Oh, come on. Household members are people you know pretty damn well. Most of them are going to be family.

But up here, LizardBreath quotes the study as defining a "household" as a group of people who live and eat together. Given that definition, I'd figure that people would be *more* likely to remember dates of death.

And the "Who keeps track of dead babies" critique doesn't affect the study, given that the excess postwar deaths aren't infants, they're adults who died violently. Unless you're asserting that an Iraqi household is unlike enough enough to an American household that people would be likely to forget deaths of adults as long as they happened more than four years ago, I don't think you're making sense here.

It looks to me like the disagreement here boils down to those who are suspicious of the study's results based on the splashy number preferring to us IBC's results as the calibrating feedback for the control loop rather than the Lancet number. The people supporting the Lancet number are arguing that since the cluster method tends to underestimate rather than overestimate that we should trust the Lancet numbers, even if at the lower end of the spectrum, as a preferable value for making adjustments.

So, accuracy and methodology aside, would a number like the one reported in the Lancet study be sufficient evidence for changing US policy in Iraq, or should we, like the administration, just ignore all feedback and continue with policy-as-given?

And what adjustments should be made based on feedback? Is this a system that can be adjusted?

How much of a reduction in the death toll will all this special pleading result in? My own initial reaction to the 600K figure was incredulity and I had and have some questions about the death certificate issue. Basically, my question comes down to whether the statistics are somehow fraudulent.

Assuming they are not, the Lancet team found 300 deaths in a group of about 13,000 people.
Roughly 2.5 percent. So you don't have to be a statistics whiz to realize this implies hundreds of thousands of deaths. I don't think we have to pledge allegiance to the Lancet's confidence interval of 400,000 to 900,000 to realize that we're talking about a violent death rate in the hundreds of thousands, because it would be rather strange that the survey team would have found 300 violent deaths in a group of 13,000 people if the true violent death rate was 50,000 out of 25,000,000. They should have found around tens of deaths, not hundreds if that were the case, unless they stumbled into several outliers like the Fallujah cluster in the previous study. And if they found several such clusters, that would be significant in itself.

But I'm happy to keep repeating the real point--what the freaking hell is wrong with a country that can't be bothered to determine how many people we might be killing in Iraq? I feel a very strong desire to violate posting rules at this point. But nevermind. We're all happy to casually spout off about the (similarly measured) death tolls in other places like Darfur. But let someone claim we've caused an even greater disaster in Iraq and rather than have the press and politicians united in demanding a serious impartial investigation, the press puts the story on page A12 (in the US) and people go out of their way to find some reason to discredit the numbers. One doesn't need to know a Bayesian from a frequentist to understand what's really driving this reaction and it's not an ethically driven motivation to know how much misery we've inflicted.

Not "doing something weird", I suspect. Something pretty straightforward.

Table 2 reports all deaths. Some of those really *do* go into the category of "would have died anyhow", i.e. even if there were no war.

So the totals in Table 2 do not correspond to the 650k excess figure. Rather, the totals in Table 2 correspond to the 650k excess, *plus* the pre-war background rate.

Subtract the pre-war background, and you'll be subtracting some of the post-war deaths. But not (as I was arguing above) the gunshot and carbomb deaths. You'll be subtracting the infant deaths, the cancer deaths, and so on.

So: violent deaths are 11/20 (=55%) of *all* deaths in the post-war period. But after you subtract from those deaths, the deaths that would have occurred anyhow, then violent deaths become 12/13 of the remaining deaths, i.e. the excess deaths.

So the lower, 55% number is just the result of keeping in the deaths that are not part of the excess. When we look at the excess, the number is still around 12/13,or around 600k.

Now we're back where we were at 2:50p. You want to say that the pre-war rate is being underestimated, and that we should re-assign some significant part of the "excess" group into the pre-war background. And now we're back to where you have a problem, because you have to claim not only that the pre-war had a lot more deaths than this study estimates it to have, but a lot more deaths due to gunshot, deaths due to car-bomb, deaths due to IED, and so on.

So thanks for confusing issues with the irrelevant 55% number. But once again: This just seems to descend deeper and deeper into special pleading. Go there if you like.

Donald:

what the freaking hell is wrong with a country that can't be bothered to determine how many people we might be killing in Iraq?

Donald, the problem is we love Iraqis TOO MUCH. This is particularly true for war enthusiasts who cited humanitarian reasons for their support. They simply care too strongly about Iraqis to be able to find out how many have died, even if the number is just one. They couldn't bear it.

It's DEFINITELY not the case that the humanitarian aspect was always transparent bullshit and vain posturing, and that they never could have cared less whether Iraqis lived or died. Don't even think that.

Okay. I'm confused.

The numbers I'm seeing thrown around are five hundred Iraqi's per day, for the entire war. My understanding was that violence has been steadily increasing for some time. As in, since '02.

Doesn't that mean that if you graphed deaths of Iraqi citizens, you'd see a steadily rising line? And doesn't that mean it's not "five hundred civilians per day", it's more like "fifty - no wait, 100 - er, 200 - um, 300.." and so on?

If these figures are true, it would mean that the current 'wave' of violence in Iraq is being vastly underreported, wouldn't it?

What I'm trying to say is - doesn't this mean there must be a thousand or more Iraqis dying every day?

zaeron

yeah. click on the link to get a copy of the study, and look at graph 2. It has just the kind of upward-trending line you're expecting.

Zaeron -- I think so. And it's not as odd as one might think, given the huge limitations on where reporters can go. When you consider that the horrific number of deaths that make, say, the Washington Post are generally all deaths in Baghdad or one of a few other cities, and that the civil war (understood to include fighting between rival militias that are e.g. both Shi'a) is going on in a lot of smaller places that no reporter can get to, the idea that the death toll is an awful lot higher than the number of deaths we read about -- possibly higher by orders of magnitude -- is not, to me, at all implausible.

They seem to be saying "this is the best we can do" not "this is really good data".

Er, that's exactly the point. At a minimum, data has to be "the best we can do" in order to be worth working with at all. It doesn't have to be "really good data" in order to be useful. What you want is some way of gauging how good your data actually is, so you can decide how useful it is. You have now (helpfully, thank you very much) quoted Brennan and Heeringa, neither of whom suggest that f(n) is a deal-breaker.

What some of us want to know is exactly why "the best we can do" (where "we" is a bunch of peer-reviewed epidemiologists) isn't good enough for you (where "you" is Sebastian Holsclaw). The answer so far appears to be that the data is "obviously wrong."

Seb: All these arguments about misremembering I don't understand. In 80% of cases they saw death certificates. Shortly that resolves the issue of double counting. As far as I know people don't issue multiple death certificates to separate households, extended family members, and cousins.

Correction: 'surely', not 'shortly'. Darned dictation software.

"The answer so far appears to be that the data is "obviously wrong."

Sure, just like we knew (or should have known) that high tension wires don't actually cause cancer.

The problem is that Lancet is paintint a picture of WWII levels of killing. That is surprising. Maybe it is true, but it is surprising.

If kid bitzer is correct and they are nearly all violent deaths, that brings us back to the important question of why they aren't bothering to compare combatants and non-combatants (which IMHO is really the important question). 600,000 combatants? That doesn't necessarily sound as awful as 5 combatants and 599,995 civilians.

And I'm way to exhausted for a "Why it is important to distinguish combatants" fight today. :)

BTW, I mentioned it a couple of hours ago on crooked timber but on rereading the thread I realize I didn't also post it here. On revisiting the numbers for Mexico, the death rate is reported very differently in different places, but the consensus appears to be much more in the 5-7 range. So my hypothesis on younger nations having lower gross death rates may have been correct--making the numbers I quoted look a lot less shocking.

why they aren't bothering to compare combatants and non-combatants (which IMHO is really the important question).

Because it's too difficult -- they can't figure out how to do it at an acceptable level of reliability. An answer would be interesting, but the lack of one isn't surprising, and doesn't render this study useless.

Donald beat me to it. The very fact that there's any room for controversy about this is itself a very strong condemnation of my country's behavior. There ought to be an official tally - and yes, I do expect that parts of it would be kept secret for a while to protect current operations, and parts longer for various good reasons under any administration. But we should have at least a broad overview of how the Iraqi people are doing, who's living, who's dying and of what, and so on.

The problem is that Lancet is paintint a picture of WWII levels of killing. That is surprising. Maybe it is true, but it is surprising.

Hm. Why?

With diffusion of advanced technology, communication is more rapid and sophisticated than in WWII. Diffusion and advancement of weapons technology would logically also raise kill rates.

"What some of us want to know is exactly why "the best we can do" (where "we" is a bunch of peer-reviewed epidemiologists) isn't good enough for you (where "you" is Sebastian Holsclaw)."

Because the best we can do isn't always "good". The best brain surgery I can do would suck. The best a specialist can do for certain cancers sucks. The best top class economists do on lots of economics topics is barely science. The best psychologists can do is pretty goodish for certain ugly cases. "The best we can do in a war with self-reported retrospective studies" isn't super great. The best we can do in calculating values of pi is pretty precise. We can compare to things we know about--like WWII. By reporting "excess" death it implies that we should measure it against civilian deaths in WWII and it suggests that the present war is worse than that. I find that surprising. I suspect the surprise is from mixing combatants and non-combatants. That is problematic (to me) maybe not so much to you.

What's so unusual about a 600K death toll in a war?

As for the civilian/insurgent distinction, suppose half of those alleged 600K are insurgents. Then there were 300,000 Iraqis who died fighting the foreign occupier. And presumably they have a lot of active and passive support. So what's our moral justification for being there? To save them from themselves if we have to kill every last military age male?

"Because it's too difficult"

Heh, don't want to "do the best we can do"?

But I'm happy to keep repeating the real point--what the freaking hell is wrong with a country that can't be bothered to determine how many people we might be killing in Iraq?

whatever is wrong with 'us', it's the same problem that leads us to kill 'them' in the first place.

but the violent/non-violent split is completely different from the combatant/non-combatant split.

What's the dodge here--'oh, if they're violent deaths then clearly they were terrorists, so it's a good thing we killed them over there'?

It may be more *obvious* when children suffer a violent death, but it is still the case that an adult's suffering a violent death does not mean that the adult was a combatant.

And it is an even further step to the claim that a combatant death is one that we need feel no compunctions about. We created this hell. We even gave a lot of these people their IEDs, by leaving the arsenals unlocked and unguarded. The scale of our culpability is gradually getting clearer.

So some people want to grasp at any straw to obscure it.

Yukoner,

I think the gut reaction to the numbers being too high may be the result of being personally offended, not how well they stand up to daily death reports (“This can’t be true. Could we really be doing this? Not my country.”), especially in the face of the phenomenally potent myth of American exceptionalism, that this adventure indeed had something to do with "spreading democracy," that the US only goes abroad to right wrongs, and that the military is something of a Peace Corps with guns (and Morat20’s 2:54 comment). I hear such sentiments all the time. Perhaps that also explains the reluctance to discuss the implications of these casualties (even if we seriously low-ball the numbers) and instead concentrate on methodology, margins of error, etc. (A second to nous 3:22 and DJ 3:23.)

-“what the freaking hell is wrong with a country that can't be bothered to determine how many people we might be killing in Iraq?”

Remember the note from Rumsfeld asking just how many insurgents US forces were killing in order that the Pentagon might track progress?

SH,

Re death rates: You may like to check out coverage of this at

http://abstractnonsense.wordpress.com/

and Unfogged.

"but the violent/non-violent split is completely different from the combatant/non-combatant split.

What's the dodge here--'oh, if they're violent deaths then clearly they were terrorists, so it's a good thing we killed them over there'?"

There is no dodge, but if I'm going to compare it to other wars I can either treat them as civilians (in which case this looks much more horrific than WWII but for silly reasons) or I can look at it.

These numbers are reported for a moral analysis purpose. I find it difficult to do the moral analysis desired when everyone gets counted as a civilian.

Correction--since the US killed 30 percent of the 600K (accepting the numbers at face value), then we have at most killed "only" 180,000 insurgents. The rest of my point stands.

I've been looking for rightwing critics of the Lancet paper to come out in favor of a serious attempt to count the dead (or estimate them using statistical methods, since counts aren't necessarily accurate.) Anyone seen any who have done this? I expect Iraq Body Count will be somewhat critical of this Lancet paper, but I suspect they will agree with the Lancet authors that there is a need for an independent body to do this.

Incidentally, Iraq Body Count criticized the previous paper, preferring the UN survey that was conducted in 2004. But when one compared the violent death toll of the 2004 paper with the UN survey, the UN number was about 30 percent less by IBC's own estimate. In other words, they were in rough agreement and IBC's only legitimate point was that this suggested the true death toll was in the lower part of the Lancet CI, not the upper part as the Lancet authors thought. But it wasn't that much lower.

"I find it difficult to do the moral analysis desired when everyone gets counted as a civilian."

Okay, maybe this would help you do the moral analysis:

In a country that is not involved in a war, pretty much everyone is a civilian.

Non-civilians aren't just born; they are people who are involved in fighting a war. They may have chosen to fight it, or it may have landed in their laps.

By your reasoning, if someone invades your house and shoots you when you resist, then they really shouldn't feel too bad about killing you--after all, you were no longer a civilian as soon as you resisted.

This is crap. If we are morally culpable for having visited a war upon an entire people, then we are also morally culpable for turning at least some civilians into non-civilians. We don't get to then turn around and say "hey, so what if we killed them--it's not like we were killing civilians."

"By your reasoning, if someone invades your house and shoots you when you resist, then they really shouldn't feel too bad about killing you--after all, you were no longer a civilian as soon as you resisted."

Actually with sectarian violence being such a huge part of it, the analogy is more like: if someone invades your house and shoots you take out a gun and shoot your wife....

Hah! That is funny. Suddenly contemplating the 600K deaths caused by our invasion of Iraq hardly bothers me at all.

Thanks, SH!

"Actually with sectarian violence being such a huge part of it, the analogy is more like: if someone invades your house and shoots you take out a gun and shoot your wife...."

does that comment have any bearing on the earlier lines of argument? Does that analogy help in any way to understand the Lancet study, or what it shows about the number of people killed by the invasion, or what we as a country should think about those facts?

Did you have a point? Or were you just making a joke?

I was responding to your analogy. It is connected to the discussion inasmuch as you think your analogy was apt.

Does it have any bearing on the Lancet study? Yes. Presuming your statistics are correct, when you talk about "excess deaths caused by 'X'" there are different levels of moral culpability between invading your house, and you intentionally murdering your wife 'because' I invade your house.

Moral culpability in the eyes of who? Are Iraqis to be swayed by this arguement?

Or rather, argument.

All of your argument amounts to putting scare-quotes around the word 'because'.

If the sectarian strife would not have happened had we not started this war, then, yes, it is happening because of us.

You can't just throw rocks at an unstable situation and then pretend you have no responsibility for the consequences of your destabilizing it.

Sebastian,

I see from your statement here that you apparently still don't understand a very simple point I made way back yesterday here.

Could you go back and read what I wrote until you do understand this? Because if you don't I'm going to have to rip off my head in frustration. Thanks.

Interesting stuff in here. I do some stats, but the fundamental difference is that I'm dealing with words/phrases, not dead people. When looked at in this way, statistics becomes another way to disconnect us from the reality rather than get us closer to what is actually going on.

The mention of the previous figures involved with the sanctions regime is also very interesting. DeLong had, about a year ago, a post with some back of the envelope calculations on child mortality and such during the period around the US Civil war and wondering how this must have affected their outlook and mentality. It occurs to me that if sanctions struck the weak and young disproportionately, it has the effect of supporting a fatalistic view of the world, while the current mortality rates in Iraq must be concentrated among the breadwinners and adults, which may have the opposite effect. I think, if this is true, it has some bearing on the use of sanctions in the future as an instrument of policy. It may also suggest why Iraqis may be much less reserved in their opposition to a US occupation as opposed to Sadaam's rule. Unfortunately, some will argue that this proves that those who did not resist Sadaam but are resisting the US occupation are unregenerate Baathists, which I do not think is the case.

You know, there really are marriages that limp along for decades in a state of quasi-stability. They aren't marital bliss, there are underlying tensions and grievances, but they raise kids, they provide for each partner's financial security, and they don't kill each other.

Do you think if you invade the house of such a marriage, tear apart their common life, and goad one of the two into shooting the other--even if the enraged shooter is acting in part on long-standing grievances--do you really think you escape all culpability?

Multi-sectarian Iraq was always a mess, and it was always going to be hard for it to evolve peacefully out of the Saddam era into a post-Saddam era. But for decades, Shia and Sunni lived in Baghdad without sectarian violence. They intermarried. They raised kids. Had things gone differently, it might have evolved peacefully.

But now that is not going to happen. Because we invaded, in what we now know was a completely needless, pointless war.

And it is simply inadequate to say "yeah, but they might have wound up squabbling even if I didn't destroy their common life and set them against each other." That is of course true. They might have.

But you can't just declare an equivalence between bad outcomes that might have occurred without your interference, and bad outcomes that, in fact, are the direct result of your interference. They are not the same at all. For the second kind, you bear responsibility.

At least I'm glad that this exchange has evolved in such a way as to show the real issues:

The real line of objections to the study are not about statistical reliability and power calculations. They are about trying to evade responsibility.

Jon(S),I do understand that point. That is called wanting to have your cake and eat it too.

Deaths of combatants aren't "excess deaths". They are intended combatant deaths.

Lancet acts on the pretense that the combatant/noncombatant distinction is not worth investigating. It treats everyone as civilians. It isn't 600,000 Iraqi soldiers killed, right? When trumpeting the number it isn't making an ad for US military efficiency, is it? No, they are strongly implying that these aren't soldiers.

That makes their numbers not particularly useful as far as I'm concerned, but insofar as you feel like using it, you can't compare it to combatants plus civilians in other wars. The distinction was drawn in the past for a reason.

To try to do otherwise is to attempt to erase the combatant/non-combatant distinction in a way that I think is morally unuseful.

"They are about trying to evade responsibility."

Quite. Though I think in the opposite way from how you intend it.

the opposite? what does that mean?

Who are you saying is attempting to avoid responsibility? And for what?

OT: UK Chief of the General Staff calls for British withdrawal "soon" - "our presence in Iraq exacerbates" the "difficulties we are facing around the world."

There are other interesting things the General says too.

Sebastian: Lancet acts on the pretense that the combatant/noncombatant distinction is not worth investigating.

No: if you read the study, you will discover that the Lancet study consciously decided not to ask whether the deaths in the household were from people fighting in the insurgency, because they feared that asking questions like that in the middle of the war might well put the research teams at risk. This is saying that the distinction is "not worth investigating", but not as a "pretence": not a distinction worth putting the lives of the people on the research teams at further risk.

The snide tone of people who don't like the answers people risked their lives to get is distinctly unpleasant.

When trumpeting the number it isn't making an ad for US military efficiency, is it? No, they are strongly implying that these aren't soldiers.

No, they are saying that these people wouldn't have been killed if Bush hadn't decided to invade Iraq. To argue that these high numbers could ever be said to be an "ad for US military efficiency" is fairly damn disgusting: are you really saying you see causing the violent death of around 600 000 people in a wholly unnecessary war as "efficient"?

Hey, Sebastian. A thought experiment. The war in Iraq ends, and a couple of years later, out of nowhere, China attacks us. For some reason, it doesn't go nuclear -- they land an invasion force in Mexico, and invade over the Californian border. Through superhuman skill and bravery, our armed forces defend us with such surpassing excellence that not a single American civilian is killed.

A whole hell of a lot of American soldiers die, though.

Have we got anything to be aggrieved about?

Sebastian:

Deaths of combatants aren't "excess deaths".

Sebastian, I don't think you actually believe this, because it's so strikingly wrong.

But if you do, I'll bet you money on this proposition:

A study of German deaths during World War II conducted according to the same criteria as the recent Lancet study in Iraq would have classified German combat deaths as "excess deaths."

As I say, I doubt you actually believe what you're saying, so I doubt you'll bet on it. But as always, I'm happy to take your money.

Seb: they are not "pretending" that all deaths are civilian deaths. They are not making any claim about civilians at all. They are just asking: how many more deaths are there than one would have expected had the death rate under Saddam continued?

They didn't attempt to figure out which deaths under Saddam were 'really' caused by Saddam's torturers, the sanctions, etc. Not doing that might, had the figures come out differently, have led (different) people to charge that the numbers were unfairly favoring the war, since of course when you remove the cause of a bunch of deaths, it looks like the death rate improves. That charge would have gotten the same reply from me, namely: all they purport to count is how many extra people -- not civilians, people -- died.

"Have we got anything to be aggrieved about?"

Sure, but that doesn't change the fact that we would have an even further complaint if they had destroyed Fresno and Davis.

"They didn't attempt to figure out which deaths under Saddam were 'really' caused by Saddam's torturers, the sanctions, etc."

Actually considering that they found only two people killed by violence in the prewar period (one of them allegedly by coalition bombing, they didn't need to try to figure it out. They apparently didn't exist--which is one of my points.

I'm still anxious to make this bet, Sebastian.

Sure, but that doesn't change the fact that we would have an even further complaint if they had destroyed Fresno and Davis.

So for the purposes of your own, internal, moral calculus, Seb, how about you assume every one of those 600K Iraqis who died was holding a gun. That way you can be sure not to feel any worse about the war than you're obliged to.

So, Seb. As a war supporter, what benefits do you see coming out of the war that justify the deaths of 600K Iraqi combatants?

Well then I will have to say that I find the Lancet study not worth the attention it has gotten. I will also say that in the media it is, like it or not (and I suspect "like it") being played as if it meant civilians.

If it was 600,000 Iraqi soldiers killed, it would be regretable in the way war is regretable, but morally understandable in that context.

If it were 600,000 civilians (killed by the US), it would be atrocious.

600,000 of 'we can't be bothered to tell whether they were civilians or soldiers, whether they were killed by the US or by sectarian violence' isn't particularly useful. As I've said before in other threads, it isn't particularly useful especially with the weird anomolies (like the low number of infectious disease deaths, like the one violent death before the war, see also perhaps the death certificate issue which d-squared doesn't believe there was a genocide in the Sudan thinks is such a non-issue.

I will say that I should have made that objection of mine evident earlier in the discussion so as not to annoy people about ancillary issues while not directly confronting the big one. (I hate it when people do that to me. I talked with someone about school vouchers for at least two days before I found out that they thought the quality of US schools was pretty much high enough already. Well of course you won't support vouchers if you think the schools are fine).

So sorry for that.

Deaths of combatants aren't "excess deaths". They are intended combatant deaths.

Well, except to the extent that they're only "combatant deaths" because they're fighting to get us the hell out of their country. You know, "but for" and all that. Alas, smarter people than me have tried to get you to understand this, but you seem stubbornly unwilling to do so.

If it was 600,000 Iraqi soldiers killed, it would be regretable in the way war is regretable, but morally understandable in that context.

Um, okay. What's the benefit that outweighs that cost for you, and makes it morally acceptable?

My thought is that the whole process of modern war has been to try and make an argument that civilian deaths and military deaths are two sides of the same coin. One of the rationales for firebombing Japan and Dresden was that civilians were contributing to the ability to wage war and as such, constituted a legitimate military target. Because, the argument goes, japan dispersed its military industries into residential areas, so that parts were fabricated not in large factories, but in household workshops, it was necessary to destroy that infrastructure. That was certainly the pattern in Korea and Vietnam. I'll agree there is a problematic tension between the Geneva convention and this notion, but the fault lies not with Iraqis taking advantage of GC, but the constant moving up and over the line by powers utilizing industrial strength to wage war. Of course, I understand that no country is going to say 'ok, let's not use industry to wage war better', but it's important to realize how we got to this point.

And that last line "So sorry for that" may sound sarcastic but I'm really serious. I hate it when people pick on the edges while ignoring such a key area of disagreement.

But I also want to make something slightly different VERY clear.

I think Bush and his administration did an awful job in Iraq. Presuming the 600,000 number is correct and even if it were all soldiers and even if none of it were sectarian violence, what we are getting out of it isn't worth it because we aren't getting to a positive end. Bush never treated this like a serious war, and it clearly shows. He declared victory too early, didn't apply enough force early, let 3 separate insurgencies start up full force before bothering to sort of pay attention to them, and the list goes on and on.

Jon,(S) [and I see you've changed your handle recently]"But if you do, I'll bet you money on this proposition:

A study of German deaths during World War II conducted according to the same criteria as the recent Lancet study in Iraq would have classified German combat deaths as "excess deaths."

I'm sure under your wording you would win.

The problem is that in 1950 very few people would think it was worth treating the numbers like that.

Sebastian: Presuming the 600,000 number is correct and even if it were all soldiers and even if none of it were sectarian violence, what we are getting out of it isn't worth it

Is this a response to LizardBreath's question at 07:18 PM?

Sebastian:

The problem is that in 1950 very few people would think it was worth treating the numbers like that.

No. That's not the problem. The problem is something else, something so simple that the fact I can't get you to address it is bringing me dangerously close to ripping off my own head.

You stated that the study is "obviously wrong." Your reason for this? "The rate of civilian deaths in Germany during WWII was 20 in 1,000 measured over the whole war. The rate implied by the Lancet report is 27 in 1,000 thusfar."

But these numbers are simply not comparable. When I (and others) pointed this out, you didn't address this, but simply moved onto other ground.

So, let me ask once again: do you understand this? The reason it matters is because if your Germany comparison is invalid (which it is) so is your main basis for calling the study "obviously wrong."

Now, it may be your statements about it being obviously wrong were just you picking around the edges, as you put it -- and your main concern with the study all along was not the numbers it came up with but the fact they didn't classify people as civilians or not.

Fine. But if that's the case, both my head and I would appreciate a clarification on your part.

<>s wr spprtr, wht bnfts d y s cmng t f th wr tht jstf th dths f K rq cmbtnts? < hrf="http://www.bbc.c.k/hstr/wrldwrs/wwtw/htlr_lbnsrm_.shtml">lbnsrm t'll tch Gd t pt r l ndr thr snd. Sb, knw y wr ChknHwk, bt ddn't knw y wr sch cld clls bstrd.

The comments to this entry are closed.

Whatnot


  • visitors since 3/2/2004

March 2015

Sun Mon Tue Wed Thu Fri Sat
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        
Blog powered by Typepad

QuantCast