We need new showrunners for this reality

by liberal japonicus

Via this article in the Chronicle of Higher Education, I was steered to this blog post on a fascinating site, Data Colada

I'll let you click on the links, but here's a taste

Two summers ago, we published a post (Colada 98: .htm) about a study reported within a famous article on dishonesty (.htm). That study was a field experiment conducted at an auto insurance company (The Hartford). It was supervised by Dan Ariely, and it contains data that were fabricated. We don’t know for sure who fabricated those data, but we know for sure that none of Ariely’s co-authors – Shu, Gino, Mazar, or Bazerman – did it [1]. The paper has since been retracted (.htm).

That auto insurance field experiment was Study 3 in the paper.

It turns out that Study 1’s data were also tampered with…but by a different person.

That’s right:
Two different people independently faked data for two different studies in a paper about dishonesty.

The chef's kiss on this? The person who seems to be behind the fabricated data authored a book titled (wait for it!) Rebel Talents: Why It Pays to Break the Rules at Work and in Life. The blurb on the website says

Whether you want to inspire others to action, build a business, or build more meaningful relationships, Rebel Talent will show you how to succeed — by breaking all the rules.

Discuss.

Comments

[Pro Bono]

It's a remarkable story.

The fraudulent manipulation of the auto insurance data seems to have been done by Dan Ariely, famous (in the sense that I've heard of him) for a book popularizing behavioural economics. Ariely denies responsibility, and I'd like to believe him.

Meanwhile, it appears that Francesca Gino fraudulently manipulated the data for Study 1. Gino is now on 'administrative leave' from Harvard.

Perhaps the strangest thing about all this is that it would be quite easy to create fraudulent data in ways which it would be impossible to detect (unless one had access to the original dataset). But these eminent professors, if guilty as charged, understand so little about what real data look like that they made a complete mess of faking it.

The whole field of social sciences has some serious questions to address.

[wj]

As Pro Bono notes, it's relatively easy to create false data that will not be obvious without access to the original raw data. Which raises a couple of obvious questions:

1. Why did these individuals not do that?
   • Were they merely ignorant of the possibility?
   • Were they so ignorant of statistics that they didn't realize that their reported data would raise red flags?
2. Were they so arrogant that they thought nobody would take a hard look at their results?
3. Were they so wedded to their theories that they felt they had to produce results which matched them? Even when the actual data said otherwise.
4. Did they assume that, even if they were caught, they were prominent enough to be untouchable by their universities?

It's not unrelated to one of the questions asked about Trump: Why does he persist in lying about things which a) are readily falsifiable and b) have no obvious benefit to him (possibly excepting proving that he can get away with anything)?

[nous]

wj - I'd guess 2 and put it down as brazenness rather than arrogance. Everyone is so busy trying to publish that very little actually gets verified and no peer reviewer is actually going to take the (largely unpaid) time to sift through the raw data as long as the actual work seems plausible and is free from methodological errors.

Also, there's the Replication Crisis going on, so the authors may have been hoping for security through obscurity due to the sheer volume of published studies.

The latter, though, does not play well with ambitious self-promotion. Oops.

I'm shocked, shocked to find that there is gambling going on in B-Schools.

[wj]

Everyone is so busy trying to publish that very little actually gets verified

It seems like it's time and past time to rethink "publish or perish." Sure, it would be more difficult to actually evaluate what was being published, rather than just counting publications. But the current regime has pretty clearly passed its sell-by date.

Especially since the current contretemps is likely just the tip of a very large iceberg. Not to mention the risks that come from one fraudulent publication getting referenced by numerous others until it becomes part of the received wisdom in the field.

[JanieM]

Not exactly on topic, but stupidity hiding behind arrogance is related to dishonesty hiding behind arrogance, right?

Submarine missing near Titanic used a $30 Logitech gamepad for steering. I have no idea whether to trust this source.

[Michael Cain]

Re wj's #1, I would add the possibility that some minimum-wage research assistant created the fake data. It's been a long time since I was an RA, but my recollection is that creating fake data would have been a lot easier/faster than doing the actual work. The lead investigator(s) probably trusted me enough that they wouldn't go to the effort to test the data I was handing them -- that was, after all, one of the things they were paying me for.

[Michael Cain]

Ars Technica has a story up today about the common use of game controllers by the military and NASA because of their users' existing familiarity with them, and the very large effort the console companies put into getting the UI right.

[JanieM]

and the very large effort the console companies put into getting the UI right.

If only Microsoft would take a hint, and all the other entities that (I assume) copied the most boneheaded idea in the history of computers from them: the disappearing scrollbar. To which I will now add, scrollbars that are too narrow to catch with the cursor, and scrollbars that don't have arrows at the top and bottom.

[nous]

Re wj's #1, I would add the possibility that some minimum-wage research assistant created the fake data.

If this were the case, though, I would think that half the data would be accurate and the other half would map off of that and would vary within the margin of error give or take a couple of outliers. The data entry people would have no motivation to create splashy results that would come under greater scrutiny. When you cheat on this sort of task the whole idea is to make the results innocuous.

These results were tailored to the splashy hypothesis. That makes me think it was someone who would gain more from the headlines - a PI or a Co-I, or it was someone at the gathering institution with a financial stake in the results coming out a particular way.

[nous]

It's really common for the US Mil to use familiar controllers, as Michael says, especially when they recruit heavily for the drone operator positions from the gamer community. The standard controllers have huge support for parts and the operators can rely on years of muscle memory and "switchology" to flatten the learning curve.

The US Navy used Xbox 360 controllers for their remotely piloted subs.

The map is the territory.

[wj]

someone at the gathering institution with a financial stake in the results coming out a particular way.

I'm thinking that a reputational stake, rather than a financial one, might be in play here.

Granted there are those for whom money is all. No matter how much they already have. But there are also those who care more than anything about their reputation, especially for the brilliance of their "groundbreaking ideas." And if "improving" the data will support that reputation....

[Michael Cain]

@nous... When I was an upper-level undergraduate, and then a graduate student, I studied the material and the professor. When I was an RA, I did research on the actual question, but also asked, "What positive results does the PI want?" PIs almost never want negative results.

Never let a chance to make a Real Genius reference pass... One of the few things that grated on me was that Lazlo had to tell them the PI was building a weapon.

[liberal japonicus]

About the submersible, there is this
https://techcrunch.com/2023/06/20/a-whistleblower-raised-safety-concerns-about-oceangates-submersible-in-2018-then-he-was-fired/

Related to hubris is this
https://www.propublica.org/article/samuel-alito-luxury-fishing-trip-paul-singer-scotus-supreme-court

[wj]

What was fun about Alito's WSJ op-ed in his own defense was him saying both
-- that he didn't need to report his free private plane ride with Singer because "personal hospitality need not be reported,” I.e. Paul Singer was a personal friend.
But also
-- that he didn't need to recuse himself from a case where Singer was a lead party because "My recollection is that I have spoken to Mr. Singer on no more than a handful of occasions, all of which (with the exception of small talk during a fishing trip 15 years ago) consisted of brief and casual comments at events attended by large groups.” In other words, he hardly knew the man.

It's bad enough when someone, in respinse to a question, contradicts something he said on an earlier occasion. But in a written piece? In the same written piece? Pathetic.

[Snarki, child of Loki]

"It's bad enough when someone, in respinse to a question, contradicts something he said on an earlier occasion. But in a written piece? In the same written piece?"

...it is what we have come to expect from six of the galaxy-sized legal brains of the US Supremely Deplorable Court.

[JanieM]

"It's bad enough when someone, in respinse to a question, contradicts something he said on an earlier occasion. But in a written piece? In the same written piece?"

They're so entrenched and entitled that they feel like they don't even have to bother to pretend to make sense anymore. It's one big sneer, just like Kavanagh's behavior at his hearing, and Collins's speech justifying her vote for him. Her "fuck you" to the rest of us couldn't have been clearer if she'd said those two words out loud. Another Leonard Leo minion, of course.

[wj]

Another Leonard Leo minion, of course.

The country would be significantly better off is Leonard Leo, Howard Jarvis, and Wayne LaPierre had never lived. Others would have arisen (or been recruited) in their place, no doubt. But their effectiveness in trashing the nation might have been less.

[JanieM]

Murdoch, don't forget Murdoch.

And for that matter, Putin, with his influencing operation (see Adam Silverman's posts on BJ from time to time).

[wj]

I see Murdoch more as an enabler then a leader or a driver. A very effective enabler, certainly, but still only that.

Similarly Putin exploits the crazies among us. But he doesn't lead or drive where they go.

[JanieM]

wj, I think you are profoundly mistaken about that, but I'm not going to get into a back and forth about it, especially at this time of night.

[wj]

JanieM -- wiser than me.

[Girl from the North Country]

Personally, I see Murdoch as a driver, but one whose aims were so focused on personal wealth, success and power that in his pursuit of those he just assumed that he was also furthering the aims of a generally right-wing worldview. I think the fact that he has enabled an anti democratic, authoritarian and/or fascistic movement (in various countries - this is an individual whose only allegiance is to himself,and perhaps Richistan), led moreover by incompetent idiots, has come as a bad shock now it's too late. Not that he has ideological objections, exactly, it's just that like most oligarchs he prefers to operate in stable societies, at least nominally backed by the rule of law, so that, for example, authoritarian leaders can't just take his empire away from him, as Putin can with inconveniences in Russia.

[wj]

like most oligarchs he prefers to operate in stable societies, at least nominally backed by the rule of law, so that, for example, authoritarian leaders can't just take his empire away from him,

I wouldn't be surprised if some of the deep pocket arch-libertarian donors here (the Kochs, the Mercers, etc.) are having glimmerings of the same Oops! reaction. (Theil is too dumb, outside his narrow technical field, to realize it.) Reaping the whirlwind is such a downer.

[nous]

I think they are all figuring on being too big to fail, and having the resources to keep themselves out of harms way if the pot boils over.

https://www.theguardian.com/news/2022/sep/04/super-rich-prepper-bunkers-apocalypse-survival-richest-rushkoff

Not that I think they actually want society to fail, but I think they would rather society fail than that it succeed at the cost of their own wealth-and-power hoarding, so they are betting a bit on fail to limit their own exposure.