by Doctor Science
Atlantic reporter Alexis Madrigal wondered how Netflix comes up with their weirdly specific genres, so he (with the help of Atlantic contributor Ian Bogost) reverse-engineered Netflix's classification system, and how its 76,897 (!!) genres are put together.
Along the way, they discovered a strange pattern in the data: the footsteps of [dum dum dum dum-dum] Perry Mason. Madrigal thinks it's a glitch, a ghost in the machine, but I propose to connect all this up, Your Honor.
Direct YouTube Link
After he'd worked on his reverse-engineering for a while, Madrigal got to talk to Todd Yellin, the Netflix VP who developed what the company calls the "altgenre" system.
As our interview concluded, I pulled my computer back out and showed Yellin this one last chart. Take a good look at it. Something should stand out.I commented:
Sitting atop the list of mostly expected Hollywood stars is Raymond Burr, who starred in the 1950s television series Perry Mason. Then, at number seven, we find Barbara Hale, who starred opposite Burr in the show.How can Hale and Burr outrank Meryl Streep and Doris Day, not to mention Samuel L. Jackson, Nicholas Cage, Fred Astaire, Sean Connery, and all these other actors in the top few dozen?
Granted, the existence of all these Raymond Burr and Barbara Hale altgenres doesn't mean that Netflix users are having these movies pop up all the time. They are much more likely to get Action Movies Starring Bruce Willis.
But, then, why have all these genres?
What was the deal? I asked Yellin.
Actually, I had a theory, which I told him. "In the DVD days, Perry Mason fans ordered a ton of Perry Mason, one after the other after the other," I said. "It created sufficient demand that you guys thought there should be categories."
The vexing conclusion is that when human and machine intelligences combine, some things happen that we cannot understand.That is not an accurate theory, Yellin told me. That's just not how it worked.
On the other hand, no one — not even Yellin — is quite sure why there are so many altgenres that feature Raymond Burr and Barbara Hale. It's inexplicable with human logic. It's just something that happened.
I tried on a bunch of different names for the Perry Mason thing: ghost, gremlin, not-quite-a-bug. What do you call the something-in-the-code-and-data which led to the existence of these microgenres?
The vexing, remarkable conclusion is that when companies combine human intelligence and machine intelligence, some things happen that we cannot understand.
"Let me get philosophical for a minute. In a human world, life is made interesting by serendipity," Yellin told me. "The more complexity you add to a machine world, you're adding serendipity that you couldn't imagine. Perry Mason is going to happen. These ghosts in the machine are always going to be a by-product of the complexity. And sometimes we call it a bug and sometimes we call it a feature."
Actually, Alexis, I am baffled by your conclusion that the Perry Mason Mystery is a "ghost in the machine".Thinking more about my comment and what's been revealed about Netflix's methodology, I don't think The Perry Mason Mystery is necessarily because the core altgenre or ecological niche is wildly popular. The phenomenon occurs because *no-one else is in it*: there are a large number of categories related to Understated and Cerebral Dramas, and no other actors are getting their altgenre numbers boosted by them.To me it seems obvious: there is a subgenre, "Understated Cerebral Mysteries with Ironclad Plots, Good Dialogue, Not Much Action or Romance, and on the Side of the Defense", that is popular with viewers yet *drastically* underpopulated. It's so drastically underpopulated that the show that's a best-fit for the category is *enormously* popular, much more popular than anyone realized.
Far from being a "bug", this is programming *platinum* for Netflix. If they're as smart as I think they are, this is the subgenre where they should be looking to make a TV series. The biggest problem will be finding a showrunner and scriptwriters who are able to go against so many of Hollywood's cliches and assumptions. They need to make something where what is visually interesting, striking, or exciting is unimportant, but where there are no holes in the plots. Very high degree of difficulty, and only profitable to Netflix, which makes money from its shows, not from the advertising.
I believe that the dearth of movies/TV in this niche isn't just because it's difficult to write really good plots -- I think scriptwriters could learn to do it, if they had the incentive. I think there's also the fact that it's a poor mix with advertising.
"Understated, Cerebral" is the *opposite* of what advertisers want in an audience. Remember, in conventional TV the audience is not the customer, the audience is the product. The emotional ambience of a Perry Mason Altgenre is cool and thoughtful, and will only appeal to advertisers who think their products will appeal to those qualities. What *most* advertisers want is to forge an emotional connection and an impulse, so they want an excited, emotionally labile audience.
The other way the Perry Mason Altgenre audience fails is that it is too old -- while most advertisers compulsively target 18-34 year-olds, preferably males. In fact, when I saw Madrigal's list I laughed and said, "it's because of my mom!" My mother is currently 88, but she was a Netflix early adopter, way back in the early 2000s. In my parents' case, the genre that brought them into Netflix was "European movies that never had a wide US release" -- and there are a *lot* of them if you don't mind subtitles. They had a great time even before Netflix offered streaming, going through award-winning movies by country and by decade, at last getting a chance to see much more than even the college film club had provided.
Now, My mother was also a big Perry Mason fan back when it was new, I grew up watching episodes (and re-runs) -- and I know that my parents keep looking for its kind of tight, logic- and not action-based mysteries in modern TV, and mostly failing to find them. If (when) Netflix puts together a Perry Mason Altgenre original series, my parents will be *so* there.
This is why I think Netflix is The Future of TV: because in their business model the audience is the customer. Only Netflix could dare make Orange Is the New Black, and possibly only on Netflix would it be an outstanding success -- because to Netflix, all subscribers (and their money) are equal. Whereas over in conventional-TV-land, NBC cancelled Harry's Law while it was one of their most-watched shows -- because the viewers were too old (and, I suspect, too female).
So, what do you think? Is the Perry Mason Altgenre a fluke of some sort? An emergent property of the system? Or is it due to some kind of subliminal influence from my mom? -- and maybe millions of other people like her who like TV that *thinks* and doesn't explode.
I watch Perry Mason three mornings a week at work. "Work" is sitting in a dark room next to a sleeping lady who wakes up every now and then and likes to see the old shows she remembers from the fifties. So we watch Lucy, Leave It to Beaver, Perry Mason, Rawhide, The Rifleman, Daniel Boone....
I grew up without a TV and have no memories of these shows. I was prepared to find Perry Mason as hamhanded and amateurish as Daniel Boone, as formulaic as Beaver, as smarmy and bogus as the Rifleman (the best moment of that show is the opening crotch shots), but, gradually, I overcame my prejudice and realized that it is in fact a good show on many levels.
It's consciously noir; the opening shots particularly show an awareness of how effectively shadows and light can be used to create mood. And the plots are interesting. I play a game of trying to guess as early as possible who the victim will be, the accused and the murderer. It's surprisingly hard; sometimes the show will be half over and no one is dead yet. Of course the climax always happens at five minutes before the hour with a courtroom confession: the show is not perfect. However it is a whole quantum jump better than Ironside with its glaringly ugly colors, talentless actors, and cheesy plots.
As you can tell, I watch a lot of TV at work. In my defense, I also read a lot of books. I spent almost two hundred dollars on Kindle last month.
Posted by: Laura Koerbeer | January 03, 2014 at 07:39 AM
It's really a form of faceted classification. I wonder how many librarians they have on staff?
Posted by: S R Ranganathan | January 03, 2014 at 02:45 PM
To me it seems obvious: there is a subgenre, "Understated Cerebral Mysteries with Ironclad Plots, Good Dialogue, Not Much Action or Romance, and on the Side of the Defense", that is popular with viewers yet *drastically* underpopulated. It's so drastically underpopulated that the show that's a best-fit for the category is *enormously* popular, much more popular than anyone realized.
This might be true but it seems very speculative and without any supporting evidence. Also, this seems like a good example of motivated reasoning.
From the article, it sounds like netflix has human viewers add a bunch of tags for every film (do they also watch all episodes?). From there, I'd guess they run some sort of high dimensionality clustering algorithm; in that case, it is totally possible that the algorithm will occasionally obsess over random parts of the clustering space. Many of these algorithms are non-deterministic (every time you run them you'll get a similar, but slightly different, result).
Posted by: Turbulence | January 03, 2014 at 05:11 PM
On their streaming video operation, Netflix does some heavy duty big data mining. Right down to the mouse click. They have, in effect, an instant response focus group the size of their customer base.
Posted by: CharlesWT | January 03, 2014 at 06:19 PM
Sheer entertainment - watching the Perry Mason episodes from the 50s - 60s, then spend some time on TCM and catch the extras in their heyday (or not). The 50 Foot Woman is always a neat find.
My wife adores Perry and it is great fun finding Mr. Burr playing a heavy or a slimeball.
Posted by: Yama001 | January 03, 2014 at 09:44 PM
This is why I think Netflix is The Future of TV: because in their business model the audience is the customer.
From your lips, etc.
If they do well with it, maybe it will catch on elsewhere.
Posted by: russell | January 04, 2014 at 08:48 AM
now if Netflix could convince TV and movie studios to make better movies, instead of pumping out the thousands and thousands of hours of predictable idiocy that they produce now...
Posted by: cleek | January 04, 2014 at 11:31 AM
Burr was the heavy in "Rear Window".
Posted by: Countme-In | January 04, 2014 at 11:49 AM
Perry Mason? Isn't that where the 5th Amendment goes to die?
Posted by: bobbyp | January 04, 2014 at 12:25 PM
Bobbyp, no one can resist Perry's giant, earnest forehead - you are bound to spill the precise incriminating details.
Posted by: Yama001 | January 04, 2014 at 06:05 PM
What I like is that occasionally Perry, Ham Burger and/or the Presiding Judge will carefully explain your rights as a defendant in the middle of some legal stategem/deposition/badgering. Fairly educational even if most cases most cases don't end with the guilty party breaking down in court.
Posted by: lige | January 05, 2014 at 01:29 AM
I wasn't planning on commenting on this thread, but after reading the post and comments twice, I started thinking about the hours I spent watching the show as a kid, probably sandwiched between "The Fugitive" and "The Rifleman", and then I remembered why I watched so faithfully: Della Reese (Barbara Hale).
What was it about her that appealed to that 13-year-old, beyond an obvious crush, and now to see the Netflix list of favorite actors with her kicking Eastwood's behind and challenging Willis.
She's almost a super-hero; in fact, if she showed up in the "Thor" movies in some pivotal role, I wouldn't be surprised, even at the age of 91, which is old Hale is now.
I think it's because somehow the viewer was invited to wonder about the character's "life" outside of her job as Mason's "confidential secretary", a thrilling, intriguing job title.
She was so stolid and loyal, and so unruffled and competent in case after case where bodies lay everywhere.
Did she go home and read after work. Did she stop and have a drink at a piano bar? Were there boyfriends, and was Mason one of them?
"From Gardner's "The Case of the Drowsy Mosquito", culled from a comments section somewhere or other:
'Mason turned to Della Street. "Know something?"
"What?"
"I bet the preacher would make a reduced rate on marrying three couples instead of two."
She looked up at him with wistful tenderness. "Forget it, Chief."
"Why?"
Her eyes looked out over the long reaches of the desert that stretched out far below. "We're happy now," she said. "You can't tell what marriage would do to us. We'd have a home. I'd be a housekeeper. You'd need a new secretary.... You don't want a home. I don't want you to have a new secretary. Right now you're tired. You've been matching wits with a murderer. You feel as though you'd like to marry and settle down. Day after tomorrow you'll be looking for a new case where you can go like mad, skin through by a thousandth of an inch. That's the way you want to be, and that's the way I want you. You'd never settle down and I don't want you to. And besides, Salty couldn't leave the camp all alone tomorrow."
Mason moved to her side, slipped his arm around her shoulders, held her close to him. "I could argue with you about all that," he said softly.'
Was Della's personal life a bit messy, as opposed to her magnificent control in the workplace?
Inquiring 13-year-old male minds wanted to know.
I don't think Paul Drake was a love interest, because I don't remember her ducking out the back entrance of Mason's office with him. Well, maybe a time or two, but she was all business.
From rooting around on Wikipedia, I learn that Erle Stanley Gardner employed three sisters as his factotums and, after his wife died, he married one of them, which I guess made her a confidential factotum after the fact and one is given to wonder about the confidentiality before the fact, one is.
And then there is this, which I had forgotten: the victims of the murders in each case were for the most part smarmy, nasty characters who frankly could have used a braining with a blunt object and so the viewer was instantly thrust into the position of sympathizing with Mason's client.
From Wikipedia:
"Each episode's format is essentially the same: the first half of the show usually depicts a prospective murder victim as being deserving of homicide, often with Perry's client publicly threatening to kill the victim."
The strict structure of each show (as pertained to "The Fugitive" as well, God, I loved that show), one after the other, gave the viewer the satisfaction of dependability. Even Hamilton Burger stopped being surprised when he lost every case, except three.
Some things I didn't know:
Hale's son is actor William Katt, the good-looking, dimpled nice guy who took Carrie to the prom and didn't live to regret it. He also played Paul Drake Jr. in some of the later Mason made-for-TV movies.
William Hopper (Paul Drake), son of Hedda Hopper, was originally up for and auditioned for the role of Perry Mason. I'm always fascinated by the fact that actors who play iconic roles, in this case, Burr, are more often than not the second, third, or forth choices for the role.
William Talman, the dependably stumped Hamilton Burger, was fired from the show in 1960 after being arrested on morals charges (smoking marijuana with a group of naked people in a private home raided by police) and later reinstated because of a fan-based letter-writing campaign.
Talman was acquitted, much to the chagrin to whomever the prosecuting district attorney happened to be, who I imagine took some ribbing down at City Hall for not being able to legally nail Hamilton Burger in flagrantly delicto, just like Hamilton Burger.
Posted by: Countme-In | January 05, 2014 at 09:12 AM
Count, I love the idea that Ham Burger can win . . . but on defense. Never** as a prosecutor.
** Actually, I have a vague recollection that Mason actually lost a couple of cases. But his win percentage was amazing.
Posted by: wj | January 05, 2014 at 03:14 PM
Ranganathan:
"It's really a form of faceted classification. I wonder how many librarians they have on staff?"
Yes, it is. And I think that number is zero.
So...this intersects with my [current] industry. And I think they're doing it wrong.
It's a clever scheme --- for being invented from scratch. But that's a ludicrous number of categories. Ludicrous. Unwieldy to the point of uselessness.
For example -- or for starters, take your pick -- "Raymond Burr" and "xxxxx genre" should be different tags, and both applied to the content. Super-over-postcoordinating the categories isn't useful.
I'd love to take a look at their scheme.
Posted by: bob_is_boring | January 05, 2014 at 11:55 PM
I never watched Perry Mason, but have been a lifelong fan of Rumpole of the Bailey and would argue that it falls into the same subgenre that you described.
Posted by: Prodigal | January 07, 2014 at 11:41 AM