Rotten Tomatoes' suspicious audience scores, part 3
Marvel Television is not Marvel Studios.
In my previous post on Rotten Tomatoes’ audience scores I mentioned that Marvel Studios TV shows in particularly look very suspicious when analyzing the Audience Scores (AS) produced by the review ratings for those shows.
I explained how the number of review ratings in Rotten Tomatoes (the sample for this analysis) should influence how much an estimated AS, estimated from review ratings, should deviate from RT’s Audience Score produced from all ratings.
I then pointed out that the deviation from the expected AS is very large for some shows with a large sample, a large number of public review ratings, and interestingly the deviation for those shows seems to indicate their AS in the Rotten Tomatoes site if too high compared with the Audience Score estimated from review ratings.
Finally, I remarked on how most of those suspicious shows are Disney shows.
What I didn’t mention in that previous post is that there is, or at least was until 2019, another production company associated with Marvel Entertainment and therefore with Disney. That company is Marvel Television.
For a brief description of Marvel Television I’ll quote Wikipedia:
Marvel Television was an American television production company responsible for live-action and animated… television shows and direct-to-DVD series based on characters from Marvel Comics… The division was transferred to Marvel Studios from Marvel Entertainment in October 2019 and was folded into the former two months later.
To be sure, Marvel Television hasn’t produced new shows after 2021 M.O.D.O.K. and Hit-Monkey, and up until that time it had produced more than 15 shows, all of them based on Marvel’s properties.
Those Marvel Television shows were produced by a different team than the one producing Marvel Studios shows. The Marvel Television shows usually credit as executive producers Jeph Loeb and Joe Quesada, with credit sometimes also for Karim Zreik, Jim Chory, Alan Fine and Stan Lee. The Marvel Studios shows credit as executive producers a different team of people usually headed by Victoria Alonso, Kevin Feige and Louis D'Esposito.
These were two different companies who happened to be owned by the same entity and worked with the same intellectual properties, Marvel’s properties.
Let’s take a look at the list of shows produced by Marvel Television. You’ll notice some of them were broadcast after 2019, but those shows were ordered in 2019 and produced by the same Marvel Television production team.
Agents of S.H.I.E.L.D. (2013–2020)
Agent Carter (2015–2016)
Daredevil (2015–2018)
Jessica Jones (2015–2019)
Luke Cage (2016–2018)
Legion (2017–2019)
Iron Fist (2017–2018)
The Defenders (2017)
Inhumans (2017)
The Gifted (2017–2019)
The Punisher (2017–2019)
Runaways (2017–2019)
Cloak & Dagger (2018–2019)
Helstrom (2020)
M.O.D.O.K. (2021)
Hit-Monkey (2021)
A few of those shows I’ve mentioned before. marvels_agents_of_shield S06 and marvels_helstrom S01 were included in my lists of shows in the upper tail of the distribution of audience scores (AS) and the lower tail respectively. In fact all Marvel Television shows broadcast in 2019 or later were included in the sample of 343 TV shows used for my previous analysis.
Why am I interested in these shows ? Because they are the closest in terms of subject matter and intended audiences to the Marvel Studios shows whose audience scores I found suspect. They were produced by a different production team but were based on the same Marvel properties as the Marvel Studios shows.
So just like I did in my previous analysis, I calculated a distribution of audience scores differences, this time for all Marvel Television shows for all seasons.
Even though it’s a small number of shows (just 34) you can see that the distribution of AS differences is a normal distribution, with a Skewness value of -0.42, standard deviation (σ) of 10.09, and its platykurtic.
Most important though is the value of it’s mean. The mean for this distribution of AS differences is 1.5, much closer to zero than the mean for the previous distribution of AS differences.
As I discussed in the first article, when doing the first analysis I decided on a cut-off value of at least 50 review ratings in order to avoid shows where just a few reviews could skew the analysis, and in fact shows with few review ratings tend to have higher absolute AS differences as I demonstrated in the second article. So in order to test the strength of this finding I calculated the distribution of AS differences for Marvel Television shows again, this time only for shows with 100 review ratings or more. It increases a little bit but not much: 2.32.
This mean AS difference, whether it’s 1.5 or 2.32 percentage points, is close enough to zero and so to what I would expect if nobody was gaming RT’s audience score system. The mean AS difference I calculated for the original 343 TV shows of my previous analysis, 5.99 percentage points, is not.
Cui bono
When I said the difference between Marvel Studios shows and Marvel Television shows is having been produced by different teams from different companies I neglected to mention another important difference. All those suspicious Marvel Studios shows were broadcast by Disney+, but none of the Marvel Television shows were broadcast by Disney+. They were broadcast by Netflix, FX, ABC, Fox, Hulu and Freeform.
Granted, ABC, FX and Freeform are networks owned by Disney Entertainment, and yet they are different companies with their own management teams.
If a show has bad reviews and low viewership that is a problem not just for the production company but also for the network who bought the streaming rights to that show. And in the case of Hulu, Netflix, etc.. that doesn’t seem to have any correlation with shows having suspicious audience scores, but in the case of Disney+, well….
In closing
With this I rest my case. What I think happened and I’ve tried to prove in these series of posts, is that one or more people inside Marvel Studios - and maybe other people in other Disney properties - intentionally gamed Rotten Tomatoes’ audience score system for their own benefit. Maybe. I’ll let you to decide whether I’ve made a convincing case for it or not.
I don’t think I’ll be doing many more posts focused on the entertaining industry in the future, so if you liked this series of posts I’m very happy that I managed to pique your curiosity and entertain you, but you might want to take a look at my other posts before deciding whether you want to subscribe.