Paradoxes in Film Ratings

Figures & data

Figure 1. Scatterplot matrix of Rating, Length, and Year. Notice that longer movies tend to have higher ratings and more recent movies tend to be longer movies, but that Year and Rating appear to be uncorrelated or, perhaps, negatively correlated.

Table 1: Pairwise correlations between 3 qunatitative variables, with P-values.

Pair of Variables	R	P-value
Length vs. Rating	0.318	0.001
Year vs. Length	0.509	0.000
Year vs. Rating	−0.148	0.143

Figure 2. Rating vs. Year, coded by Length. Movies less than 90 minutes are coded as short, while movies 90 minutes or longer are coded as long.

Table 2: Rating versus Year correlations, controlling for Length. The negative correlation between Rating and Year is more evident: within each Length category more recent movies get lower ratings.

Pair of Variables	R	P-value
Rating vs. Year (short movies)	−0.520	0.000
Rating vs. Year (long movies)	−0.280	0.033

Table 3: Here is a summary of favorite examples of Simpson's paradox. In each case, the direction of an X-versus-Y relationship is reversed when controlling for the Z variable.

See the references for the complete data and the stories behind the data.

Subject	X	Y	Z	Reference
Berkely Admissions Data	sex of applicant	accept or reject	grad program applied to	Freedman, et al. 1998, pp 17–20.
Airlines on-time data	airline	on-time or late	airport location	Moore 2003, p 143.
Death sentence data	race of convicted murderer	death sentence: yes or no	race of murder victim	Witmer 1992, pp 110–112.
Comparing batting averages	person batting	hit or out	year of that at bat	Friedlander 1992, p 845.
Prenatal care	care status	infant mortality	clinic	Bishop, Fienberg and Holland 1975, pp 41–42.

Table 4: The percentage of good movies is higher for new movies (32%) than for old (30%).

But this comparison reverses itself when controlling for movie length (i.e., when disaggregating into Short or Long movies.)

Short Movies	bad	good	good%	Long Movies	bad	good	good%	All Movies	bad	good	good%
new	7	0	0.0%	new	27	16	37.2%	new	34	16	32.0%
old	29	6	17.1%	old	6	9	60.0%	old	35	15	30.0%
									69	31	31.0%

Figure 3. The distribution of ratings for a SRS of 100 movies. We include both a dotplot and a frequency table. The average rating is about 2.5 (**1/2).

Freedman, D., Pisani, R., and Purves, R. (1998), Statistics (3rd ed.), New York, NY: W.W. Norton and Company.

 Google Scholar

Moore, D. S. (2003), The Basic Practice of Statistics (3rd ed.), New York, NY: W.H. Freeman.

 Google Scholar

Witmer, J. A. (1992), Data Analysis: An Introduction, Prentice-Hall, Engelwood Cliffs, NJ.

 Google Scholar

Friedlander, R. J. (1992), “Ol'Abner Has Done it Again,” American Mathematical Monthly, 99 (9), 845.

 Web of Science ®Google Scholar

Bishop, Y. M., Fienberg, S. E., and Holland, P. W. (1975), Discrete Multivariate Analysis: Theory and Practice, Cambridge, Massachusetts: The MIT Press.

 Google Scholar