A Statistical "Proof" of the Odd Even Star Trek Movie Rule

Background

Amongst trekkies, there is a commonly held belief that:

"... the even-numbered Star Trek films are superior to the odd-numbered Star Trek films".

This is usually believed to be purely a rule of thumb, however after a conversation with a friend (who shall remain anonymous) I wondered whether it might be true.

Obviously whether one film is better than another is quite a subjective matter, so I limited myself to looking purely at the ratings that each film received on the internet movie database (IMDB).

Raw Scores

Scores from IMDB as of 29th January 2006.

#TitleScore
1.Motion Picture5.9
3.Search for Spock6.3
5.Final Frontier4.7
7.Generations6.3
9.Insurrection6.3
2.Wrath of Khan7.6
4.Voyage Home7.2
6.Undiscovered Country7.0
8.First Contact7.4
10.Nemesis6.5

Interpretation

The first thing to notice here is that the worst of the even films (Nemesis) has a score of 6.5. This is still better than the best of the odd films (Search for Spock, Generations & Insurrection) which weigh in at 6.3.

This clear seperation of the score ranges (4.7-6.3 vs. 6.5-7.6) should be a pretty good clue as to which set of films is "better".

Analysis

We can further validate this interpretation by using a Mann-Whitney test. This type of test is useful for comparing sets of small size, to see whether two samples of data actually are from the same data set or not. This involves comparing the two sets, by ranking each element in both sets. In this case we are basically trying to see if the even films are of an entirely different class to the odd films.

As stated aboved there are no films in the odd set of films that are better than any film from the even set of films. This gives us a U value of 0.

Looking up this U value in a table (pdf) gives us a confidence value of 0.01 that the two sets are the same.

Therefore we can be 99% confident that the odd and even films represent two different classes of films, with the even films being the "better" of the two sets.