Movie Ratings Explorer: A Full EDA on a Public Dataset

What We're Building

An exploration of a MovieLens-style ratings dataset — a real, well-known public research dataset of user ratings for movies, freely available and widely used for exactly this kind of practice. Two files: movies.csv (movie ID, title, genre, release year) and ratings.csv (user ID, movie ID, rating, timestamp). The question driving the whole exploration: what actually makes a movie "highly rated," and does the obvious answer hold up once the data gets a real look?

Loading and Merging

import pandas as pd

movies = pd.read_csv("movies.csv")
ratings = pd.read_csv("ratings.csv")

df = ratings.merge(movies, on="movie_id")
print(df.shape)
print(df.head())
print(df["rating"].describe())

The merge itself is ds1-5's own join material put to real use — a ratings file alone has no genre or title, and a movies file alone has no ratings; the interesting questions only exist once both are joined. .shape and .describe() here aren't a separate "Step 1" — they're just the first thing worth checking before trusting anything built on top of this data.

What Does a Rating Actually Look Like?

import seaborn as sns
import matplotlib.pyplot as plt

sns.histplot(df["rating"], bins=10, color="#15803D")
plt.title("Distribution of Individual Ratings")
plt.savefig("rating_distribution.png")

Ratings cluster heavily around 3-4 out of 5, with genuinely low ratings comparatively rare — a real, common pattern in rating data worth noticing before trusting any average built from it: the scale isn't used symmetrically, so "average rating" already means something skewed toward the positive end before any other analysis happens.

Which Genres Rate Highest?

genre_avg = df.groupby("genre")["rating"].mean().sort_values(ascending=False)
print(genre_avg)

genre_avg.plot(kind="barh", color="#EAB308")
plt.title("Average Rating by Genre")
plt.savefig("genre_ratings.png")

A one-line groupby, but already worth a second look before drawing conclusions — some genres will naturally have far fewer total ratings than others, which is exactly the kind of thing that turns into a genuine trap two sections from now.

Popular vs. Highly Rated — Not the Same Question

title_stats = df.groupby("title").agg(
    avg_rating=("rating", "mean"),
    num_ratings=("rating", "count"),
)

print("Top 10 by average rating:")
print(title_stats.sort_values("avg_rating", ascending=False).head(10))

print("\nTop 10 by number of ratings:")
print(title_stats.sort_values("num_ratings", ascending=False).head(10))

A Statistical Trap: Small-Sample Extremes

Why does a one-rating movie so easily top the list? With only one or two data points, the average has nothing to pull it toward a typical value — a single 5 or a single 1 is the entire average. As the number of ratings grows, the law of large numbers (ds1-6) pulls the average toward the movie's own genuine, underlying quality, since the extremes of individual opinion start to cancel out. This is the same small-sample-size caution ds1-6 raised in the abstract, now caught red-handed in a real dataset.

reliable = title_stats[title_stats["num_ratings"] >= 20]
print(reliable.sort_values("avg_rating", ascending=False).head(10))

Forming a Real Hypothesis: Do Older Movies Rate Higher?

df["decade"] = (df["year"] // 10) * 10
decade_avg = df.groupby("decade")["rating"].mean()

decade_avg.plot(kind="line", marker="o", color="#15803D")
plt.title("Average Rating by Decade")
plt.savefig("decade_trend.png")

What's Next

Chapter 6: Multi-Source Data Aggregator — combining two independent public APIs into one clean, merged dataset, extending Chapter 3's own single-API pattern to genuine multi-source integration.