Pandas Fundamentals — Series & DataFrame

ds1-2 built the ndarray. This chapter delivers the second row of ds1-1's own stack roadmap: pandas, the library this entire course's own idea of "a dataset" is actually built around. Every remaining chapter through ds1-9 works with the two structures introduced here.

Series — A Labeled ndarray

A pandas Series is, structurally, exactly what ds1-2's own ndarray already was, plus one addition: a labeled index alongside the values.

import pandas as pd

s = pd.Series([120, 340, 95, 410], index=["Mon", "Tue", "Wed", "Thu"])
s["Tue"]      # 340 — access by label
s.values      # array([120, 340, 95, 410]) — the underlying ndarray, unchanged from ds1-2

That last line is worth pausing on: .values hands back a genuine NumPy ndarray. A Series isn't a new kind of data underneath — it's an ndarray with labels attached on top.

DataFrame — A Table of Aligned Series

A DataFrame extends the same idea to two dimensions: a table where each column is its own Series, and every column shares the same row index. ds1-1's own throughline promised this explicitly — a DataFrame is a labeled wrapper around an ndarray, not a separate structure invented from scratch.

df = pd.DataFrame({
    "product": ["Latte", "Espresso", "Latte", "Mocha"],
    "quantity": [12, 8, 15, 6],
    "revenue": [54.00, 24.00, 67.50, 30.00],
})
df.values     # a 2D ndarray underneath — same relationship as Series.values above

Reading Real Data — CSV and JSON

py2-7 covered Python's own json module — json.load() parsing a file into nested dicts and lists by hand, followed by manually walking that structure to pull out the fields you actually want. At dataset scale, pandas replaces that manual walk with a single call:

df = pd.read_csv("sales.csv")
df = pd.read_json("sales.json")   # the py2-7 json.load() step, plus the table-construction step, in one call

.loc vs. .iloc — A Genuinely Common Beginner Confusion

Both select rows and columns. The difference is what they select by:

Boolean Filtering — ds1-2's Own Mechanism, One Layer Up

df[df["quantity"] > 10]   # only rows where quantity exceeds 10

This is ds1-2's own boolean masking, applied to whole rows instead of individual array elements — df["quantity"] > 10 produces a Series of True/False values (one per row), and indexing the DataFrame with it keeps only the matching rows. Exactly the pattern ds1-2's own warn-box flagged as worth remembering.

A First Look, Column Selection & Derived Columns

df.head()      # first 5 rows — a quick look
df.info()      # column types, non-null counts
df.describe()  # summary statistics — previews ds1-6

df["revenue"]                 # select one column (a Series)
df["avg_price"] = df["revenue"] / df["quantity"]   # add a computed column

This Course's Own Running Dataset

Picking up ds1-1's own retail example directly: a small daily sales log for a chain of coffee shops. This exact table — messy details included — is what ds1-4 spends its own chapter cleaning up, so nothing here needs fixing yet.

Four real, distinct problems, deliberately left visible here rather than fixed on sight: a missing store_id, an inconsistent date format on row 1004, an inconsistent product-name capitalization on row 1005, and a fully duplicated row at the end. ds1-4 addresses each one by name.

Hands-On Exercises