Week 1: Python Data Science Toolchain

1. Setting Up Your Environment

You want an isolated, reproducible Python environment per project — so upgrading a package for one course doesn't silently break another. Two mainstream options: venv (built into Python) and conda (a separate package/environment manager popular in data science because it also handles non-Python dependencies).

python3 -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

pip install numpy pandas jupyterlab matplotlib scikit-learn

conda create -n ml-course python=3.11
conda activate ml-course

conda install numpy pandas jupyterlab matplotlib scikit-learn

Either way, launch JupyterLab to get a notebook interface — the format you'll use for almost every exercise in this course, because it lets you run code in small pieces and immediately see the output (including plots) inline.

jupyter lab

Once JupyterLab opens, freeze your exact package versions into a file — this is what makes an environment reproducible rather than "works on my machine":

pip freeze > requirements.txt

# On a fresh machine, later:
pip install -r requirements.txt

2. NumPy Essentials

NumPy's ndarray is the data structure underneath nearly every ML library you'll touch — Pandas, scikit-learn, and PyTorch's tensors are all conceptually extensions of it. The core idea: store numbers in contiguous memory and operate on whole arrays at once, instead of looping in Python.

import numpy as np

a = np.array([1, 2, 3, 4])
matrix = np.array([[1, 2], [3, 4], [5, 6]])

print(a.shape)        # (4,)
print(matrix.shape)   # (3, 2)  -> 3 rows, 2 columns
print(matrix.dtype)   # int64 -- every element shares one type

# Indexing & slicing
print(matrix[0])      # [1 2]        -- first row
print(matrix[:, 1])   # [2 4 6]      -- second column, all rows

# Vectorized operations -- no explicit loop
doubled = a * 2                    # [2 4 6 8]
row_sums = matrix.sum(axis=1)      # sum across each row: [3 7 11]

That last line — matrix.sum(axis=1) — is worth sitting with: axis=0 collapses rows (sums down each column), axis=1 collapses columns (sums across each row). Getting this backwards is one of the most common early bugs in ML code, because it fails silently — you just get numbers, the wrong ones.

# Broadcasting: NumPy stretches smaller shapes to match, without copying data
prices = np.array([10.0, 20.0, 30.0])
tax_rate = 1.08

with_tax = prices * tax_rate   # [10.8, 21.6, 32.4] -- scalar "broadcasts" to every element

# Broadcasting also works between a matrix and a smaller array,
# as long as their trailing dimensions line up (or are 1)
matrix = np.array([[1, 2, 3], [4, 5, 6]])   # shape (2, 3)
col_means = matrix.mean(axis=0)              # shape (3,)  -> [2.5, 3.5, 4.5]

centered = matrix - col_means                # (2, 3) - (3,) broadcasts fine

Vectorized operations aren't just more concise than a Python for loop — they're typically 10–100x faster, because NumPy pushes the loop down into compiled C code. You'll rely on this constantly once datasets grow past a few thousand rows, and again in Week 11 when a "vectorized operation" becomes "a whole neural network layer."

flat = np.arange(12)          # [0, 1, 2, ..., 11]
grid = flat.reshape(3, 4)     # same 12 numbers, viewed as 3 rows x 4 columns

# np.newaxis inserts a new length-1 axis -- useful for broadcasting shapes
column = flat[:, np.newaxis]  # shape (12, 1) instead of (12,)

3. Pandas for Tabular Data

Pandas builds on NumPy to give you labeled, tabular data — a DataFrame is essentially a spreadsheet you can manipulate in code, with a Series as its single-column building block.

import pandas as pd

df = pd.read_csv('customers.csv')

df.head()                      # first 5 rows
df.info()                      # column names, dtypes, non-null counts
df.describe()                  # count, mean, std, min/max per numeric column

# Selecting
ages = df['age']                       # a Series (one column)
subset = df[['name', 'age', 'city']]    # a DataFrame (several columns)

# Filtering
adults = df[df['age'] >= 18]

# Grouping and aggregating
avg_age_by_city = df.groupby('city')['age'].mean()

df[df['age'] >= 18] looks unusual the first time you see it: the inner expression produces a Series of True/False values (one per row), and indexing the DataFrame with that boolean Series keeps only the rows where it's True. This "boolean mask" pattern is everywhere in Pandas code.

# Sort by a column, descending
df_sorted = df.sort_values('age', ascending=False)

# Derived columns are just assignments -- vectorized, like NumPy
df['is_adult'] = df['age'] >= 18
df['age_in_months'] = df['age'] * 12

4. Handling Missing Data & Merging Tables

Real datasets almost never arrive clean. Two skills you'll use in nearly every exercise from here on: finding and handling missing values, and combining data that lives across more than one table.

df.isna().sum()                 # count of missing values per column

# Three common strategies, in increasing order of "aggressiveness":
df['age'] = df['age'].fillna(df['age'].mean())   # fill with a sensible default
df_complete = df.dropna(subset=['email'])         # drop rows missing a required field
df_complete = df.dropna()                          # drop any row with any missing value

Which strategy is right depends entirely on why the value is missing and what you'll do with the column later — filling a numeric column's gaps with its mean is a common default, but it quietly shrinks that column's variance. You'll revisit this tradeoff properly when you build a training pipeline in Week 4.

orders = pd.read_csv('orders.csv')       # columns: order_id, customer_id, total
customers = pd.read_csv('customers.csv') # columns: customer_id, name, city

# Join on a shared key column -- like a SQL JOIN
enriched = orders.merge(customers, on='customer_id', how='left')

# how='left' keeps every order, even if a matching customer is missing
# how='inner' would drop orders with no matching customer_id

5. Exploratory Visualization with Matplotlib

Before you model anything, you look at it. A histogram catches a skewed distribution; a scatter plot catches a relationship (or a cluster of outliers) a table of numbers would hide. Matplotlib is the standard plotting library almost every other Python visualization tool builds on.

import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 2, figsize=(10, 4))

# Distribution of a single numeric column
axes[0].hist(df['age'], bins=20)
axes[0].set_title('Age distribution')

# Relationship between two numeric columns
axes[1].scatter(df['hours_studied'], df['score'])
axes[1].set_xlabel('Hours studied')
axes[1].set_ylabel('Score')

plt.tight_layout()
plt.show()

This kind of quick look — called exploratory data analysis (EDA) — is the very first thing you'll do with any new dataset in Week 5, before choosing a model or even deciding whether the data needs cleaning. A five-minute histogram now routinely saves an hour of debugging a model that "isn't learning" later.

6. Hands-on Exercise

students = [
    {"student_id": 1, "name": "Ada",    "hours_studied": 5},
    {"student_id": 2, "name": "Grace",  "hours_studied": 4},
    {"student_id": 3, "name": "Alan",   "hours_studied": None},
    {"student_id": 4, "name": "Barbara","hours_studied": 6},
    {"student_id": 5, "name": "Edsger", "hours_studied": 3},
]

scores = [
    {"student_id": 1, "score": 92},
    {"student_id": 2, "score": 88},
    {"student_id": 3, "score": 76},
    {"student_id": 4, "score": 95},
    # student_id 5 is missing from this table on purpose
]

7. Knowledge Check

Four quick questions. Expand each to check your answer.