Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Activity 4.1

This page is an ipython notebook, meaning you can download it and run the executable cells. It directly follows Activity 4.1. Reference answers appear after the questions, challenges, predictions, and reflections. They show one reasonable solution or interpretation, but other correct approaches may exist.

Modular Programming & Functions

Estimated time: 2–3 hours
Tools: Python, pandas, Jupyter/VS Code

Learning objectives

By the end of this tutorial, you should be able to:

  • define reusable Python functions;

  • distinguish parameters from arguments;

  • return values rather than relying on printing;

  • use defaults, keyword arguments, type hints, and docstrings;

  • recognize pure functions and side effects;

  • understand local and global scope;

  • reduce dependence on global variables;

  • compose small functions into workflows;

  • use map(), filter(), and reduce();

  • recognize when comprehensions are more readable;

  • apply functions to pandas data;

  • move reusable functions into modules.

Part 0 — Set up

Create:

notebooks/week07_functions.ipynb

and:

src/week07_utils.py

Import:

import pandas as pd
import numpy as np

Part 1 — Repeated code

scores_a = [81, 88, 92, 79]
scores_b = [74, 85, 90, 95]

Calculate:

mean_a = sum(scores_a) / len(scores_a)
mean_b = sum(scores_b) / len(scores_b)

Question

What is duplicated here?

What happens if this calculation occurs in 20 places?

Part 2 — Write a function

def mean_score(values):
    return sum(values) / len(values)

Run:

mean_score(scores_a)
mean_score(scores_b)

What has been abstracted away?

Reference answer

The mean calculation is duplicated. A function abstracts the repeated operation so changes are made in one place rather than many.

Part 3 — Parameters and arguments

For:

def mean_score(values):
    return sum(values) / len(values)

identify:

function name:
parameter:
argument in mean_score(scores_a):

Part 4 — Return vs. print

Compare:

def print_mean(values):
    print(
        sum(values) / len(values)
    )

with:

def return_mean(values):
    return (
        sum(values) / len(values)
    )

Run:

x = print_mean(scores_a)
y = return_mean(scores_a)

Inspect:

x
y

Why is the returned result more reusable?

Reference answer

return gives a reusable value back to the caller; print only displays text. Returned values can be stored, tested, combined, or passed to other functions.

Part 5 — Multiple outputs

def summarize(values):
    return (
        min(values),
        max(values),
        sum(values) / len(values)
    )

Then:

minimum, maximum, mean = (
    summarize(scores_a)
)

Challenge

Also return the number of observations.

Reference answer

Returning multiple outputs lets the caller reuse each result programmatically instead of parsing printed output.

Part 6 — Default arguments

def classify_score(
    score,
    threshold=80
):
    if score >= threshold:
        return "high"

    return "low"

Try:

classify_score(85)
classify_score(85, threshold=90)

What behavior does the default encode?

Reference answer

A default argument specifies the function’s standard behavior when the caller omits that argument, while still allowing an override.

Part 7 — Keyword arguments

def simulate_scores(
    n,
    mean,
    sd,
    seed
):
    rng = np.random.default_rng(seed)

    return rng.normal(
        mean,
        sd,
        n
    )

Compare:

simulate_scores(
    100,
    80,
    10,
    556
)

with:

simulate_scores(
    n=100,
    mean=80,
    sd=10,
    seed=556
)

Which is easier to interpret?

Reference answer

Keyword arguments improve readability because each supplied value is explicitly associated with a parameter name.

Part 8 — Type hints

def classify_score(
    score: float,
    threshold: float = 80
) -> str:
    if score >= threshold:
        return "high"

    return "low"

Inspect:

classify_score.__annotations__

What information do the annotations provide?

Reference answer

Type hints document expected input/output types and help editors/static checkers catch likely mistakes. They do not usually enforce types at runtime.

Part 9 — Docstrings

def classify_score(
    score: float,
    threshold: float = 80
) -> str:
    """Classify a score using a threshold."""
    if score >= threshold:
        return "high"

    return "low"

Inspect:

help(classify_score)

Challenge

Expand the docstring to describe:

  • score;

  • threshold;

  • return value.

Reference answer

A useful docstring should describe the function purpose, the score and threshold parameters, the return value, and important assumptions/errors.

Part 10 — Pure function

def square(x):
    return x ** 2

Run:

square(4)
square(4)
square(4)

Does anything outside the function change?

Reference answer

A pure function depends only on its arguments and avoids hidden side effects. The same inputs therefore produce the same output, which simplifies testing.

Part 11 — Side effects

results = []
def record_result(x):
    results.append(x)

Run:

record_result(10)
record_result(20)
results

What external state changed?

Part 12 — Mutation of inputs

def append_score(
    scores,
    score
):
    scores.append(score)

Then:

values = [80, 90]

append_score(
    values,
    100
)

values

Why could this surprise a caller?

Reference answer

The function changed external state or mutated an input object. That can surprise a caller because later code sees a changed object even if the caller did not explicitly request mutation.

Part 13 — Non-mutating alternative

def append_score(
    scores,
    score
):
    return scores + [score]

Then:

values = [80, 90]

new_values = append_score(
    values,
    100
)

Inspect:

values
new_values

Which is easier to reason about?

Reference answer

A non-mutating alternative creates and returns a new object rather than silently altering the caller’s object, making data flow easier to reason about.

Part 14 — Local scope

def calculate():
    local_value = 42
    return local_value

Run:

calculate()

Then try:

local_value

Why is the name unavailable outside the function?

Part 15 — Global scope

threshold = 80
def classify(score):
    if score >= threshold:
        return "high"

    return "low"

Run:

classify(85)

Change:

threshold = 90

Run again:

classify(85)

Why did the same function call change behavior?

Reference answer

A local variable exists only inside the function call that created it, so its name is unavailable outside that local scope.

Part 16 — Remove hidden global dependence

Rewrite:

def classify(
    score,
    threshold=80
):
    if score >= threshold:
        return "high"

    return "low"

Now:

classify(85)
classify(85, threshold=90)

Why is this interface clearer?

Reference answer

The same function call changed because the function depended on hidden global state. Passing the value explicitly as an argument removes that hidden dependency.

Part 17 — LEGB exploration

x = "global"
def outer():
    x = "enclosing"

    def inner():
        x = "local"
        return x

    return inner()

Run:

outer()

Remove the local x from inner() and repeat.

Then remove the enclosing x.

Complete:

L:
E:
G:
B:

Explain the lookup order.

Reference answer

LEGB means Python searches Local, Enclosing, Global, then Built-in scopes and uses the first matching name.

Part 18 — Refactor a large function

Start:

def analyze(data):
    clean = data.dropna(
        subset=["age", "score"]
    )

    clean = clean.loc[
        clean["age"] >= 18
    ]

    mean = clean["score"].mean()

    return mean

Split it into:

def remove_missing(data):
    ...
def select_adults(data):
    ...
def mean_score(data):
    ...

Reference answer

Breaking a large function into smaller functions separates responsibilities so each component can be tested and reused independently.

Part 19 — Compose the functions

Use:

clean = remove_missing(df)
adults = select_adults(clean)
result = mean_score(adults)

Why might this be easier to test than one large function?

Part 20 — Create a DataFrame

df = pd.DataFrame({
    "id": [1, 2, 3, 4, 5, 6],
    "age": [24, 31, 17, 42, 29, 35],
    "group": ["A", "B", "A", "B", "A", "B"],
    "score": [81, 94, 88, np.nan, 91, 85]
})

Use your functions from the previous parts.

Reference answer

Small functions with DataFrame inputs/outputs create clear interfaces: each transformation can be tested using a small synthetic table.

Part 21 — Functions are objects

def square(x):
    return x ** 2

Then:

f = square
f(5)
type(square)

What does it mean to say functions are first-class objects?

Part 22 — Higher-order function

def apply_twice(
    function,
    value
):
    return function(
        function(value)
    )

Run:

apply_twice(
    square,
    2
)

Challenge

def increment(x):
    return x + 1

Predict:

apply_twice(
    increment,
    10
)

before running it.

Reference answer

Functions are first-class objects because they can be assigned, stored, passed as arguments, and returned. A higher-order function accepts or returns another function.

Part 23 — map()

values = [1, 2, 3, 4, 5]
mapped = map(
    square,
    values
)

Inspect:

mapped
list(mapped)

Why does map() not immediately return a list?

Part 24 — map() vs. comprehension

Compare:

list(
    map(
        square,
        values
    )
)

with:

[
    square(x)
    for x in values
]

Which is clearer?

Reference answer

map() returns a lazy iterator in Python 3, while a list comprehension constructs the result immediately. For simple transformations, the comprehension is often clearer.

Part 25 — filter()

def is_even(x):
    return x % 2 == 0
filtered = filter(
    is_even,
    values
)

list(filtered)

Now write the equivalent comprehension.

Part 26 — Statistical predicate

def is_high_score(
    score,
    threshold=85
):
    return score >= threshold

Use:

scores = [72, 81, 91, 68, 88, 95]

Then:

list(
    filter(
        is_high_score,
        scores
    )
)

Compare with a comprehension.

Reference answer

A predicate returns True/False. The equivalent comprehension keeps the values for which that Boolean condition is true.

Part 27 — reduce()

from functools import reduce
def add(x, y):
    return x + y

Run:

reduce(
    add,
    [1, 2, 3, 4]
)

Write the sequence of reductions manually.

Reference answer

reduce() repeatedly combines an accumulator with the next item until one result remains. Built-ins are usually clearer when they already express the desired reduction.

Part 28 — Reduction and statistics

scores = [81, 94, 88, 91]

Calculate:

total = reduce(
    add,
    scores
)

mean = total / len(scores)

Why is built-in sum() preferable here?

The goal is to understand reduction rather than replace clearer built-ins.

Reference answer

A manual reduction starts with an accumulator and repeatedly combines it with each subsequent value. This is the general pattern behind reductions.

Part 29 — Lambda expressions

Compare:

def square(x):
    return x ** 2

with:

lambda x: x ** 2

Use:

list(
    map(
        lambda x: x ** 2,
        values
    )
)

Challenge

Use a lambda with filter() to retain values above 3.

Reference answer

sum() is preferable because it clearly communicates intent and is well tested/optimized. A custom reduction is mainly useful for understanding the pattern.

Part 30 — When not to use lambda

Would this be clear?

lambda x: (
    complicated_transformation(x)
    if complicated_condition(x)
    else another_transformation(x)
)

Use lambdas for short, local behavior—not to hide complicated logic.

Reference answer

Example: list(filter(lambda x: x > 3, values)). Use a lambda only when the operation is short and obvious; otherwise a named function is clearer.

Part 31 — Apply a function to pandas

def categorize_score(score):
    if pd.isna(score):
        return "missing"

    if score >= 90:
        return "high"

    if score >= 80:
        return "medium"

    return "low"

Apply:

df["performance"] = (
    df["score"]
    .map(categorize_score)
)

Inspect df.

Part 32 — Reusable DataFrame function

def select_group(
    data: pd.DataFrame,
    group: str
) -> pd.DataFrame:
    return data.loc[
        data["group"] == group
    ].copy()

Test:

select_group(df, "A")
select_group(df, "B")

Part 33 — Add a defensive check

def select_group(
    data: pd.DataFrame,
    group: str
) -> pd.DataFrame:
    if "group" not in data.columns:
        raise ValueError(
            "data must contain a group column"
        )

    return data.loc[
        data["group"] == group
    ].copy()

Test on a DataFrame without group.

Why is an informative error useful?

Part 34 — Build an analysis pipeline

def remove_missing_scores(data):
    return data.dropna(
        subset=["score"]
    ).copy()
def select_adults(data):
    return data.loc[
        data["age"] >= 18
    ].copy()
def summarize_scores(data):
    return {
        "n": len(data),
        "mean": data["score"].mean(),
        "sd": data["score"].std()
    }

Use:

clean = remove_missing_scores(df)
adults = select_adults(clean)
summary = summarize_scores(adults)

Reference answer

A clean pipeline is a sequence of small functions with explicit inputs/outputs, for example load → validate → transform → summarize. Avoid hidden globals and accidental mutation.

Part 35 — One-expression composition

Try:

summary = summarize_scores(
    select_adults(
        remove_missing_scores(df)
    )
)

Is this clearer or less clear than named intermediate objects?

Readable code matters more than compact code.

Part 36 — Move functions into a module

Open:

src/week07_utils.py

Move these functions into it:

categorize_score
select_group
remove_missing_scores
select_adults
summarize_scores

Part 37 — Import your module

From the notebook:

from src.week07_utils import (
    categorize_score,
    select_group,
    remove_missing_scores,
    select_adults,
    summarize_scores
)

Use one function.

How has the notebook changed now that reusable logic lives elsewhere?

Part 38 — Inspect the module boundary

Your project now resembles:

project/
├── notebooks/
│   └── week07_functions.ipynb
├── src/
│   └── week07_utils.py
├── data/
├── figures/
└── tests/

Which code belongs in the notebook?

Which belongs in src/?

Part 39 — Mini refactoring challenge

Start with:

df = pd.read_csv("data/study.csv")

df = df.dropna(
    subset=["score"]
)

df = df.loc[
    df["age"] >= 18
]

df["performance"] = (
    df["score"]
    .map(categorize_score)
)

summary = {
    "n": len(df),
    "mean": df["score"].mean(),
    "sd": df["score"].std()
}

Refactor into a workflow resembling:

raw = load_data(...)
clean = clean_data(raw)
analysis = prepare_analysis(clean)
summary = summarize_scores(analysis)

Reference answer

A strong refactor removes duplicated logic, extracts conceptually distinct work into named functions, makes dependencies explicit, and gives each function one clear responsibility.

Part 40 — Function design critique

For each function, ask:

Interface

  • Are inputs explicit?

  • Is the return value clear?

Purity

  • Does it unexpectedly mutate data?

  • Does it rely on globals?

Scope

  • Are internal variables local?

Responsibility

  • Does it do one coherent job?

Documentation

  • Does it need a docstring?

  • Would type hints improve clarity?

Revise at least two functions.

Part 41 — Git checkpoint

Run:

git status

Stage:

git add .

Commit:

git commit -m "Complete Week 7 modular programming exercises"

Push:

git push

Part 42 — Final reflection

1. Functions

Why are functions useful in statistical workflows?

2. Return values

Why is returning generally more reusable than printing?

3. Purity

What makes a function pure?

4. Side effects

Give two examples.

5. Scope

What is the difference between local and global scope?

6. Global state

Why can global variables make functions harder to test?

7. Higher-order functions

What does it mean for a function to accept another function?

8. Functional patterns

Explain map, filter, and reduce conceptually.

9. Python style

Why might a comprehension be preferable to map() or filter()?

10. Modularity

Why move functions from notebooks into modules?

Completion checklist

  • Created Week 7 notebook

  • Created a Week 7 Python module

  • Refactored duplicated code into functions

  • Distinguished parameters and arguments

  • Used return values

  • Returned multiple values

  • Used default arguments

  • Used keyword arguments

  • Added type hints

  • Added docstrings

  • Identified a pure function

  • Identified side effects

  • Compared mutating and non-mutating functions

  • Explored local and global scope

  • Demonstrated the LEGB rule

  • Removed hidden global dependencies

  • Split one large function into smaller ones

  • Composed functions

  • Treated a function as an object

  • Wrote a higher-order function

  • Used map()

  • Used filter()

  • Used reduce()

  • Compared functional tools with comprehensions

  • Used lambda expressions

  • Applied a function to a pandas Series

  • Wrote a reusable DataFrame function

  • Added a defensive validation check

  • Built a modular analysis pipeline

  • Imported functions from src/

  • Refactored an analysis into reusable components

  • Committed Week 7 work to Git

  • Pushed work to GitHub

What you should now understand

Repeated code
      ↓
Functions
      ↓
Explicit inputs + outputs
      ↓
Pure transformations
      ↓
Small reusable components
      ↓
Composition
      ↓
Modules
      ↓
Maintainable statistical software

Next week we will use these modular components inside reproducible reports and literate programming workflows.

Reference answer

Functions improve reproducibility by making operations explicit and reusable. Prefer reusable return values, avoid hidden state/mutation, document interfaces, and use higher-order tools only when they improve clarity.

9. Reference Activity Notebooks
Activity 3.1
9. Reference Activity Notebooks
Activity 4.2