Activity 7.1

This page is an ipython notebook, meaning you can download it and run the executable cells. It directly follows Activity 7.1. Reference answers appear after the questions, challenges, predictions, and reflections. They show one reasonable solution or interpretation, but other correct approaches may exist.

Debugging, Testing & Defensive Programming¶

Tools: Python, NumPy, pandas, pytest, Jupyter/VS Code, terminal

Learning objectives¶

By the end of this tutorial, you should be able to:

• distinguish syntax, runtime, and logic errors;

• read Python tracebacks;

• reduce bugs to minimal examples;

• validate assumptions using assertions and exceptions;

• write defensive statistical functions;

• create pytest unit tests;

• test expected exceptions;

• use approximate numerical comparisons;

• test deterministic and stochastic code;

• write fixtures and parametrized tests;

• add regression tests for discovered bugs;

• organize tests alongside reusable modules.

Part 0 --- Set up the Week 13 project¶

Create:

src/week13_utils.py
tests/test_week13_utils.py
notebooks/week13_testing.ipynb

bash pytest --version

bash pytest

Part 1 --- Three kinds of failure¶

Consider:

A¶

python if x > 2 print(x)

python x = 1 / 0

python def variance(x): return np.std( x )

text syntax error runtime error logic error

Part 2 --- Read a traceback¶

Run:

python def divide( x, y ): return x / y

def calculate(): return divide( 10, 0 )

calculate()

text exception type error message line where failure occurred function-call chain

Part 3 --- Create a minimal example¶

Suppose a large analysis fails during:

python X @ beta

python X = np.ones( ( 5, 3 ) )

beta = np.ones( 4 )

X @ beta

Part 4 --- Debug with object inspection¶

Create:

python x = np.array([ 1, 2, 3 ])

X = x.reshape( -1, 1 )

python type(x) x.shape X.shape x.dtype

Reflection¶

How might shape inspection have prevented a Week 9 broadcasting bug?

Reference answer¶

A minimal reproducible example is easier to debug because it removes unrelated code/data while preserving the failure. Small shapes make dimensional problems immediately visible.

Part 5 --- Assertions¶

Create:

python def standardize( x ): x = np.asarray( x, dtype=float )

assert x.ndim == 1
assert len(x) > 0

return (
    x - x.mean()
) / x.std()

python standardize([ 1, 2, 3 ])

python standardize( np.ones( ( 2, 2 ) ) )

Reference answer¶

Inspecting .shape would reveal incompatible dimensions before matrix multiplication or broadcasting.

Part 6 --- Identify a missing invariant¶

Try:

python standardize([ 5, 5, 5 ])

Part 7 --- Replace developer assertions with validation¶

Rewrite:

python def standardize( x ): x = np.asarray( x, dtype=float )

if x.ndim != 1:
    raise ValueError(
        "x must be one-dimensional."
    )

if len(x) == 0:
    raise ValueError(
        "x must not be empty."
    )

sd = x.std()

if sd == 0:
    raise ValueError(
        "x must have positive variability."
    )

return (
    x - x.mean()
) / sd

Reference answer¶

The missing invariant is a condition valid inputs must satisfy—for example matching lengths, nonempty input, or positive values. Add explicit validation at the boundary.

Part 8 --- Improve error messages¶

Modify each validation message to include the received value/shape where helpful.

For example:

python raise ValueError( “x must be one-dimensional; " f"received shape {x.shape}.” )

Reference answer¶

A public function should validate input and raise a specific informative exception rather than relying on an internal assertion that may be disabled or unclear to callers.

Part 9 --- Exception handling¶

Create:

python def read_numeric( text ): try: value = float( text ) except ValueError: return None else: return value

python read_numeric( “3.14” )

python read_numeric( “hello” )

Part 10 --- Catch specific exceptions¶

Compare:

python except:

python except ValueError:

Reference answer¶

Catch only exceptions you can handle meaningfully. Narrow exception handling preserves unexpected bugs instead of hiding them.

Part 11 --- Do not hide failures¶

Consider:

python def safe_analysis( data ): try: return run_analysis( data ) except Exception: return None

Reference answer¶

A bare except catches almost everything and can hide programming errors. Catch the narrowest expected exception or let the failure propagate.

Part 12 --- Create a convergence exception¶

In src/week13_utils.py:

python class ConvergenceError( RuntimeError ): pass

python def check_convergence( converged ): if not converged: raise ConvergenceError( "Numerical algorithm " “failed to converge.” )

Part 13 --- First pytest test¶

Add to src/week13_utils.py:

python def square(x): return x ** 2

text tests/test_week13_utils.py

python from src.week13_utils import ( square )

def test_square(): assert square(3) == 9

bash pytest

Reference answer¶

A basic pytest test calls the function on a known input and asserts that the returned value equals the expected result.

Part 14 --- Make a test fail deliberately¶

Change:

python assert square(3) == 10

bash pytest

Part 15 --- Arrange--Act--Assert¶

Rewrite:

python def test_square(): # Arrange x = 4

# Act
result = square(
    x
)

# Assert
assert result == 16

Reference answer¶

Arrange--Act--Assert separates setup, the operation, and verification, making tests easier to read and failures easier to localize.

Part 16 --- Test standardize¶

Move standardize() into:

text src/week13_utils.py

python def test_standardize_mean(): x = np.array([ 1.0, 2.0, 3.0 ])

z = standardize(
    x
)

assert np.isclose(
    z.mean(),
    0.0
)

Reference answer¶

This structure scales because setup remains explicit, the action is isolated, and assertions document the behavior being checked.

Part 17 --- Test approximate equality with pytest¶

Import:

python import pytest

python def test_standardize_sd(): x = np.array([ 1.0, 2.0, 3.0 ])

z = standardize(
    x
)

assert z.std() == pytest.approx(
    1.0
)

Part 18 --- Test invalid input¶

Write:

python def test_standardize_constant(): with pytest.raises( ValueError ): standardize([ 5, 5, 5 ])

bash pytest

Reference answer¶

Use pytest.approx, np.isclose, or np.allclose for floating-point results because exact equality can fail due to tiny rounding differences.

Part 19 --- Test the error message¶

Use:

python with pytest.raises( ValueError, match=“positive variability” ): standardize([ 5, 5, 5 ])

Part 20 --- Parametrized tests¶

Write:

python pytest.mark.parametrize( “x, expected”, [ (0, 0), (1, 1), (2, 4), (-3, 9) ] ) def test_square_many( x, expected ): assert ( square(x) == expected )

Reference answer¶

Checking the error message verifies that the function failed for the intended reason and provided a useful diagnostic.

Part 21 --- Create a fixture¶

Add:

python pytest.fixture def sample_scores(): return np.array([ 70.0, 80.0, 90.0 ])

python def test_score_mean( sample_scores ): assert ( sample_scores.mean() == 80.0 )

Reference answer¶

Parametrization removes duplicated tests that differ only in inputs and expected outputs.

Part 22 --- Test a Week 11 statistic¶

Add:

python def difference_in_means( x, y ): return ( np.mean(x) - np.mean(y) )

python x = np.array([ 4.0, 6.0 ])

y = np.array([ 1.0, 3.0 ])

text 3

Reference answer¶

A fixture provides reusable test setup while pytest controls when/how that setup is created.

Part 23 --- Add defensive validation¶

Modify:

python def difference_in_means( x, y ):

text empty x empty y

Reference answer¶

Test both valid behavior and invalid inputs, including the expected exception type/message.

Part 24 --- Test random code deterministically¶

Write:

python def simulate_normal( n, rng ): if n <= 0: raise ValueError( “n must be positive.” )

return rng.normal(
    size=n
)

python def test_simulate_normal_shape(): rng = np.random.default_rng( 556 )

result = simulate_normal(
    10,
    rng
)

assert result.shape == (
    10,
)

Reference answer¶

Test random code deterministically by passing a seeded RNG/fixed seed or by testing stable properties rather than a single uncontrolled random realization.

Part 25 --- Test reproducibility¶

Write:

python def test_simulate_normal_reproducible(): rng1 = np.random.default_rng( 556 )

rng2 = np.random.default_rng(
    556
)

x1 = simulate_normal(
    5,
    rng1
)

x2 = simulate_normal(
    5,
    rng2
)

assert np.array_equal(
    x1,
    x2
)

Reference answer¶

A reproducibility test checks that identical seeded RNG state and operation order produce identical output.

Part 26 --- Test probability bounds¶

Suppose a function returns a permutation p-value.

Write a property test:

python assert ( 0 <= p_value <= 1 )

Reference answer¶

A property test checks an invariant such as probability bounds rather than expecting a random sample statistic to equal a theoretical value exactly.

Part 27 --- Test bisection from Week 12¶

Move your bisection() function into src/.

Write a test using:

python def f(x): return x ** 2 - 2

python np.sqrt(2)

python pytest.approx( np.sqrt(2), abs=1e-7 )

Reference answer¶

For bisection, test a known root/bracket and assert that the returned value is close to the true root.

Part 28 --- Test invalid bisection brackets¶

Write:

python def test_bisection_invalid_bracket(): with pytest.raises( ValueError ): bisection( f, 2, 3 )

Reference answer¶

An invalid bracket should raise a clear exception because the sign-change assumption required by bisection is violated.

Part 29 --- Test optimization properties¶

For the simple loss:

python def objective(x): return ( x - 3 ) ** 2

text 3

text final objective <= initial objective

Part 30 --- Deliberately introduce a bug¶

Change the gradient from:

python 2 * (x - 3)

python 2 * (x + 3)

Reference answer¶

The introduced bug violates a mathematical property even though Python can run the code. A test should therefore fail and expose the logic error.

Part 31 --- Regression test workflow¶

Introduce a realistic bug:

python standardize( [5, 5, 5] )

text bug discovered ↓ test written ↓ test fails ↓ validation added ↓ test passes

Reference answer¶

A regression test captures a previously observed bug: reproduce it minimally, add a failing test, fix the code, and keep the test to prevent recurrence.

Part 32 --- Test a DataFrame function¶

Create:

python def require_columns( data, columns ): missing = [ column for column in columns if column not in data.columns ]

if missing:
    raise ValueError(
        "Missing required columns: "
        + ", ".join(
            missing
        )
    )

return True

Part 33 --- Minimal datasets¶

For testing, use tiny datasets.

Example:

python df = pd.DataFrame({ “id”: [1, 2], “score”: [80, 90] })

Reference answer¶

Minimal synthetic datasets make expected behavior easy to reason about and keep tests fast. Cover normal cases, boundaries, and invalid inputs.

Part 34 --- Test independence¶

Create two tests.

Make the first modify a global list.

Make the second depend on the modified list.

Run the suite in different orders if possible.

Reflection¶

Why is shared mutable state dangerous in tests?

Rewrite so each test creates its own data.

Reference answer¶

Shared mutable state makes test results depend on execution order. Create fresh data per test or use fixtures that return fresh objects.

Part 35 --- Build a Week 13 test suite¶

Your final test file should include tests for:

text square standardize difference_in_means simulate_normal bisection gradient descent required DataFrame columns

Part 36 --- Run subsets of tests¶

Run all:

bash pytest

bash pytest tests/test_week13_utils.py

bash pytest -k standardize

bash pytest -v

Part 37 --- Debug a failing test¶

Choose one passing test and temporarily break the corresponding function.

Then:

1. run only that test;

2. read the failure;

3. inspect inputs/intermediate values;

4. form a hypothesis;

5. fix one thing;

6. rerun;

7. run the full suite.

This is the core development cycle.

Reference answer¶

State the function contract first: valid/invalid inputs, expected output, exact cases, edge cases, and mathematical invariants. Then use the failing test to narrow the cause.

Part 38 --- Testing audit¶

For one statistical function, answer:

text What is its contract? What inputs are valid? What inputs are invalid? What normal case can be checked exactly? What edge case is important? What mathematical invariant should hold? Does it involve floating-point tolerance? Does it involve randomness? What past bug should become a regression test?

Part 39 --- Project structure¶

Your repository should now look approximately like:

text project/ ├── src/ │ └── week13_utils.py ├── tests/ │ └── test_week13_utils.py ├── notebooks/ │ └── week13_testing.ipynb └── ...

Part 40 --- Git checkpoint¶

Run:

bash pytest

bash git status git add . git commit -m “Complete Week 13 debugging and testing exercises” git push

Reference answer¶

Separating source functions from tests keeps responsibilities clear and supports standard test discovery and maintenance.

Part 41 --- Final reflection¶

Answer in Markdown.

1. Bugs¶

What is the difference between syntax, runtime, and logic errors?

2. Debugging¶

Why is debugging similar to hypothesis testing?

3. Tracebacks¶

What information does a traceback provide?

4. Assertions¶

What is an invariant?

5. Exceptions¶

When should a function raise an exception rather than continue?

6. Defensive programming¶

What does “fail fast” mean?

7. Unit testing¶

What should a unit test verify?

8. Floating point¶

Why use approximate equality?

9. Randomness¶

How can random code be tested reproducibly?

10. Regression tests¶

Why should a fixed bug usually lead to a new permanent test?

Completion checklist¶

• Created Week 13 notebook

• Created src/week13_utils.py

• Created tests/test_week13_utils.py

• Distinguished syntax/runtime/logic errors

• Read a traceback

• Built a minimal failing example

• Inspected shape/type/value diagnostics

• Added assertions

• Replaced public assertions with explicit validation

• Improved error messages

• Used try/except

• Caught specific exceptions

• Avoided silent exception swallowing

• Created a custom convergence exception

• Wrote a first pytest test

• Deliberately observed a failing test

• Used Arrange--Act--Assert

• Tested standardize

• Used pytest.approx

• Tested expected exceptions

• Checked an exception message

• Wrote parametrized tests

• Used a fixture

• Tested difference in means

• Tested random simulation deterministically

• Tested reproducibility

• Tested statistical invariants

• Tested bisection

• Tested invalid brackets

• Tested gradient-descent properties

• Introduced and caught a logic bug

• Created a regression test

• Tested DataFrame requirements

• Removed shared mutable state between tests

• Built a multi-function test suite

• Ran all and selected tests

• Debugged a failing test systematically

• Audited a function’s test design

• Ran the full suite before committing

• Committed Week 13 work to Git

• Pushed work to GitHub

What you should now understand¶

text Statistical function ↓ Contract ↓ Validation ↓ Normal + edge + failure cases ↓ Automated tests ↓ Bug discovered ↓ Regression test ↓ More reliable statistical software

Next week we move into **profiling, bottleneck identification,
vectorization, parallel computation, and Numba**.

Reference answer¶

Syntax errors prevent parsing, runtime errors occur during execution, and logic errors return incorrect results. Tracebacks locate failure paths, invariants define required properties, and fail-fast validation rejects invalid states early.