Errors, Modules and the Standard Library — Python Primer

Something will go wrong. That is not pessimism, it is arithmetic: enough code runs enough times that the unlikely becomes the inevitable. The useful question is whether it goes wrong loudly, at the moment it happens, or quietly, in a report somebody presents to your board six months later.

This last chapter of the language section covers three practical things: failing usefully, splitting code across files, and the parts of the standard library you will actually reach for.

Errors

An exception stops your program and prints a traceback. Read tracebacks from the bottom up: the last line is what went wrong, the lines above are the path that got there.

Traceback (most recent call last):
  File "analysis.py", line 42, in <module>
    result = compute(df)
  File "analysis.py", line 30, in compute
    return df['revene'].sum()
KeyError: 'revene'

Bottom line first: a KeyError for 'revene'. That is a typo. The lines above tell you it happened inside compute, called from line 42.

Catch narrowly

try:
    value = int(user_input)
except ValueError:
    value = 0            # only catches the conversion failing

Catch the specific exception you expect. A bare except: catches everything, including your own typos and someone pressing Ctrl-C:

try:
    result = process(data)
except:                  # never do this
    result = None        # you have just hidden every possible bug

If you genuinely cannot predict the failure, catch Exception and log it, so the information survives:

try:
    result = process(data)
except Exception as e:
    logger.warning(f"processing failed: {type(e).__name__}: {e}")
    result = None

The exceptions you will meet most in data work:

Exception	Usually means
`KeyError`	A dict key or DataFrame column does not exist. Check the spelling
`IndexError`	A list index is out of range
`TypeError`	Wrong kind of thing, often `None` where a value was expected
`ValueError`	Right type, impossible value. `int("abc")`
`AttributeError`	Called a method that does not exist, often on `None`
`FileNotFoundError`	The path is wrong. Print it and look

AttributeError: 'NoneType' object has no attribute ... is worth recognising on sight. It nearly always means a function returned None further up and you used the result. The bug is not where the error appears.

Failing early, on purpose

The most valuable error handling in analysis code is not catching errors. It is causing them.

def load_orders(path):
    df = pd.read_csv(path)
    assert not df.order_id.duplicated().any(), "duplicate order ids"
    assert df.revenue.notna().all(), "revenue has missing values"
    return df

Two lines that turn a silent wrong answer into a loud stop. A pipeline that crashes is annoying. A pipeline that quietly produces a wrong number for six months is a catastrophe.

For anything another person might hit, raise a real exception with a message that helps:

if 'revenue' not in df.columns:
    raise ValueError(f"expected a 'revenue' column, got: {list(df.columns)}")

Including the actual columns means the reader diagnoses the problem from the error alone, without opening the file.

Modules and Imports

Every .py file is a module. Importing runs it and gives you its names.

import math                          # the whole module
from math import sqrt, pi            # specific names
import numpy as np                   # with an alias
import pandas as pd

The aliases np, pd, plt and sns are near-universal. Use them.

Avoid from module import *. It dumps every name into your namespace, you cannot tell where anything came from, and it silently overwrites things.

Organising a project

Once analysis outgrows one file:

project/
├── pyproject.toml
├── data/                 ← not in version control
├── notebooks/            ← exploration
│   └── 01-explore.ipynb
├── src/
│   └── myproject/
│       ├── __init__.py
│       ├── load.py       ← reading and validating
│       ├── clean.py      ← transformations
│       └── features.py
└── tests/
    └── test_clean.py

Then from a notebook:

from myproject.clean import clean_age

The rule that keeps this healthy: notebooks call functions, they do not define them. Anything worth keeping moves into src/ where it can be imported, tested and reused. Notebooks stay thin: load, call, plot, look.

Circular imports

If load.py imports from clean.py and clean.py imports from load.py, you get an ImportError that reads confusingly.

The cause is always a design problem. Two modules that need each other are really one module, or there is a third thing both depend on. Extract the shared part.

flowchart TB
    subgraph BAD["Circular — will not import"]
        A1["load.py"] <--> B1["clean.py"]
    end
    subgraph GOOD["Extract what both need"]
        A2["load.py"] --> C2["schema.py"]
        B2["clean.py"] --> C2
    end
    style BAD fill:#f7e8e8,stroke:#c02020
    style GOOD fill:#eef1fc,stroke:#2141c8

The Standard Library Worth Knowing

pathlib — file paths

Stop building paths with string concatenation. pathlib handles separators, works the same on Windows and Unix, and reads better.

from pathlib import Path

data = Path('data')
raw = data / 'raw' / 'orders.csv'      # / joins paths

raw.exists()
raw.suffix                              # '.csv'
raw.stem                                # 'orders'
raw.parent                              # Path('data/raw')

for f in data.glob('**/*.csv'):         # every CSV, recursively
    print(f)

Path('output').mkdir(parents=True, exist_ok=True)

exist_ok=True means “do not complain if it already exists”, which is nearly always what you want.

datetime — dates and times

from datetime import datetime, date, timedelta, timezone

now = datetime.now(timezone.utc)        # always store UTC
now + timedelta(days=7)
(date(2026, 1, 1) - date.today()).days

datetime.strptime('2026-03-15', '%Y-%m-%d')     # string → datetime
now.strftime('%Y-%m-%d %H:%M')                  # datetime → string
now.isoformat()                                  # the safe interchange format

Two rules that save real pain. Store UTC, convert only for display. And write ISO format, because 2026-03-04 is unambiguous where 03/04/2026 is not.

collections — better containers

from collections import Counter, defaultdict, namedtuple

Counter(['a', 'b', 'a']).most_common()
groups = defaultdict(list)
Point = namedtuple('Point', ['x', 'y'])

json — reading and writing JSON

import json

with open('config.json') as f:
    config = json.load(f)               # file → dict
with open('out.json', 'w') as f:
    json.dump(data, f, indent=2)        # dict → file

json.loads('{"a": 1}')                  # string → dict
json.dumps({'a': 1})                    # dict → string

load and dump work on files; loads and dumps work on strings. The trailing s is for “string”, which is the mnemonic that finally makes it stick.

JSON has no date type, so dates arrive as strings and you must parse them.

dataclasses — small structured records

from dataclasses import dataclass

@dataclass
class ModelConfig:
    learning_rate: float = 0.01
    epochs: int = 100
    shuffle: bool = True

config = ModelConfig(learning_rate=0.05)
config.epochs               # 100
print(config)               # ModelConfig(learning_rate=0.05, epochs=100, ...)

You get a constructor, a readable representation and equality for free. Better than a dictionary for configuration, because a typo in an attribute name raises rather than silently returning None.

logging — better than print

import logging

logging.basicConfig(level=logging.INFO,
                    format='%(asctime)s %(levelname)s %(message)s')
log = logging.getLogger(__name__)

log.info("loaded %d rows", len(df))
log.warning("%d rows had missing revenue", n_missing)

Print is fine while exploring. The moment code runs unattended you want timestamps, levels you can filter, and the ability to turn detail up without editing the file.

random — and when not to use it

import random
random.seed(42)              # reproducible
random.sample(range(100), 10)

For anything security-related use secrets instead; random is predictable by design. For numerical work prefer NumPy’s default_rng, covered in the NumPy chapter.

What Carries Forward

Read tracebacks from the bottom. The last line is the error.

Catch narrowly. A bare except: hides your own bugs.

Assert your assumptions. A loud crash beats a quiet wrong number by an enormous margin.

Notebooks call functions; modules define them.

That completes the language section. Next: NumPy, and the array thinking everything above it is built on.