Chapter 9: Python Modules and Packages

As programs grow, organisation becomes as important as syntax. Python’s answer is the module system. A module is a single .py file that defines names (functions, classes, variables) inside its own namespace. A package is a directory that groups related modules together so you can build clear, reusable libraries rather than a single monolithic script.

Importing brings names from one module into another. The import machinery locates a module (or package), compiles it to bytecode if needed, executes it once to create its globals, then caches the result in sys.modules. Later imports reuse that cached module. This behaviour is simple to use in small scripts, yet it scales to large codebases because each file has a clean boundary and a predictable namespace.

Packages let you layer structure. You can arrange modules into folders that mirror the concepts of your project, then import with dotted paths. Since Python 3.3, namespace packages also allow package directories without an __init__.py file (useful when splitting a package across multiple locations). Traditional packages still use __init__.py to initialise package-level behaviour or expose selected symbols for import.

Import mechanics and aliasing

The import statement is one of Python’s most important features. It lets you bring external code into the current namespace so that you can reuse functionality without rewriting it. When Python encounters an import, it searches through a list of directories defined in sys.path, which includes the current working directory, the standard library, and any site-packages directories for installed modules.

import math
print(math.sqrt(16))  # 4.0

Once imported, a module is executed only once per session. After that, the interpreter retrieves it from the cache stored in sys.modules. This means that repeated imports are efficient and fast. If you need to reload a module after modifying it interactively, you can use importlib.reload().

import importlib
importlib.reload(math)

Python also supports importing with an alias using the as keyword. This is a simple yet powerful way to shorten long module names or clarify context in your code.

import numpy as np
import datetime as dt

You can also import specific names from a module to avoid repeating the module prefix each time you use a function or class:

from math import sqrt, pi
print(sqrt(9) * pi)

Understanding how imports resolve, execute, and cache helps prevent confusion when working with multiple files. It ensures that your codebase remains modular, efficient, and predictable.

The module search path

When Python imports a module, it follows a well-defined search process looking through a sequence of directories known as the module search path. This path is stored in the list sys.path and is initialised each time the interpreter starts.

Understanding the search path is crucial when structuring projects or debugging import errors. By knowing where Python looks (and in what order) you can predict and control how your modules are found and loaded. The typical search order is:

• The directory containing the script that was run (or the current directory in interactive mode)
• The standard library directories
• Site-specific directories such as site-packages, where third-party modules are installed

import sys
for path in sys.path:
  print(path)

💡 You can temporarily alter sys.path when experimenting, but in production it is better to manage module locations using environment variables like PYTHONPATH or proper package installation tools such as pip.

This list is just an ordinary Python list, so you can inspect or modify it at runtime. Appending new directories lets you import modules from custom locations, though this should be done with care.

import sys
sys.path.append('/path/to/my/modules')

⚠️ The current working directory is always searched first. This can lead to accidental shadowing if a local file shares the same name as a standard module (for example, a file named random.py could override the built-in random module).

Creating your own modules

Any Python file can act as a module. If you save a script as mymodule.py and import it from another file, Python will execute the code once and make its functions, classes, and variables available through the module’s namespace. This is the simplest and most direct way to structure reusable code.

# mymodule.py
def greet(name):
  return f"Hello, {name}!"

# main.py
import mymodule
print(mymodule.greet("Ada"))

When the import statement runs, Python creates a module object and stores it in sys.modules. The module’s global variables become attributes of that object. This means you can inspect or even modify them dynamically.

print(dir(mymodule))
print(mymodule.__name__)

💡 By default, the module name is the filename without its .py extension. Keeping module names short, descriptive, and lowercase (such as utils.py or helpers.py) helps readability and follows PEP 8 conventions.

Modules can contain any combination of top-level functions, classes, and variables. However, code at the top level runs as soon as the module is imported. For scripts that can also be run directly, you can add a special block to distinguish those cases:

if __name__ == "__main__":
  print(greet("world"))

⚠️ The conditional if __name__ == "__main__" ensures that certain code runs only when the file is executed directly, not when imported. This pattern keeps your modules reusable while still allowing standalone testing.

Package structure and __init__.py

When you outgrow single files, you can group related modules into a directory. This directory, known as a package, must either contain a special file named __init__.py or be recognised as a namespace package (available since Python 3.3). Packages help you organise functionality into layers, keeping large projects manageable and logical.

A basic package might look like this:

shapes/
  __init__.py
  circle.py
  square.py

After creating this structure, you can import from it using dotted paths:

import shapes.circle
from shapes.square import area

The __init__.py file is executed when the package is imported. It can be empty, or it can initialise variables, set up imports, or define what symbols should be exposed when someone uses from package import *.

# shapes/__init__.py
from .circle import area as circle_area
from .square import area as square_area

💡 Use __all__ inside __init__.py to explicitly declare which names are public. This keeps your package interface clean and avoids unintentionally exposing helper functions or constants.

# shapes/__init__.py
__all__ = ["circle_area", "square_area"]

Namespace packages, introduced in PEP 420, are directories without an __init__.py file. They allow a single package to be spread across multiple locations. This is especially useful in large ecosystems or modular plugin systems.

⚠️ Avoid mixing traditional and namespace packages in the same hierarchy. It can make import resolution unpredictable and cause subtle bugs if modules exist in multiple paths.

The Python standard library

Python ships with a rich collection of modules known as the standard library. These modules cover everyday tasks (file handling, text processing, dates and times, data formats, networking, concurrency, testing) so you can build useful programs without installing anything else. The standard library is designed to be consistent and well documented, and it follows the language’s philosophy of clarity and practicality.

You load a module with import and then call its functions or use its classes. Each import adds focused capabilities without adding external dependencies.

import math
from pathlib import Path

print(math.sqrt(49)) # 7.0

p = Path("notes.txt")
print(p.exists())    # True or False

Files, paths, and the operating system

• pathlib (object oriented paths across platforms)
• os, os.path, shutil (environment, process, and filesystem utilities)
• glob, fnmatch (filename pattern matching)
• tempfile, io (temporary files and file-like interfaces)

from pathlib import Path
import shutil

src = Path("images")
dst = Path("backup/images")
dst.mkdir(parents=True, exist_ok=True)

for img in src.glob("*.png"):
  shutil.copy2(img, dst / img.name)

💡 Prefer pathlib for path work. It is clearer and more portable than combining strings with os.path.

Text, data formats, and persistence

• json, csv, configparser (common interchange and configuration formats)
• sqlite3 (embedded SQL database stored in a single file)
• pickle (Python specific serialization)

import json
from pathlib import Path

prefs = {"theme": "dark", "items_per_page": 20}
Path("prefs.json").write_text(json.dumps(prefs, indent=2), encoding="utf-8")

loaded = json.loads(Path("prefs.json").read_text(encoding="utf-8"))
print(loaded["theme"])  # dark

Numeric, statistics, and randomness

• math, cmath (real and complex math)
• statistics (mean, median, variance)
• random, secrets (pseudorandom utilities and secure tokens)
• decimal, fractions (exact arithmetic where it matters)

from statistics import mean
from decimal import Decimal

scores = [8.5, 9.0, 7.5]
print(mean(scores))  # 8.333...
price = Decimal("19.99") + Decimal("0.01")
print(price)         # 20.00 (no binary rounding issues)

Dates, times, and time zones

• datetime (dates, times, timedeltas)
• zoneinfo (IANA time zones, Python 3.9+)

from datetime import datetime
from zoneinfo import ZoneInfo

meeting = datetime(2025, 10, 22, 15, 0, tzinfo=ZoneInfo("Europe/London"))
print(meeting.isoformat())

Collections and functional tools

• collections (Counter, deque, defaultdict, namedtuple)
• itertools (efficient iteration building blocks)
• functools (lru_cache, partial, cached_property)
• dataclasses, enum (lightweight data containers and enumerations)

from collections import Counter
from functools import lru_cache

counts = Counter("mississippi")
print(counts.most_common(2))  # [('s', 4), ('i', 4)]

@lru_cache(maxsize=256)
def fib(n):
  return n if n < 2 else fib(n-1) + fib(n-2)

print(fib(30))

Networking and internet

• urllib.request, urllib.parse (HTTP requests and URL parsing)
• http.client, http.server (low level HTTP and a simple dev server)
• socket (raw network sockets)

from urllib.request import urlopen

with urlopen("https://example.com") as resp:
  print(resp.status)
  print(resp.read(100))

Concurrency

• threading (lightweight threads)
• multiprocessing (separate processes for CPU bound work)
• concurrent.futures (thread and process pools with a unified interface)
• asyncio (asynchronous I/O with async and await)

from concurrent.futures import ThreadPoolExecutor

def square(n): return n*n

with ThreadPoolExecutor() as pool:
  results = list(pool.map(square, range(5)))
print(results)  # [0, 1, 4, 9, 16]

CLI, logging, and testing

• argparse (command line interfaces)
• logging, traceback (observability and error reports)
• unittest, doctest (testing frameworks)

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--limit", type=int, default=10)
args = parser.parse_args()
print(args.limit)

Imports, packaging, and resources

• importlib, pkgutil (introspection and dynamic imports)
• importlib.resources (read data files packaged with your code)
• venv (create isolated environments)
• typing (type hints for better tooling and clarity)

from importlib.resources import files

data_path = files("mypackage.data") / "schema.json"
print(data_path)  # path inside the installed package

⚠️ The standard library evolves across versions (3.10 through 3.14 in this book). Names can move or gain improvements. When something behaves differently on your system, check the versioned docs for your Python release.

The standard library is a practical toolbox. Learn a few modules well (path handling, JSON, datetime, collections, logging) and add others as your projects require them. This approach keeps code lean, portable, and easy to maintain.

Installing and managing third-party packages

Python’s power grows when you add libraries from the wider ecosystem. You install these libraries with pip and keep your project dependencies isolated with venv. The basic flow is: create a virtual environment for your project, activate it, install what you need, then record the versions so you can reproduce the environment later.

Create and use a virtual environment

# Create a virtual environment in the ".venv" folder
python -m venv .venv

# Activate it (macOS/Linux)
source .venv/bin/activate

# Activate it (Windows PowerShell)
.venv\Scripts\Activate.ps1

💡 Always prefer python -m pip over a bare pip command. It guarantees you are using the same interpreter’s installer as your active environment.

Install, upgrade, and remove packages

# Install a package (latest version)
python -m pip install requests

# Install a specific version
python -m pip install "requests==2.32.3"

# Upgrade a package
python -m pip install --upgrade requests

# Uninstall a package
python -m pip uninstall requests

# See what is installed
python -m pip list

Record and reproduce environments

# Freeze the current environment to a requirements file
python -m pip freeze > requirements.txt

# Recreate the same environment elsewhere
python -m pip install -r requirements.txt

⚠️ A simple requirements.txt pins all transitive versions. This is ideal for applications you deploy. For libraries you publish, pin less aggressively and test across ranges.

Installing your own project

If your project has a pyproject.toml with a [project] table, you can install it into the environment (useful for editable imports during development):

# Editable install (live changes without reinstalls)
python -m pip install -e .

Platform-friendly launcher notes

# Windows can use the Python launcher
py -m venv .venv
py -m pip install requests

# Many Unix systems use python3
python3 -m venv .venv
python3 -m pip install requests

⚠️ Avoid using administrator privileges (for example, sudo pip install) for global installs. Prefer per-project virtual environments to prevent conflicts and permission issues.

Command-line tools vs project libraries

Some packages are primarily command-line tools rather than libraries you import. Install these in isolated contexts so they do not clash with your project dependencies:

# Install a CLI tool globally but isolated per tool
python -m pip install --user pipx
pipx install black

💡 Use pipx for CLI tools (formatters, linters, scaffolding utilities). Use venv + pip for your application’s importable libraries.

Troubleshooting

# If pip itself needs an update
python -m pip install --upgrade pip

# If an install fails, clear the build cache and try again
python -m pip cache purge
python -m pip install PACKAGE_NAME

# Inspect where a package came from
python -m pip show PACKAGE_NAME

With a reliable pattern (create a virtual environment, install with python -m pip, freeze versions for apps, and keep tools separated with pipx), you can manage third-party packages confidently and reproducibly.