Python Packaging Guide

A comprehensive reference for packaging, versioning, and publishing Python libraries.

1. pyproject.toml Anatomy

pyproject.toml is the single source of truth for modern Python projects (PEP 621).

Build System

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

The [build-system] table tells pip and build which backend to use. Alternatives include flit-core, hatchling, and pdm-backend.

Project Metadata

[project]
name = "my-package"          # PyPI package name (use hyphens)
version = "1.0.0"            # Follow semver
requires-python = ">=3.9"    # Minimum Python version
dependencies = ["httpx"]     # Runtime dependencies

Key fields:

• name: Must be unique on PyPI. Use lowercase with hyphens.
• version: Semantic versioning (MAJOR.MINOR.PATCH).
• classifiers: Trove classifiers for PyPI search/filtering.
• optional-dependencies: Groups like [dev], [docs].

Tool Configuration

Tools like ruff, mypy, and pytest read config from [tool.*] tables, keeping everything in one file.

2. Versioning Strategies

Semantic Versioning (semver)

• MAJOR: Breaking API changes
• MINOR: New features, backwards-compatible
• PATCH: Bug fixes, backwards-compatible

Single-Source Version

Keep the version in one place. Options:

1. __init__.py: __version__ = "1.0.0" + version = attr: package.__version__

2. pyproject.toml: Canonical, read at build time

3. setuptools-scm: Derive version from git tags automatically

Calendar Versioning (calver)

Some projects use YYYY.MM.DD format (e.g., pip, Ubuntu). Useful for projects without a stable API contract.

3. Wheels vs Source Distributions

Source Distribution (sdist)

• A tarball of the source tree (.tar.gz)
• Requires a build step on the user's machine
• Controlled by MANIFEST.in

Wheel (bdist_wheel)

• A pre-built .whl file (a zip with metadata)
• Installs instantly — no build step required
• Always preferred when available

Rule of thumb: Always publish both sdist and wheel. Use python -m build which creates both by default.

4. The src/ Layout

my-project/
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── core.py
├── tests/
│   └── test_core.py
└── pyproject.toml

Why src/ layout?

• Prevents accidental imports from the project root
• Forces you to install the package before testing (catches packaging bugs)
• Standard for modern Python projects

5. PEP 561: Type Stub Packages

Add an empty py.typed marker file to your package root:

src/my_package/py.typed

Then include it in your package data:

[tool.setuptools.package-data]
my_package = ["py.typed"]

This tells type checkers (mypy, pyright) that your package ships inline type annotations.

6. Namespace Packages

For large organizations splitting a namespace across multiple packages:

# Package A installs: myorg/auth/...
# Package B installs: myorg/billing/...

Use implicit namespace packages (PEP 420) — omit __init__.py in the shared namespace directory. Each sub-package is an independent distribution.

7. Publishing to PyPI

Manual Publishing

python -m build
python -m twine check dist/*
python -m twine upload dist/*

TestPyPI First

Always test on TestPyPI before publishing to production:

python -m twine upload --repository testpypi dist/*
pip install --index-url https://test.pypi.org/simple/ my-package

CI Publishing (GitHub Actions)

- name: Publish to PyPI
  uses: pypa/gh-action-pypi-publish@release/v1
  with:
    password: ${{ secrets.PYPI_API_TOKEN }}

Use Trusted Publishers (OIDC) for keyless authentication — no API tokens needed.

8. MANIFEST.in Patterns

include LICENSE README.md
recursive-include src *.py py.typed
recursive-include tests *.py
global-exclude *.pyc __pycache__

Common mistakes:

• Forgetting to include py.typed in the sdist
• Including .git/, node_modules/, or CI configs
• Not testing the sdist: pip install dist/*.tar.gz

9. Pre-commit and CI Integration

Pre-commit

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    hooks:
      - id: ruff
      - id: ruff-format

GitHub Actions Matrix

strategy:
  matrix:
    python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
    os: [ubuntu-latest, macos-latest, windows-latest]

10. Checklist Before Publishing

• [ ] Version bumped in pyproject.toml (or __init__.py)
• [ ] CHANGELOG updated
• [ ] All tests pass (make check)
• [ ] py.typed marker included
• [ ] License file included
• [ ] README renders correctly on PyPI (check with twine check)
• [ ] Tested install from sdist: pip install dist/*.tar.gz
• [ ] Tested install from wheel: pip install dist/*.whl
• [ ] Published to TestPyPI first