← Back to all products

Database Migration Toolkit

$39

Schema migration frameworks, data migration scripts, zero-downtime migration patterns, and rollback procedures.

33 files
PythonMarkdownSQLJSONPostgreSQL

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free Sample

File Structure 33 files

database-migration-toolkit/ ├── LICENSE ├── README.md ├── checklists/ │ └── migration-review-checklist.md ├── docs/ │ ├── cross-engine-checklist.md │ ├── data-backfill.md │ ├── online-schema-change.md │ ├── rollback-playbook.md │ ├── versioned-migrations.md │ └── zero-downtime-expand-contract.md ├── examples/ │ ├── backfill_batched.sql │ └── postgres_expand_contract/ │ ├── 00_setup.sql │ ├── 01_expand.sql │ ├── 02_backfill.sql │ ├── 03_contract.sql │ └── README.md ├── free-sample.zip ├── guide/ │ ├── 01_why-this-exists.md │ ├── 02_requirements.md │ ├── 03_cli-reference.md │ └── 04_license.md ├── index.html ├── interactive.html ├── migrations/ │ ├── V001__create_core_schema.down.sql │ ├── V001__create_core_schema.up.sql │ ├── V002__add_indexes.down.sql │ ├── V002__add_indexes.up.sql │ ├── V003__expand_split_customer_name.down.sql │ ├── V003__expand_split_customer_name.up.sql │ ├── V004__contract_drop_full_name.down.sql │ └── V004__contract_drop_full_name.up.sql ├── runner/ │ └── migrate.py └── tests/ └── test_migrate.py

Documentation Preview README excerpt

Database Migration Toolkit

A dependency-free, runnable toolkit for changing database schemas the safe way:

versioned up/down migrations, a tiny SQL runner you can actually read, and the

expand/contract patterns that let you ship schema changes to a busy database **without

downtime**. Everything here runs on the Python standard library and SQLite out of the box —

no pip install, no services to start — and the same patterns scale up to production

PostgreSQL and MySQL.

This is not another opaque migration framework. It's a small runner plus a set of worked,

proven examples and field-tested documentation, so you can learn the patterns and lift

the code.

Why this exists

The SQL in a schema migration is rarely the hard part. The hard parts are:

  • Two code versions hit the database at once during a rolling deploy — the schema has

to be valid for both.

  • A single ALTER or UPDATE can lock or rewrite a huge table and stall production.
  • Rolling back is often impossible (data's gone) or unsafe (DDL auto-commits on MySQL).
  • Teams collide on version numbers and silently edit already-applied migrations.

This toolkit gives you a concrete, working model for all four, with the trade-offs written

down.

Features

  • runner/migrate.py — a ~480-line, stdlib-only migration runner: ordered

up/down, a schema_migrations tracking table, SHA-256 checksums that detect a

changed-after-apply migration, status, validate, and a new scaffolder. Each

migration runs in one transaction.

  • Drives 3 engines through one thin seam: SQLite with zero setup (default), plus

PostgreSQL and MySQL by passing a --url (install that driver).

  • A proven expand/contract example — four bundled migrations that split a full_name

column into first_name/last_name with zero downtime, fully reversible end to end.

  • Production PostgreSQL DDL — the same change with CREATE INDEX CONCURRENTLY-class

concerns, a transition trigger, a batched backfill procedure, and the lock-light

NOT NULL technique.

  • A reusable batched-backfill template (PostgreSQL + MySQL) using keyset pagination.
  • Six field guides covering versioning, zero-downtime expand/contract, online schema

change per engine, the rollback playbook, large-table backfills, and a cross-engine

cheat sheet — plus a reviewer's checklist for migration PRs.

What's inside


database-migration-toolkit/
├── README.md
├── LICENSE
├── runner/
│   └── migrate.py                       # the stdlib migration runner (CLI)
├── migrations/                          # runnable on SQLite with zero setup
│   ├── V001__create_core_schema.up.sql / .down.sql
│   ├── V002__add_indexes.up.sql / .down.sql
│   ├── V003__expand_split_customer_name.up.sql / .down.sql   # EXPAND
│   └── V004__contract_drop_full_name.up.sql / .down.sql      # CONTRACT
├── examples/
│   ├── backfill_batched.sql             # reusable keyset-batched backfill (PG + MySQL)
│   └── postgres_expand_contract/        # the production-grade PostgreSQL version

*... continues with setup instructions, usage examples, and more.*

Code Sample .py preview

runner/migrate.py #!/usr/bin/env python3 """migrate.py -- a small, dependency-free SQL migration runner. Database Migration Toolkit ========================== A versioned migration runner you can actually read and trust. It applies ordered ``V###__name.up.sql`` files, records what it applied (with a checksum so a changed migration is detected), reverts with the matching ``.down.sql``, and reports status. It uses ONLY the Python standard library. Out of the box it drives **SQLite** (``sqlite3`` is in the stdlib) so the bundled ``migrations/`` run with zero setup -- that is how the examples and the README's "how to run" work. The connection layer is a thin abstraction: point it at PostgreSQL or MySQL by installing that driver (``psycopg2`` / ``mysql-connector``) and passing the matching URL; the runner uses the DB-API the same way for all of them. The bundled migrations are written in SQLite-portable SQL; see ``examples/postgres_expand_contract/`` for engine-specific production DDL. Commands -------- python3 runner/migrate.py status python3 runner/migrate.py up # apply all pending python3 runner/migrate.py up --to V003 # apply up to and including V003 python3 runner/migrate.py down # revert the most recent migration python3 runner/migrate.py down --steps 2 # revert the last two python3 runner/migrate.py validate # checksum drift + missing files python3 runner/migrate.py new add_widgets # scaffold a new up/down pair Options (all commands) --url database URL. Default: sqlite:///migrations_demo.db sqlite:///relative/or/abs.db | sqlite://:memory: postgresql://user:pass@host:5432/dbname (needs psycopg2) mysql://user:pass@host:3306/dbname (needs mysql-connector) --dir migrations directory. Default: ../migrations relative to this file. NOTE: a migration runner runs DDL against your database. Always run it against a scratch database first, and take a backup before running it anywhere you care about. See docs/rollback-playbook.md. """ from __future__ import annotations import argparse import dataclasses import hashlib import os import re import sys import time from pathlib import Path from typing import Any, Iterable # ... 431 more lines ...