Unity Catalog Migration Kit
Step-by-step migration toolkit from Hive Metastore to Unity Catalog. Assessment scripts, automated migration, validation tests, and rollback procedures.
Product Preview
Try the interactive reader and demo tools below, or get the full product with all content unlocked.
Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free SampleFile Structure 14 files
Documentation Preview README excerpt
Unity Catalog Migration Kit
By [Datanest Digital](https://datanest.dev) | Version 1.0.0 | $69
A comprehensive, production-ready toolkit for migrating from Databricks Hive Metastore to Unity Catalog. Includes automated assessment, migration notebooks, Terraform infrastructure-as-code, planning tools, and operational guides.
What's Included
Databricks Notebooks
| Notebook | Purpose |
|---|---|
pre_migration_assessment.py | Scans Hive Metastore, catalogs all databases/tables/views/permissions, generates assessment report |
automated_table_migration.py | Migrates tables from Hive to Unity Catalog using CTAS with full validation |
permission_mapper.py | Maps legacy Hive permissions to Unity Catalog grants |
post_migration_validation.py | Validates migration integrity: row counts, schema comparison, data sampling |
Terraform Module
| File | Purpose |
|---|---|
unity-catalog-setup/main.tf | Full Unity Catalog metastore, catalogs, schemas, external locations |
unity-catalog-setup/variables.tf | Configurable variables for all resources |
unity-catalog-setup/outputs.tf | Outputs for downstream consumption |
CLI Tools
| Tool | Purpose |
|---|---|
migration_planner.py | Generates migration waves from assessment output |
timeline_estimator.py | Estimates migration timeline based on table count and data volume |
Guides
| Guide | Purpose |
|---|---|
migration_guide.md | Step-by-step migration walkthrough |
day2_operations.md | Post-migration Unity Catalog operations |
rollback_procedures.md | Rollback procedures for each migration stage |
Templates
| Template | Purpose |
|---|---|
migration_planning_spreadsheet.csv | CSV template for migration wave planning |
Quick Start
1. Run the Pre-Migration Assessment
Import notebooks/pre_migration_assessment.py into your Databricks workspace and run it. This produces a JSON assessment report cataloging every database, table, view, and permission in your Hive Metastore.
2. Plan Migration Waves
Use the CLI planner to generate migration waves from the assessment:
python tools/migration_planner.py \
--assessment-file assessment_report.json \
--max-tables-per-wave 50 \
--output waves.json
*... continues with setup instructions, usage examples, and more.*