← Back to all products

Multi-Cloud Lakehouse Blueprint

$69

Architecture blueprint and Terraform templates for Databricks on Azure + AWS with unified governance, cost management, and Delta Sharing.

17 filesv1.0.0
PythonTerraformYAMLMarkdownJSONAWSAzureDatabricksDelta LakeRedis

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free Sample

File Structure 17 files

multi-cloud-lakehouse-blueprint/ ├── README.md ├── cicd/ │ └── multi-cloud-pipeline.yml ├── docs/ │ ├── architecture_decision_records.md │ ├── compliance_matrix.md │ ├── identity_federation.md │ ├── migration_guide.md │ └── network_architecture.md ├── dr/ │ └── disaster_recovery_patterns.md ├── terraform/ │ ├── aws/ │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ ├── azure/ │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ └── shared/ │ └── unity-catalog-multicloud.tf └── tools/ └── cost_comparison_model.py

Documentation Preview README excerpt

Multi-Cloud Lakehouse Blueprint

Product by [Datanest Digital](https://datanest.dev)

Overview

The Multi-Cloud Lakehouse Blueprint is a production-ready architecture package for

organizations building or extending a Databricks Lakehouse across both Microsoft Azure

and Amazon Web Services (AWS). It provides battle-tested Terraform modules, CI/CD

pipelines, governance configurations, and operational runbooks that eliminate months

of cross-cloud integration work.

Who Is This For?

  • Platform engineers tasked with deploying Databricks workspaces across Azure and AWS
  • Data architects designing multi-cloud data mesh or lakehouse topologies
  • Cloud architects evaluating or implementing cross-cloud disaster recovery
  • Engineering managers who need cost models and architecture decision records to

justify multi-cloud investments to leadership

What's Included

Documentation

DocumentDescription
docs/architecture_decision_records.md12 ADRs covering when and why to adopt multi-cloud lakehouse
docs/network_architecture.mdCross-cloud connectivity patterns — VPN, peering, Private Link
docs/identity_federation.mdAzure AD + AWS IAM federation with SCIM provisioning
docs/compliance_matrix.mdGDPR, CCPA, data residency requirements across jurisdictions
docs/migration_guide.mdStep-by-step guide to extend Azure-only deployments to multi-cloud

Terraform Modules

ModuleDescription
terraform/azure/Databricks workspace on Azure with VNet injection and Private Link
terraform/aws/Databricks workspace on AWS with VPC, NAT, and PrivateLink
terraform/shared/unity-catalog-multicloud.tfUnity Catalog with cross-cloud Delta Sharing

CI/CD

PipelineDescription
cicd/multi-cloud-pipeline.ymlUnified Azure DevOps pipeline deploying to both clouds

Tools

ToolDescription
tools/cost_comparison_model.pyInteractive Azure vs AWS DBU pricing comparison calculator

Disaster Recovery

RunbookDescription
dr/disaster_recovery_patterns.mdActive-passive cross-cloud DR patterns and failover procedures

Architecture Overview

... continues with setup instructions, usage examples, and more.

Code Sample .py preview

tests/test_cost_comparison_model.py import sys from pathlib import Path import pytest sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "tools")) import cost_comparison_model as cm # noqa: E402 from cost_comparison_model import WorkloadType as WT # noqa: E402 def W(t=WT.JOBS_COMPUTE, h=1000): return cm.WorkloadProfile("w", t, h) def test_dbu_cost_sums_hours_times_rate(): assert cm.calculate_dbu_cost([W(h=1000), W(WT.SQL_PRO, 100)], {WT.JOBS_COMPUTE: 0.15, WT.SQL_PRO: 0.5}) == pytest.approx(200.0) def test_dbu_cost_missing_rate_is_zero_and_empty(): assert cm.calculate_dbu_cost([W()], {}) == 0.0 assert cm.calculate_dbu_cost([], cm.AZURE_DEFAULT_RATES) == 0.0 def test_compute_cost(): inst = [cm.ComputeInstance("a", 8, 32, 0.5, 2), cm.ComputeInstance("b", 8, 32, 1.5, 2)] assert cm.calculate_compute_cost(inst, 100) == pytest.approx(200.0) assert cm.calculate_compute_cost([], 100) == 0.0 assert cm.calculate_compute_cost(inst, 0) == 0.0 def test_storage_cost_per_cloud(): s = cm.StorageCost(10, azure_rate_per_tb=20, aws_rate_per_tb=25) assert cm.calculate_storage_cost(s, "azure") == 200 and cm.calculate_storage_cost(s, "aws") == 250 assert cm.calculate_storage_cost(s, "gcp") == 250 # anything non-azure falls back to the AWS rate def test_network_cost_splits_vpn_and_prices_egress(): n = cm.NetworkCost(vpn_monthly=300, cross_cloud_egress_gb=100, azure_egress_per_gb=0.1, aws_egress_per_gb=0.2) assert cm.calculate_network_cost(n, "azure") == pytest.approx(160) assert cm.calculate_network_cost(n, "aws") == pytest.approx(170) def test_cost_result_total(): assert cm.CostResult("x", 1, 2, 3, 4).total == 10 def test_compare_clouds_defaults_and_overrides(): wl = cm.build_sample_workloads() az, aw = cm.build_sample_instances() st, nw = cm.StorageCost(10), cm.NetworkCost() # ... 33 more lines ...