Warehouse And Lakehouse — enterprise stack.
For each requirement below, pick the option that fits your build — recommended first, then free and cheaper alternatives — or skip what your project doesn't need. Tap the info icon next to any requirement to see why it matters.
Warehouse Compute Primary
Where your code actually runs and serves requests. Picking the right host affects speed, scaling, and how much ops work you do.Pricing & free-tier limitsFREE: Snowflake $400 trial credits (14d), DuckDB unlimited local. PAID: Standard $2.00/credit, Enterprise $3.00/credit; XS=1 cr/hr (~$2/hr), S=2 cr/hr ($4/hr), M=4 cr/hr ($8/hr). Billed per-second min 60s. Overage $2-3/credit. Free: 0 after trial.
Lakehouse Storage Format (Delta / Iceberg / Hudi)
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsFREE: OSS unlimited tables, Apache 2.0 license, local. PAID: Databricks runtime included in DBU $0.40/DBU (SQL Compute). Onehouse $0.35/vCPU/hr + $500/mo min cluster. Storage separate. Free tier: OSS unlimited, managed 14d trial.
Object Storage for Lakehouse (S3 / R2 / GCS)
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsFREE: R2 10GB storage/mo, 10M Class A ops/mo, zero egress; MinIO unlimited self-hosted. PAID: S3 $0.023/GB/mo first 50TB, $0.005/1k PUT, $0.0004/1k GET, egress $0.09/GB. R2 $0.015/GB/mo, no egress. Overage linear.
Data Lake Catalog / Metastore
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Hive Metastore unlimited self-hosted, Glue first 1M requests/mo free + 1M objects free. PAID: Glue $1.00/100k requests after free, Unity Catalog included with Databricks $0.40/DBU, Tabular $0/until $0.05/10k API calls, $349/mo starter. Overage $1/100k req.
Query Engine / Federation Layer
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Trino OSS unlimited, self-hosted, 0 license. PAID: Starburst Galaxy $0 free dev 25$ credit, then $1.20/CPU-hr ($0.30/ECU-hr), Enterprise $3,000/mo min. Dremio $0.80/vCPU/hr. Free tier: 1 cluster paused after 7d inactivity.
ETL / ELT Ingestion into Warehouse
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Airbyte OSS unlimited connectors self-hosted, Cloud free 2M records/mo. PAID: Airbyte Cloud $2.50/credit (~$0.015/M credits), Team $15k credits/mo $1,500. Fivetran $1.00/credit + MAR $120/mo min, free 500k rows/mo. Overage $2.50/credit.
BI Connector / SQL Gateway (JDBC/ODBC)
Persistent storage for your app’s data — users, products, orders. The single most important architectural decision for most projects.Pricing & free-tier limitsFREE: Snowflake/Trino/DuckDB drivers free Apache 2.0/MIT, unlimited connections. PAID: CData $999/yr per driver (standard), Enterprise $2,499/yr per driver, OEM $500/mo. Free tier: drivers free, CData 30d trial.
Caching Layer for Warehouse Queries
Stores frequent results in fast memory so you serve less from the database. Big lever for speed and cost.Pricing & free-tier limitsFREE: Snowflake result cache free 24h for identical queries, OSS cache unlimited self-hosted. PAID: Databricks predictive cache included (no extra DBU), CubeStore self-hosted free, Cloud $0.04/GB-hr cache, $256/mo min. No extra overage for snowflake cache.
Backup & Time Travel
Copies of your data so a bug, hack, or outage doesn’t become permanent loss. Cheap insurance every serious project needs.Pricing & free-tier limitsFREE: Delta/Iceberg time travel free, S3 versioning $0.023/GB/mo ver. PAID: Snowflake Time Travel Standard 1 day included (7d enterprise), Fail-safe 7d extra billed at $23/TB/mo (Standard) $30/TB/mo (Ent). Databricks backup incremental S3 cost only. Free limits: 1 day (std) then recovery via fail-safe.
Secrets Management for Warehouse Credentials
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Vault OSS unlimited self-hosted, Infisical Free 3 envs, AWS Secrets 30d free rotation. PAID: AWS Secrets $0.40/secret/mo + $0.05/10k API calls. HCP Vault $0.03/hr dev, $1.58/hr Enterprise starter $630/mo. Free tier: AWS 30d trial, Infisical 3 projects free.
CI/CD & Schema Migration for Warehouse
Automates testing and deploying your code. Saves enormous time and prevents “works on my machine” releases.Pricing & free-tier limitsFREE: dbt Core unlimited, free IDE; dbt Cloud Dev free 1 seat, 1 project. PAID: dbt Cloud Team $50/user/mo ($100/mo min), Enterprise $250/user/mo. Flyway Teams $3,000/yr. Bytebase Pro $499/mo 10 instances. Overage dbt usage seats $50 ea.
Serverless Warehouse Compute (On-demand Slots)
Where your code actually runs and serves requests. Picking the right host affects speed, scaling, and how much ops work you do.Pricing & free-tier limitsFREE: BigQuery free 10GB storage + 1TB queries/mo free. MotherDuck 10GB free. PAID: BigQuery On-demand $6.25/TB scanned (2026), Flat-rate $2,000/mo for 100 slots (capacity). Snowflake serverless $3/credit. Overage $6.25/TB on-demand, $0.04/slot-hr commitment.
Change Data Capture (CDC) for Warehouse Sync
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Debezium OSS unlimited self-hosted (Kafka req). Estuary free 50GB/mo. PAID: Fivetran HVR $1.20/CDC credit, min $500/mo. Confluent $0.15/GB CDC ingress + connector $0.20/hr. Overage $0.15/GB.
Columnar File Optimization & Compaction (OPTIMIZE/VACUUM)
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsFREE: OSS OPTIMIZE/VACUUM unlimited, self-hosted compute only. PAID: Databricks Auto-Optimize included in DBU, serverless DBU $0.70/DBU for compaction jobs. Glue auto-compaction $0.44/DPU-hr. Overage is DPU compute only.
Monitoring - Query Performance & Warehouse Health
Tells you when things break and why. You cannot fix what you cannot see — this is how you keep downtime short.Pricing & free-tier limitsFREE: Grafana OSS unlimited, Snowflake QUERY_HISTORY free 365d. PAID: Datadog Infra Pro $23/host/mo + DBM $15/host/mo, log $0.10/GB. SigNoz $0.30/GB ingest. Free: Datadog 14d trial 5 hosts, SigNoz 30d. Overage host $23/mo.
Cost Tracking & Credit Usage Attribution
Rates, labels, and tracking for physical goods. Needed once you actually ship products.Pricing & free-tier limitsFREE: OpenCost unlimited self-hosted, Snowflake cost views free, CloudZero free up to $100k spend. PAID: SelectDB $25/compute/hr attribution free included, CloudZero Pro $4k/mo starting at $500k cloud spend, Finout $0.8% spend. Free tier: first $100k analyzed free.
Security - RBAC & Basic Access Control
How users sign up, log in, and are authorized. Getting roles and access control right early prevents painful rewrites.Pricing & free-tier limitsFREE: Ranger OSS unlimited self-hosted, Lake Formation first 50k requests free. PAID: Lake Formation $0.00035/Governed table access req after free, Immuta $65k/yr starter (100 users), Okta $2/user/mo SSO. Overage $0.00035/req.
Workflow Orchestration for Warehouse Pipelines
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Airflow OSS unlimited self-hosted. MWAA small env $0.49/hr ($360/mo) + workers. PAID: Astro Developer $150/mo includes $100 usage, workers $0.35/hr ($0.10/Astro unit). Enterprise $2,500/mo min. Dagster Cloud $0.30/credit ~$450/mo. Overage $0.42/hr worker.
Materialized Views / Result Acceleration
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: DuckDB/Trino MV free OSS. PAID: Snowflake MV compute charged same as warehouse (refresh $2-3/credit) + storage $23/TB/mo; maintenance serverless up to $3/credit. Dremio reflections included $0.80/vCPU/hr. No extra license fee.
Lakehouse Compute / Spark SQL Warehouse
Where your code actually runs and serves requests. Picking the right host affects speed, scaling, and how much ops work you do.Pricing & free-tier limitsFREE: Spark OSS unlimited, Databricks Free Edition 1 workspace, no credits; EMR Studio free. PAID: Databricks SQL Classic $0.40/DBU, Pro $0.55/DBU, Serverless $0.70/DBU (~2-5 DBU/hr per cluster). EMR Serverless $0.052/vCPU-hr + $0.00573/GB-hr. Free: DB 14d $500 trial.
Streaming Ingestion (Kafka/Kinesis -> Warehouse)
Delivers and processes audio/video to users. Bandwidth-heavy, so plan the cost.Pricing & free-tier limitsFREE: Kafka OSS unlimited, Redpanda OSS, Confluent free $400 trial 30d. PAID: Confluent $0.20/GB-month storage, $0.11/GB ingress/egress, MSK Serverless $0.20/vCPU-hr. Snowpipe Streaming serverless $3/credit. Free tier: Confluent up to 10GB/mo on trial.
Data Quality & Observability Platform
Tells you when things break and why. You cannot fix what you cannot see — this is how you keep downtime short.Pricing & free-tier limitsFREE: GE OSS unlimited self-hosted, Soda Free 5K tests/mo. PAID: Monte Carlo Pro $25k/yr starter (50 tables) ~$2,083/mo, Enterprise $60k+/yr. Soda Cloud $1,500/mo team. Great Expectations Cloud $1,000/mo. Overage $0.50/table/mo beyond.
Data Lakehouse Governance (Unity Catalog / Lake Formation)
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Atlas/OpenMetadata OSS unlimited. PAID: Lake Formation governed requests $0.00035/req after 50k free, Unity Catalog $0 included + DBU, Alation Cloud $100k/yr starting. Free Atlan 2 seats + 10M catalog ops/mo.
Audit Logging & Query History Retention
Controls who can do what inside your admin tooling. Limits blast radius if an account is compromised.Pricing & free-tier limitsFREE: Snowflake query history 365d free, CloudTrail 90d free console. PAID: CloudTrail Lake $2.50/GB ingested + $0.50/GB-month stored (90d retention 1GB free), Datadog logs $0.10/GB, Splunk $75/GB/mo. Overage $2.50/GB for lake.
Resource Management - Auto-scaling & Warehouse Sizing
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: K8s HPA unlimited OSS. PAID: Snowflake multi-cluster included in Enterprise (no extra fee, credits scale: 1-10 clusters) e.g., S multi-cluster 2 credits/hr min, up to 20 credits/hr max, billed per sec. ClickHouse auto-scale $0.16/CPU-hr burst over provisioned.
Encryption at Rest & In-Transit for Lakehouse
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: S3 SSE-S3 free, TLS free. PAID: KMS $1.00/key/mo + $0.03/10k requests. Tri-Secret Secure included Ent+ Business Critical (no extra, but Ent costs $3/credit vs $2). HSM KMS $1.12/hr. Overage $0.03/10k KMS req.
Data Lineage Tracking
Rates, labels, and tracking for physical goods. Needed once you actually ship products.Pricing & free-tier limitsFREE: OpenLineage/Marquez OSS unlimited self-hosted. PAID: Atlan Pro $30k/yr starter 5 users ($2,500/mo), Alation $120k/yr ($10k/mo). DataHub Cloud $1/hr ($720/mo). Free: Atlan 2 users unlimited lineage.
Data Discovery & Warehouse Catalog Search
Lets users find things fast. A slow or missing search directly costs engagement and conversions.Pricing & free-tier limitsFREE: DataHub/OpenMetadata OSS unlimited self-hosted. PAID: Atlan Pro $30k/yr starter (covers discovery), Alation $100k+/yr ($8,333/mo). DataHub Cloud $800/mo dev. Collibra $150k/yr. Free Atlan 2 users + 10M assets search/mo.
Workload Management & Query Queuing
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Trino Resource Groups unlimited OSS. PAID: Snowflake Query Acceleration Service $3/credit serverless for eligible queries, included with Ent; Workload Mgmt no extra fee. Databricks workload mgmt included in DBU. Free tier included.
Metadata Statistics & Auto ANALYZE / Auto Optimize
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Trino/Delta ANALYZE OSS compute-only cost. PAID: Snowflake Auto Clustering serverless $3/credit (credits auto consumed), avg $200-600/mo per large table. Databricks Auto Optimize included free in DBU. Overage serverless credits.
Data Sharing & Secure Data Exchange (Delta Sharing)
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Delta Sharing OSS unlimited shares, Snowflake sharing no extra cost (compute on consumer). PAID: Snowflake Secure Share free, Private Listing $0 + consumer compute. Databricks Sharing $0, Marketplace $0 + DBU for consumer queries. No per-share fee; storage $0.023/GB/mo.
Vacuum & Retention Policy Management
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: S3 Lifecycle free (actions billed as deletes), OSS VACUUM compute only. PAID: Databricks managed vacuum DBU $0.70/DBU for serverless jobs (~$0.10/TB vacuum). S3 delete free, version delete marker cost negligible. Overage compute-only.
Cross-Region Replication & DR Failover
Copies of your data so a bug, hack, or outage doesn’t become permanent loss. Cheap insurance every serious project needs.Pricing & free-tier limitsFREE: rclone OSS unlimited. PAID: S3 CRR $0.023/GB-mo dest + $0.02/1k replication req + transfer $0.02/GB. Snowflake Replication storage $23/TB/mo + 1 credit/TB transferred. Business Critical edition $4.00/credit (Ent+). Free: Snowflake 48h failover trial.
Dynamic Data Masking & Row-Level / Column-Level Security
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Trino/Ranger OSS masking unlimited. PAID: Snowflake Masking/RLS included Ent ($3/credit) no extra fee, Business Critical $4/credit. Immuta Enterprise $65k/yr start, $5,400/mo. No per-query overage for native Snowflake masking.
Semantic Layer / Metrics Store
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: Cube OSS unlimited self-hosted, dbt Semantic Layer Dev free 1 project. PAID: Cube Cloud Team $810/mo includes 2 prod deployments, Enterprise $4,500/mo. dbt Semantic Layer included Team $50/user/mo+. AtScale $40k/yr starter. Overage Cube $0.25/query after bundle.
Lakehouse File System Caching (Alluxio / Native Cache)
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsFREE: Alluxio CE unlimited self-hosted, Disk Cache free on Databricks (NVMe free). PAID: Alluxio Enterprise $8k/mo cluster (or $0.15/GB cache/mo managed). CubeStore $0.04/GB-hr. Dremio cache free tier included in $0.80/vCPU-hr. Free OSS unlimited.
Table Maintenance - Z-Ordering / Clustering & Liquid Clustering
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFREE: ZORDER OPTIMIZE OSS compute-only. PAID: Snowflake Auto Clustering $3/credit serverless, typical $400-1,200/mo per TB clustered. Databricks Liquid Clustering $0.70/DBU serverless auto maintenance, ~$0.20/TB/hr. Overage same serverless credit rate.
Secrets Rotation & Compliance Vault (HSM)
Stores API keys and credentials safely, separate from your code. Leaking secrets is one of the most common breaches.Pricing & free-tier limitsFREE: Vault OSS rotation unlimited self-hosted. PAID: AWS Secrets rotation $0.40/secret/mo + lambda compute $0.0000167/GB-sec, CloudHSM $1.45/hr/cluster (~$1,045/mo), HCP Vault Enterprise HSM $1.58/hr ($1,137/mo) starter cluster. Overage $0.40/secret/mo.
How this enterprise warehouse and lakehouse checklist works.
Each requirement below is something a enterprise warehouse and lakehouse build typically needs. Pick one of the four researched options — recommended, free, cheaper or paid — add your own with "Other", or skip the requirement if your project doesn't need it. Nothing is mandatory; the plan on the right tracks what you've decided so nothing gets forgotten.
Your picks are saved in this browser automatically, so you can come back anytime. Options are researched per build level and refreshed as vendors change their plans — always verify details on the provider's page before committing.