Integration And ETL ELT — hobby stack.
For each requirement below, pick the option that fits your build — recommended first, then free and cheaper alternatives — or skip what your project doesn't need. Tap the info icon next to any requirement to see why it matters.
ETL/ELT Platform
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirbyte OSS Free unlimited connectors self-hosted; Cloud Free: 14-day trial + 1M credits/mo dev free. $2.50/credit after (1 credit=1k rows synced). Fivetran $0.10/credit, 500k MAR/mo free for 14d. Hevo $200/mo for 15M events. Overage $0.0002/record (~$0.20/1k)
Connector Marketplace / Pre-built Connectors
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirbyte OSS 600+ free marketplace access self-host; Cloud marketplace included in credit. Singer OSS free 200+ taps. Fivetran connector pack $120/mo per connector MAR based, MAR overage $0.10/credit (~1.2M rows=$120). Estuary Free 100GB/mo ingress, Pro $0.15/GB after.
Transformation Engine (T Layer)
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsdbt Core free unlimited models self-hosted. dbt Cloud Free: 1 dev seat + 1 prod job, Team $100/mo/seat includes 1k model runs. Overage $0.20/model run. Dataform GCP free within BigQuery quotas. Coalesce $2k/mo starter 5 seats.
Data Pipeline Queue / Message Bus
Runs slow work (emails, exports, payouts) in the background so users don’t wait. Keeps your app snappy.Pricing & free-tier limitsKafka OSS free self-hosted. Confluent Cloud Free $0/mo, $400 trial credit + 10GB ingress free. Standard $0.11/GB ingress + $0.21/hr per CKU. AWS SQS Free 1M req/mo, $0.40/1M after. MSK $0.21/hr per broker m5.large + $0.10/GB storage.
Batch Processing Engine
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsSpark OSS free. Glue Free 1M DPU-seconds trial month, then $0.44/DPU-hr (1 DPU=4vCPU 16GB). Databricks $0.40/DBU + infra $0.10/DBU. Dataflow Batch $0.056/vCPU-hr, free 25 hr/mo. EMR Serverless $0.27/hr m5.xlarge + EC2.
Object Storage for Staging / Landing Zone
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsMinIO free self-host unlimited. S3 Standard Free 5GB for 12mo, after $0.023/GB/mo, PUT $0.005/1k, GET $0.0004/1k. B2 $0.006/GB ($6/TB/mo), egress $0.01/GB first 10GB/day free. R2 Free 10GB/mo storage, $0.015/GB after. Overage retrieval $0.01/GB.
Orchestration for Integration Jobs
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirflow OSS free unlimited DAGs. Astro Free dev deployment 14-day trial $0, Team $30/day (~$900/mo) includes 5 workers + 20 task runs/day. MWAA $0.50/hr env + $0.10/worker-hr. Prefect Cloud Free 20k task runs/mo, $0.002/task after, Team $200/mo.
Secrets Management for Connector Creds
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsVault OSS free self-host. HCP Vault Free 25 secrets, after $0.50/secret/mo. AWS Secrets Manager Free 30d trial, $0.40/secret/mo + 10k API calls $0.05. Doppler Free 100 secrets, Starter $18/user/mo. Infisical Free 500 secrets, $5/user/mo Pro after.
Monitoring for Integration / Data Observability
Tells you when things break and why. You cannot fix what you cannot see — this is how you keep downtime short.Pricing & free-tier limitsGrafana OSS free. Datadog Free 1-day retention up to 5 hosts. Pro $15/host/mo + logs $0.10/GB ingested first 1GB/day free. Monte Carlo Starter $20k/yr 100 tables, $150/table extra. Overage $0.30/1k pipeline events after 100k/mo free.
Audit Logs (WORM Immutable)
Controls who can do what inside your admin tooling. Limits blast radius if an account is compromised.Pricing & free-tier limitsS3 Object Lock free feature - storage $0.023/GB. CloudTrail Lake Free 100k events/mo 3mo promo, $2.50/GB ingested after. CloudWatch Logs Free 5GB/mo, $0.50/GB after retention 7yr WORM $0.01/GB/mo. Splunk Cloud $1.80/GB/day ingest min ~$108/mo.
Backup for Pipeline State / Checkpointing
Copies of your data so a bug, hack, or outage doesn’t become permanent loss. Cheap insurance every serious project needs.Pricing & free-tier limitsVelero free OSS. AWS Backup Free 50GB warm first month, $0.05/GB/mo warm, $0.01/GB cold. S3 Versioning storage at $0.023/GB. Restic free + B2 $6/TB/mo. Veeam $0.50/workload/day min $250/mo. Cross-region transfer $0.02/GB.
How this hobby integration and etl elt checklist works.
Each requirement below is something a hobby integration and etl elt build typically needs. Pick one of the four researched options — recommended, free, cheaper or paid — add your own with "Other", or skip the requirement if your project doesn't need it. Nothing is mandatory; the plan on the right tracks what you've decided so nothing gets forgotten.
Your picks are saved in this browser automatically, so you can come back anytime. Options are researched per build level and refreshed as vendors change their plans — always verify details on the provider's page before committing.