Integration And ETL ELT — small scale stack.
For each requirement below, pick the option that fits your build — recommended first, then free and cheaper alternatives — or skip what your project doesn't need. Tap the info icon next to any requirement to see why it matters.
ETL/ELT Platform
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirbyte OSS Free unlimited connectors self-hosted; Cloud Free: 14-day trial + 1M credits/mo dev free. $2.50/credit after (1 credit=1k rows synced). Fivetran $0.10/credit, 500k MAR/mo free for 14d. Hevo $200/mo for 15M events. Overage $0.0002/record (~$0.20/1k)
Connector Marketplace / Pre-built Connectors
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirbyte OSS 600+ free marketplace access self-host; Cloud marketplace included in credit. Singer OSS free 200+ taps. Fivetran connector pack $120/mo per connector MAR based, MAR overage $0.10/credit (~1.2M rows=$120). Estuary Free 100GB/mo ingress, Pro $0.15/GB after.
Transformation Engine (T Layer)
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsdbt Core free unlimited models self-hosted. dbt Cloud Free: 1 dev seat + 1 prod job, Team $100/mo/seat includes 1k model runs. Overage $0.20/model run. Dataform GCP free within BigQuery quotas. Coalesce $2k/mo starter 5 seats.
Data Pipeline Queue / Message Bus
Runs slow work (emails, exports, payouts) in the background so users don’t wait. Keeps your app snappy.Pricing & free-tier limitsKafka OSS free self-hosted. Confluent Cloud Free $0/mo, $400 trial credit + 10GB ingress free. Standard $0.11/GB ingress + $0.21/hr per CKU. AWS SQS Free 1M req/mo, $0.40/1M after. MSK $0.21/hr per broker m5.large + $0.10/GB storage.
Batch Processing Engine
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsSpark OSS free. Glue Free 1M DPU-seconds trial month, then $0.44/DPU-hr (1 DPU=4vCPU 16GB). Databricks $0.40/DBU + infra $0.10/DBU. Dataflow Batch $0.056/vCPU-hr, free 25 hr/mo. EMR Serverless $0.27/hr m5.xlarge + EC2.
Object Storage for Staging / Landing Zone
Where you keep files users upload or you serve — images, videos, documents — and how fast they reach visitors around the world.Pricing & free-tier limitsMinIO free self-host unlimited. S3 Standard Free 5GB for 12mo, after $0.023/GB/mo, PUT $0.005/1k, GET $0.0004/1k. B2 $0.006/GB ($6/TB/mo), egress $0.01/GB first 10GB/day free. R2 Free 10GB/mo storage, $0.015/GB after. Overage retrieval $0.01/GB.
Orchestration for Integration Jobs
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsAirflow OSS free unlimited DAGs. Astro Free dev deployment 14-day trial $0, Team $30/day (~$900/mo) includes 5 workers + 20 task runs/day. MWAA $0.50/hr env + $0.10/worker-hr. Prefect Cloud Free 20k task runs/mo, $0.002/task after, Team $200/mo.
Secrets Management for Connector Creds
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsVault OSS free self-host. HCP Vault Free 25 secrets, after $0.50/secret/mo. AWS Secrets Manager Free 30d trial, $0.40/secret/mo + 10k API calls $0.05. Doppler Free 100 secrets, Starter $18/user/mo. Infisical Free 500 secrets, $5/user/mo Pro after.
Monitoring for Integration / Data Observability
Tells you when things break and why. You cannot fix what you cannot see — this is how you keep downtime short.Pricing & free-tier limitsGrafana OSS free. Datadog Free 1-day retention up to 5 hosts. Pro $15/host/mo + logs $0.10/GB ingested first 1GB/day free. Monte Carlo Starter $20k/yr 100 tables, $150/table extra. Overage $0.30/1k pipeline events after 100k/mo free.
Audit Logs (WORM Immutable)
Controls who can do what inside your admin tooling. Limits blast radius if an account is compromised.Pricing & free-tier limitsS3 Object Lock free feature - storage $0.023/GB. CloudTrail Lake Free 100k events/mo 3mo promo, $2.50/GB ingested after. CloudWatch Logs Free 5GB/mo, $0.50/GB after retention 7yr WORM $0.01/GB/mo. Splunk Cloud $1.80/GB/day ingest min ~$108/mo.
Backup for Pipeline State / Checkpointing
Copies of your data so a bug, hack, or outage doesn’t become permanent loss. Cheap insurance every serious project needs.Pricing & free-tier limitsVelero free OSS. AWS Backup Free 50GB warm first month, $0.05/GB/mo warm, $0.01/GB cold. S3 Versioning storage at $0.023/GB. Restic free + B2 $6/TB/mo. Veeam $0.50/workload/day min $250/mo. Cross-region transfer $0.02/GB.
CDC Tool (Change Data Capture)
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsDebezium OSS free unlimited connectors self-hosted. Confluent Debezium Premium Connector $0.22/connector-hr. Striim Free dev 1 connector, Prod $3,500/mo per server, $0.001 per change event after 10M included. Fivetran HVR $1,200/mo per source.
Stream Processing Engine
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsFlink OSS free self-host. Confluent Flink Free up to 10 CU trial, $0.11/CU-hr Standard. Ververica Cloud Free $300 trial credit, Standard $0.60/CU-hr (1 CU=1vCPU 4GB). Kinesis Analytics $0.11/KPU/hr. State storage $0.10/GB/mo.
Schema Registry & Evolution Management
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsApicurio OSS free unlimited schemas. Confluent Cloud Free 100 schemas. Essentials $1/schema/mo after. Glue Schema Registry Free 1M validations/mo, $1/1M after no per-schema charge. Enterprise $2.50/schema/mo with evolution rules + compatibility.
Data Quality & Validation Framework
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsGE OSS free unlimited tests. Soda Cloud Free 10k checks/mo, Team $300/mo 50k checks $0.006/check. Metaplane Free 1k tests/mo. Monte Carlo $30k/yr starter 100 tables $150/table extra, $0.05/check overage.
Data Warehouse / Lakehouse Destination
A piece of your stack you may or may not need, depending on scope. Pick the option that fits — or skip it if your project doesn’t require this capability yet.Pricing & free-tier limitsDuckDB free OSS; MotherDuck Free 10GB storage $0. BigQuery Free 10GB storage + 1TB queries/mo, $0.02/GB storage after $6.25/TB query. Snowflake Standard $2/credit compute storage $23/TB/mo free 30d $400 credits. ClickHouse Cloud Free 1GB storage.
Rate Limiting & Throttling for Source APIs
Controls and protects your APIs — quotas, abuse prevention, and firewalls. Important once you have real traffic or many clients.Pricing & free-tier limitsNGINX OSS free. AWS API Gateway Free 1M calls/mo 12mo, REST $3.50/1M, HTTP $1.00/1M. Cloudflare Workers Free 100k req/day, Pro $5/mo includes 10M req $0.50/M after. Kong Konnect Free 10k req/min, Team $250/mo after. Overage throttled requests free.
API Gateway for Data APIs
Controls and protects your APIs — quotas, abuse prevention, and firewalls. Important once you have real traffic or many clients.Pricing & free-tier limitsKong OSS free unlimited routes. AWS API Gateway HTTP $1/M req after free 1M, REST $3.50/M, WebSocket $1/M + $0.25/conn-hr. Apigee X Free trial $300 credit, $0.02/10k calls paygo. Cloudflare API Gateway included with Pro/Enterprise.
Cost Management & Credit Consumption Monitoring
Tells you when things break and why. You cannot fix what you cannot see — this is how you keep downtime short.Pricing & free-tier limitsOpenCost free OSS unlimited. Infracost OSS free. CloudZero Free trial 30d, Starter $350/mo for $20k spend tracked. Datadog CCM free if Datadog customer standalone $0.01/resource-hr. CloudHealth $15k/yr minimum. Overage tracking $1.50/1k credits.
How this small scale integration and etl elt checklist works.
Each requirement below is something a small scale integration and etl elt build typically needs. Pick one of the four researched options — recommended, free, cheaper or paid — add your own with "Other", or skip the requirement if your project doesn't need it. Nothing is mandatory; the plan on the right tracks what you've decided so nothing gets forgotten.
Your picks are saved in this browser automatically, so you can come back anytime. Options are researched per build level and refreshed as vendors change their plans — always verify details on the provider's page before committing.