Airbyte Data Integration

Move data with Airbyte ELT. Configures sources and destinations, builds and tests custom connectors with the CDK, sets up incremental and CDC replication, schedules syncs, and wires failure alerting across 300+ APIs and databases.

This skill sets up Airbyte ELT pipelines: configuring sources and destinations, building and testing custom connectors with the low-code and Python CDK, choosing full-refresh vs incremental vs CDC replication, scheduling and monitoring syncs, and integrating failure alerting with Slack or PagerDuty. Covers both Airbyte Cloud and self-hosted OSS.

When to use

Use when configuring Airbyte sources or destinations, building a custom connector, setting up incremental or CDC syncs, scheduling replication, or debugging failed Airbyte syncs.

Examples

Set up a database sync

Postgres to a warehouse

Configure an Airbyte connection to replicate a Postgres database into Snowflake with incremental CDC and hourly scheduling

Build a custom source

Connector for a private API

Build a custom Airbyte source connector for our internal REST API using the low-code CDK, with pagination and incremental cursor support

Alert on sync failures

Wire up notifications

Set up Airbyte failure notifications to Slack and add retry and dedupe settings for a flaky API source