Soda Data Quality

Catch bad data before it hits dashboards with Soda. Writes SodaCL checks for freshness, completeness, uniqueness, validity, and custom business rules, profiles datasets, and wires quality gates into pipelines and CI to stop bad data early.

This skill applies Soda for data quality testing, monitoring, and profiling. It authors SodaCL checks in YAML that validate freshness, completeness, uniqueness, schema, validity, and custom SQL business rules, profiles datasets to find anomalies, and embeds quality gates in orchestrators and CI so bad data is caught before it reaches dashboards and ML models.

When to use

Use when writing Soda data quality checks, adding freshness or completeness monitoring, profiling a dataset, or embedding data quality gates in a pipeline or CI run.

Examples

Write freshness checks

Guard a critical table

Write SodaCL checks for our orders table that assert freshness under 1 hour, no nulls in customer_id, and unique order_id

Add a pipeline quality gate

Fail on bad data

Add a Soda scan step to my Airflow pipeline that fails the run if row count drops more than 20% day over day

Custom business rule

SQL-based validity check

Write a Soda custom SQL check that flags orders where total is negative or the discount exceeds the subtotal