Data quality KPIs from Elementary

Raw test results tell you what passed and what failed. KPIs translate that into metrics that mean something over time — numbers you can track week-over-week, report to stakeholders, and use to make decisions about where to invest in data quality work.

Elementary stores everything you need in elementary_test_results, dbt_run_results, and dbt_models. These five KPIs cover the most important dimensions of data quality health.

Test pass rate

The simplest and most important metric: what percentage of tests are passing?

pass_rate = passed_tests / total_tests * 100

Track this daily. A declining trend signals growing problems even if nothing has triggered a critical alert yet. A 100% pass rate that’s been stable for weeks might mean your tests are strict enough to trust — or it might mean your tests aren’t catching anything real. Context matters.

The metric is most useful as a trend rather than an absolute number. A project with 80% pass rate trending upward is healthier than one at 95% trending down. A sudden drop from 98% to 85% deserves immediate attention regardless of the absolute level.

Data freshness SLA compliance

What percentage of your data sources meet their freshness requirements?

This requires defining SLAs first. Tag tables with their expected update frequency, then measure compliance against those expectations:

SELECT
  COUNT(CASE WHEN status = 'pass' THEN 1 END) * 100.0 / COUNT(*) AS sla_compliance
FROM elementary_test_results
WHERE test_type = 'freshness_anomalies'
  AND detected_at >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY);

Without explicit SLA definitions, freshness compliance is meaningless — you’re measuring against thresholds that might have been set arbitrarily rather than based on actual stakeholder commitments. The metric is only as good as the underlying agreements.

Model success rate

What percentage of model runs complete successfully?

success_rate = successful_runs / total_runs * 100

Track this separately from test pass rate. A model can run successfully and fail tests (the SQL executed correctly but produced data that violates your quality rules). A model can also fail to run at all (a dependency failed, a schema changed). These are different failure modes that require different responses.

Model success rate that drops below 100% signals execution problems — broken dependencies, timeout issues, permission errors. Test pass rate that drops while model success rate stays high signals data quality problems in the transformation logic or upstream sources.

Anomaly detection rate

How many anomalies is Elementary catching per day or per week?

This metric is more useful as a trend than as an absolute number. A sudden spike often points to upstream data issues — a source that changed its schema, a pipeline that started sending malformed data, a business event that changed expected patterns. A flat zero over several weeks is also worth investigating: either your data is genuinely clean and stable, or your anomaly sensitivity is too low, or you’re not monitoring the right columns.

Neither extreme is inherently good or bad. The goal is to understand what “normal” looks like for your system and notice when that changes.

Test coverage

What percentage of your models have at least one test?

SELECT
  COUNT(DISTINCT CASE WHEN has_tests THEN model_name END) * 100.0 /
  COUNT(DISTINCT model_name) AS coverage
FROM (
  SELECT
    m.name AS model_name,
    EXISTS (
      SELECT 1 FROM elementary_test_results t
      WHERE t.model_unique_id = m.unique_id
    ) AS has_tests
  FROM dbt_models m
  WHERE m.resource_type = 'model'
);

Coverage tells you where you have blind spots. A model with zero tests is a model where problems go undetected. Low coverage on mart models that feed dashboards is more concerning than low coverage on intermediate models used only for transformation.

Coverage as a KPI creates the right incentive: teams that track it tend to add tests to newly built models rather than treating testing as something to add later. “Later” rarely comes.

Mapping to standard data quality dimensions

These KPIs connect to the standard data quality dimensions that appear in governance frameworks and stakeholder conversations:

When reporting to stakeholders who use this vocabulary, framing pass rate and coverage in terms of these dimensions makes the metrics more legible. Instead of “our test pass rate is 94%,” you can say “our completeness and uniqueness checks are passing at 98%, and our timeliness compliance is at 87% against SLA.” That’s a more useful conversation.