Dagster Branch Deployments for dbt

Dagster+‘s branch deployments create an ephemeral preview environment for each pull request. For dbt teams, this allows inspection of which assets changed, the new dependency graph, and test materializations before merging — without affecting production.

Branch deployments cover the entire asset graph, not just dbt models. If a PR changes a Python asset upstream of a dbt model, the full impact is visible. If a sensor that triggers dbt runs is modified, the branch deployment reflects the new trigger behavior. This is broader in scope than dbt Cloud’s CI jobs, which cover only dbt models.

What Branch Deployments Show

When you open a PR that modifies dbt models, the branch deployment:

1. Creates a separate Dagster instance with your branch’s code, including the updated manifest.json from your modified dbt project.
2. Displays the asset graph diff. New assets appear highlighted. Modified assets show what changed. Removed assets are flagged.
3. Allows test materializations. You can materialize modified assets in the preview environment to verify they build correctly, produce expected data, and pass their asset checks.
4. Shows lineage impact. If you renamed a model, added a new dependency, or changed a model’s upstream references, the graph diff makes this visible alongside the code diff.

For teams practicing thorough dbt testing, branch deployments add a visual review layer. Reviewers can inspect the asset graph diff alongside the code diff in the PR. This catches a category of issues that code review alone misses: a model that builds and passes tests but introduces an unexpected dependency path or breaks the intended lineage.

State-Based Selection

For CI/CD workflows, Dagster supports state-based selection that replicates dbt Cloud’s state:modified behavior. This means running only changed models and their downstream dependencies in CI, keeping builds fast even in large projects.

The pattern works by comparing the branch manifest against the production manifest. Models that differ — new models, modified SQL, changed configurations — are selected for execution. Their downstream dependents are included to verify that changes don’t break anything downstream.

For a BigQuery environment, this is particularly valuable for cost control. A dbt project with 200 models where a PR changes 3 would run only those 3 plus their downstream dependencies, not all 200. On BigQuery’s per-byte pricing, this can reduce CI build costs by 90%+ compared to full project builds.

Partitioned Execution

Dagster’s partition system works with dbt incremental models. You can define daily partitions and materialize specific date ranges, which maps to passing --vars to dbt with the partition date.

from dagster import DailyPartitionsDefinition

@dbt_assets(
    manifest=my_project.manifest_path,
    partitions_def=DailyPartitionsDefinition(start_date="2025-01-01"),
)
def my_dbt_assets(context, dbt: DbtCliResource):
    time_window = context.partition_time_window
    dbt_vars = {
        "min_date": str(time_window.start),
        "max_date": str(time_window.end),
    }
    yield from dbt.cli(
        ["build", "--vars", str(dbt_vars)],
        context=context,
    ).stream()

This gives you:

• Targeted backfills. In the Dagster UI, select a date range and materialize just those partitions. No need to write custom backfill scripts or manually pass date parameters.
• Partition-level monitoring. Each partition has its own materialization status. You can see that January 15 succeeded but January 16 failed, and re-run just the failed partition.
• Incremental alignment. The partition dates map directly to the WHERE clause in your incremental model’s is_incremental() block. Dagster passes the dates; dbt filters on them.

For teams using the insert_overwrite strategy on BigQuery or the microbatch strategy (dbt 1.9+), Dagster partitions align naturally with dbt’s own partition handling. The orchestrator and the transformation tool agree on what “process data for January 16” means.

Selective Materializations

Beyond CI/CD and partitioned execution, the Dagster UI supports ad-hoc selective materialization. You can:

• Select a single asset and materialize it with or without its upstream dependencies.
• Select an asset and all its downstreams — the “I fixed this model, now rebuild everything that depends on it” pattern.
• Use dbt selection syntax in the run configuration to target specific tags, folders, or model names.

This is useful for operational scenarios that dbt Cloud handles but simpler orchestrators (Cloud Run Jobs, cron-based schedulers) don’t:

• A mart model produced incorrect data. Fix the SQL, materialize that model and its downstream marts, leave everything else untouched.
• A source table was backfilled with corrected historical data. Materialize the base model and cascade the refresh through intermediate and mart layers.
• A new model was added mid-sprint. Materialize just the new model and its tests to verify it works in the production environment before the next scheduled run.

The CI/CD Flow

A typical CI/CD workflow for dbt in Dagster combines branch deployments with state-based selection:

1. Developer opens a PR with modified dbt models.
2. CI pipeline builds the manifest from the branch (dbt deps && dagster-dbt project prepare-and-package).
3. Branch deployment creates a preview environment with the new manifest.
4. State-based selection identifies modified models and their downstreams.
5. Test materializations run in the preview environment with asset checks.
6. Reviewer inspects both the code diff and the asset graph diff.
7. Merge deploys the updated manifest to production.

This workflow provides more confidence than dbt Cloud CI for teams with mixed Python/dbt pipelines, because the branch deployment tests the entire asset graph, not just the dbt portion. For pure-dbt teams, the experience is comparable to dbt Cloud CI but with the added benefit of visual graph diffing.

When Branch Deployments Apply

Branch deployments and state-based selection are features of Dagster+ (the managed cloud offering). Self-hosted Dagster OSS does not include them. For teams evaluating Dagster vs dbt Cloud, the CI/CD capabilities are comparable: Dagster covers cross-system scope; dbt Cloud requires no additional setup for pure-dbt projects.

The value scales with project complexity. A 20-model dbt project with no Python assets gets limited benefit — dbt build in CI achieves the same validation. A 200-model project with upstream Python ingestion and downstream ML features benefits from full graph diffing on every PR.