dbt Features Without a Dataform Equivalent

Most dbt features map to Dataform equivalents: {{ ref() }} becomes ${ref()}, Jinja macros become JavaScript includes, YAML schema tests become inline assertions. Some dbt capabilities have no Dataform equivalent — not a degraded version, not a manual workaround, but nothing. These gaps are the primary blockers in dbt-to-Dataform migrations.

Snapshots (SCD Type 2)

dbt’s snapshot command implements slowly changing dimension Type 2 automatically. You define which columns to track, choose a timestamp or check strategy, and dbt handles the rest — writing dbt_valid_from, dbt_valid_to, and dbt_scd_id columns, merging new records against the existing snapshot table, and maintaining full history.

-- dbt snapshot definition
{% snapshot customer_snapshot %}
{{ config(
    target_schema='snapshots',
    unique_key='customer_id',
    strategy='timestamp',
    updated_at='updated_at'
) }}
SELECT * FROM {{ source('app_db', 'customers') }}
{% endsnapshot %}

Dataform has no equivalent. Zero. If you need to track how records change over time, you implement it manually using incremental tables with custom merge logic. You write your own valid_from/valid_to logic, your own history management, your own handling of late-arriving updates that should modify an existing history record rather than appending a new one.

This is not a weekend project. SCD2 implemented correctly requires handling edge cases around late-arriving data, out-of-order updates, and initial load behavior. dbt snapshots handle all of this. Dataform makes you figure it out yourself.

Each snapshot table requires custom re-implementation in Dataform before migration is complete.

The Package Ecosystem

dbt’s package hub has 200+ community packages. Packages cover source-specific transformations (GA4, Shopify, Stripe, Salesforce), cross-cutting utilities (dbt_utils, dbt_date, dbt_audit_helper), data quality testing (dbt_expectations, Elementary), and specialized domains (attribution modeling, sessionization, revenue recognition).

The packages that matter most for migration planning:

dbt_utils — Commonly used functions like surrogate_key, star, pivot, unpivot, and date_spine. If you’re using these, you’ll need JavaScript equivalents in Dataform’s includes/ directory. Some are straightforward; surrogate_key is a well-known conversion. Others require careful porting.

dbt_expectations — 50+ data quality tests ported from the Great Expectations library. Range checks, regex pattern validation, distribution shift detection, cross-table comparisons. Dataform’s built-in assertions cover three cases: uniqueness, nulls, and row conditions. Everything else from dbt_expectations requires custom assertion files written from scratch.

Source-specific packages — If you’re using dbt_ga4, dbt_shopify, fivetran_utils, or similar, the transformation logic those packages contain must be reimplemented. Some of these packages represent months of community effort and handle edge cases you haven’t thought of yet.

Dataform has no centralized package hub. A few community repositories exist, but discovery is fragmented, maintenance is inconsistent, and coverage is a fraction of dbt’s ecosystem.

Every package actively in use becomes a custom implementation to build and maintain in Dataform.

Microbatch Incremental Strategy

dbt’s microbatch strategy, introduced in 2024, processes data in discrete time-bounded batches. Instead of a single incremental run that processes “everything since the last run,” microbatch executes separate queries for each time period — one query per hour, or per day, depending on configuration.

This matters for:

• Late-arriving data — Microbatch can reprocess specific time windows to catch records that arrived after the initial run, without a full refresh.
• Cost control — Processing data in small, predictable batches makes BigQuery costs more predictable than a single large incremental merge.
• Backfill isolation — Reprocessing a historical period doesn’t require touching the entire table.

Dataform has no equivalent strategy. Incremental tables in Dataform support uniqueKey-based merges and updatePartitionFilter, but there is no mechanism for automatic time-bounded batch processing with late-data handling. If your dbt project uses microbatch for these patterns, you’re looking at custom implementation work in Dataform — and the custom version will likely be less robust than dbt’s built-in handling.

This gap mostly affects teams processing high-volume event data or managing pipelines with strict late-data SLAs. If you’re running simple updated_at > last_run incremental logic, it doesn’t apply. But if you’ve adopted microbatch specifically to handle the edge cases it was designed for, losing it matters.

Slim CI

dbt Cloud’s Slim CI feature automatically identifies which models changed in a pull request and builds only those models plus their downstream dependents. It creates PR-specific schemas so the build doesn’t touch production. It reports results back to the PR. The whole thing runs with a few clicks of configuration — no YAML pipelines, no custom scripts, no API calls.

The value isn’t just convenience. Slim CI means every PR gets validated before merge, with a cost proportional to what changed rather than the full project. A change to one base model triggers a build of that model and its downstream chain, not a full project rebuild.

Dataform workflows don’t trigger from git events natively. Replicating Slim CI in Dataform requires:

1. GitHub Actions (or Cloud Build) listening to PR events
2. A script calling the Dataform REST API to create a compilation result
3. Custom logic to identify changed models and their dependents
4. Schema creation and teardown for the PR environment
5. Reporting results back to the PR

As the tooling gap note covers in detail, this is 2-4 weeks of engineering work to build the first time, plus ongoing maintenance. Teams that consider CI non-negotiable — and they should — often end up either building this infrastructure or skipping CI entirely. Neither outcome is as good as dbt Cloud’s out-of-the-box behavior.

For teams running dbt Core rather than dbt Cloud, this gap is less pronounced: dbt Core also requires manual CI construction. But dbt build --select state:modified+ gives you the model selection logic; Dataform doesn’t have a comparable concept.

Applicability by project

Each gap applies conditionally:

• Projects without snapshots: SCD2 gap is irrelevant
• Minimal package usage beyond dbt_utils basics: manageable to port
• Standard updated_at > last_run incremental logic: microbatch gap does not apply
• Teams willing to invest in custom CI infrastructure: Slim CI can be replicated (2–4 weeks of engineering work)

The decision framework addresses how many gaps apply simultaneously — one gap is a conversion task; three or four is a migration risk.