Dataform’s core transformation engine — compiling SQLX to SQL and executing it against BigQuery — is mature and production-ready. The gaps are in the surrounding ecosystem: CI/CD automation, local development tooling, the package ecosystem, and platform portability. These are areas where dbt’s decade of community investment creates a compounding advantage.
CI/CD Automation
dbt Cloud provides Slim CI out of the box. When a developer opens a pull request, dbt Cloud automatically:
- Identifies which models changed (and their downstream dependents)
- Creates a PR-specific schema so builds do not interfere with production
- Builds only the affected models
- Runs associated tests
- Executes SQL linting
- Reports results back to the PR
A few clicks in the dbt Cloud UI and this workflow is live. No YAML pipeline files, no custom scripts, no infrastructure to manage.
Dataform requires all of this to be built manually. Workflows cannot be triggered by git events natively. Implementing comparable automation means:
- Calling the Dataform REST API from external CI tools (GitHub Actions, Cloud Build, or similar)
- Writing custom logic to determine which models changed
- Managing schema creation and teardown for PR environments
- Building the reporting integration back to your git platform
This is not theoretical work. It is 2-4 weeks of engineering effort to build, plus ongoing maintenance as the API evolves and edge cases surface. Many Dataform teams skip CI entirely and review transformation changes through manual code review alone. The absence of CI does not cause immediate failures — it causes slow accumulation of untested changes that surface as production incidents weeks later.
For teams using dbt Core (self-hosted) rather than dbt Cloud, the CI gap is less pronounced. dbt Core also requires custom CI pipeline construction, though the dbt build --select state:modified+ command simplifies the “what changed” detection that Dataform lacks.
IDE and Developer Tooling
The developer experience gap is cumulative across every engineer on the team.
dbt’s Power User extension for VS Code and Cursor has over 1 million installs. It provides:
- Model lineage visualization — see upstream and downstream dependencies without leaving the editor
- Query preview with execution — run models directly from the IDE
- Column-level auto-complete — suggestions based on actual upstream schemas
- AI-powered documentation generation — draft descriptions from model SQL
- BigQuery cost estimation — see expected query cost before running
Nothing comparable exists for Dataform. Development options are:
- The Cloud Console IDE — browser-based, provides real-time compilation feedback and cost estimates, but lacks the extensibility and keyboard-driven workflows of a desktop editor
- A basic text editor — syntax highlighting for SQLX at best, no transformation-aware features
The Cloud Console IDE is not bad. Its real-time compilation feedback is genuinely useful — you see immediately whether your SQLX is valid and what the compiled SQL costs. But it is a single tool with no plugin ecosystem, no community extensions, no path to customization. Cursor with dbt Power User represents what is possible when a large community builds tooling around a platform. Dataform’s smaller user base means that level of tooling investment never materializes.
For teams that have standardized on cloud-based development (Cloud Workstations, Cloud Shell Editor), the IDE gap matters less. For teams accustomed to rich local development environments, it is a daily friction point.
Package Ecosystem
dbt has 200+ packages on hub.getdbt.com covering:
- Source-specific transformations — GA4, Shopify, Stripe, Salesforce, HubSpot, Facebook Ads, Google Ads
- Cross-cutting utilities — dbt-utils, dbt-expectations, dbt-date, dbt-audit-helper
- Observability — Elementary, re_data
- Specialized patterns — attribution modeling, sessionization, revenue recognition
Each package represents hundreds of hours of community development, testing, and maintenance. When a dbt team needs GA4 sessionization, they install a package and configure it in an afternoon. When a Dataform team needs the same capability, they build it from scratch.
Dataform has no centralized package hub. The Devoteam dataform-assertions package is one of the few third-party options. Individual teams occasionally open-source their Dataform utilities, but discovery is fragmented and maintenance is inconsistent.
This is a network effects problem: more dbt users generates more packages, which generates more reasons to choose dbt. Dataform’s smaller user base cannot generate the package volume that would make the ecosystem self-sustaining. There is no indication that Google plans to invest in first-party packages.
Platform Lock-In
dbt connects to 20+ data platforms through its adapter architecture: BigQuery, Snowflake, Databricks, Redshift, Postgres, DuckDB, and many others. Transformation logic written for dbt is portable. The dispatch pattern even handles SQL dialect differences across databases.
Dataform works exclusively with BigQuery. This is not a current limitation that might expand — it is a design choice reflecting Dataform’s identity as a GCP-native service. If your organization adopts Snowflake for a new workload, acquires a company running on Databricks, or needs a Redshift cluster for a specific use case, Dataform cannot follow.
The lock-in risk is probabilistic. If you are certain BigQuery is your only warehouse for the foreseeable future, single-platform support is irrelevant. But “foreseeable future” in data infrastructure rarely extends beyond 3-5 years, and organizational changes (mergers, new product lines, vendor negotiations) often force multi-platform scenarios that nobody planned for.
The Compounding Effect
These gaps interact. Without CI/CD, testing becomes more important — but testing is also limited. Without packages, teams write more custom code — but without rich IDE tooling, writing that code is slower. Without platform portability, switching costs rise — making the decision to stay with Dataform increasingly irreversible over time.
Each gap individually might be acceptable. The combination creates an ecosystem deficit that grows with project complexity. For a 20-model project with basic needs, the gaps barely matter. For a 200-model project with CI requirements, data quality SLAs, and potential multi-cloud futures, they define the daily experience of every engineer on the team.