dlt RESTClient vs REST API Source

dlt offers two approaches for building custom API pipelines. Both turn API data into warehouse tables but represent different tradeoffs between control and development speed.

The Two Approaches

RESTClient is the lower-level, imperative option. You write Python code that explicitly instantiates a client, handles requests, processes responses, and yields data. Every behavior is explicit: you configure the paginator, set the auth strategy, and define how data flows through. The client wraps Python’s requests library with pagination and auth handling built in.

REST API Source is the declarative option. You define a Python dictionary describing the API’s structure — endpoints, authentication, pagination, write dispositions — and dlt generates the pipeline from that description. Less code, faster development, but the framework makes more decisions for you.

The Decision Rule

Start with REST API Source when the API follows common patterns: JSON responses, standard pagination (offset, cursor, link header), and authentication that fits one of the built-in types (bearer token, API key, OAuth2 client credentials). For a step-by-step walkthrough, see dlt for AI-Assisted Pipeline Development — REST API Source is particularly well-suited to AI-assisted development because its declarative structure is predictable for LLMs to generate.

Switch to RESTClient when you need:

• Custom pagination logic — the API uses a proprietary pagination scheme that doesn’t fit any built-in paginator
• Complex authentication flows — refresh token rotation, multi-step auth, or anything beyond standard OAuth2
• Fine-grained control over request/response handling — custom error handling, response transformation before yielding, or request-level logic that config can’t express
• Gradual migration — wrapping an existing requests-based script into a dlt pipeline without a full rewrite

The choice isn’t always obvious upfront. Starting with REST API Source and hitting its limits is a completely reasonable path. The declarative config is easy to discard if you end up needing RESTClient’s flexibility.

What They Share

Both approaches produce dlt resources that participate in the same pipeline machinery. Once you have a @dlt.resource — whether built with RESTClient’s paginate() or REST API Source’s config — you get the same benefits: automatic schema inference, schema evolution handling, write disposition control, and BigQuery (or any other destination) support.

# Both end up here — same pipeline, same destination
pipeline = dlt.pipeline(destination="bigquery", dataset_name="api_data")
pipeline.run(source)

The per-resource configuration — write_disposition, primary_key — is available in both approaches. So is incremental loading via cursor tracking. The difference is in how you express the API extraction logic, not in what dlt does with the data afterwards.

REST API Source Unnesting

One meaningful functional difference: REST API Source automatically unnests nested JSON data into relational child tables, with referential keys back to the parent row. With RESTClient, nested structures come through as raw JSON unless you handle flattening yourself. For APIs with deeply nested responses — common in marketing platforms and CRM APIs — this automatic unnesting is a real productivity advantage for REST API Source.

For most new integrations, REST API Source is the right starting point. RESTClient is there for the cases where you genuinely need more control.