There is no pipeline object to maintain. A workflow is what the lineage already says: every transform and sync declares what it reads and what it writes, and the platform derives the graph from those declarations. Building a dataset builds what it depends on; changing a source propagates to what depends on it. This page is about driving that.

​
Building a dataset and its dependencies

A build targets one dataset, model or mediaset. The platform walks the graph upstream, orders the producers so that each one runs after those it reads, creates one build per node, and runs the chain. Every node upstream that has a producer is built again, whether its inputs changed or not; target_only builds the target alone. A producer that fails stops the chain and cancels what was still waiting.
The answer is a batch: a group_id that correlates the builds of this trigger and the list of builds, each with its target_id, its status (PENDING, RUNNING, SUCCESS, FAILED, CANCELLED) and the builds it depends_on. Follow them with GET /orchestration/builds?target_id=, read one with GET /orchestration/builds/{id}, its log with .../logs (tail= for the last lines, since= for those after the next of a previous answer), and stop one with POST .../cancel. POST /orchestration/builds/plan takes the same body and answers what the build would run, without creating or running anything: the steps in order, each with its target_id, its transform_id, the outputs it rewrites and the steps it depends_on, and the data branch they would write to. It is refused (403, 400, 409) exactly where the build would be. The library does the same and follows a build to its end; target_only=True builds the one node without its upstream chain:
See Command line for what the commands print. A node fed by a sync alone is built by running the sync; a transform with no input (a Python function that reads a connection, or generates data) is built on its own. A transform’s own execute endpoint (POST /transforms/{id}/execute, POST /syncs/{id}/execute) runs the one producer without its dependencies.

​
Propagating a change downstream

When a source has changed, rebuild what depends on it rather than each dataset by hand:
The platform walks the graph downstream from the root, orders the descendants and runs them in sequence; the root itself is not rebuilt. The call answers a task_id at once: a worker resolves the descendants and creates their builds, and GET /orchestration/tasks/{id} gives their ids once it has (result.build_ids), or noop when nothing downstream has a producer. GET /orchestration/lineage/{id}?direction=upstream|downstream shows the graph a build or a propagation will walk.

​
Schedules

A schedule runs a build, a maintenance or a flow on a cron, in a time zone of your choice, and records each firing. See Schedules.

​
Testing a change on a branch

A build takes a branch: its outputs are written on that data branch, its tabular inputs are read on it when they have it and on the trunk otherwise, and the code checked out is the git branch of the same name when the repository has one.
A fork creates the branch on every dataset of the root’s lineage, upstream and downstream, in constant time. A merge promotes each one; DELETE /orchestration/branches/{name}?root_id= drops a branch without promoting it. Promoting a single dataset’s branch is POST /datasets/{id}/branches/{name}/promote.

​
Arranging the graph

The Atlas interface lays a project’s workflow out by itself, left to right. Drag a node and it stays where you put it, for everyone who opens the project: the position is kept on the resource behind the node — a dataset, a mediaset, a model, a connection, a transform or a sync — and the nodes nobody has moved keep taking their automatic place around it. Auto-layout forgets every position of the project. The layout is the project’s, under its own rights: reading it takes projects:read, moving a node projects:write. A resource moved to another project leaves its position behind.

​
Keeping the graph in step with the code

The graph is read from the code. A repository is scanned when it is saved, after each successful build, and on demand with POST /orchestration/repositories/{id}/scan-lineage?ref=, or client.repositories.scan_lineage(repository_id, wait=True) and gwenlake repositories scan res.repository.… --wait, which answer the scan’s counts (transforms_found, edges_created) and warnings. A scan of the default branch is authoritative; a scan of another git branch adds what was born there. See Lineage, builds and branches.

​
Flows

A flow is a no-code graph of nodes edited in the Atlas interface and versioned like a SQL query, for the pipelines that are not dataset-to-dataset transformations: calling a model on each row, posting a result to an external system, orchestrating agents. A schedule runs a flow with the flow action. Flows are not covered by the Python library. From the library, a transform is run by calling it with a client, which runs it where the code is, with the caller’s identity.
These endpoints are served by the platform and used by the Atlas interface; they are being opened on the public gateway progressively, and the Python library does not wrap them yet. The API reference lists what the gateway exposes today.