A first flow
flows/news_summary.yaml
gwenflow run flows/news_summary.yaml.
The definition
Nodes
Connections
connections maps a source node to the nodes that receive its output. The value is a list of output ports, each a list of targets:
analyse waits for both fetch_prices and fetch_news, which run in parallel; archive starts as soon as fetch_prices is done.
The flow itself takes name and settings. Flow-level settings (node_timeout, node_retries) are defaults for the nodes of a durable run; a local run only reads each node’s own settings.
What a node receives
A node gets its parents’ results as a dict keyed by parent name. An agent node fills the{node_name} placeholders of its task with them and appends the parents it does not reference as context. Writer nodes take the output of their parent (or of the one named in source).
Validation
A flow is validated when it is loaded: duplicate node names, connections to unknown nodes and cycles raiseFlowError, before anything runs.
Running
FlowRunner(path_or_flow) wraps a flow with a tracing span and graph plotting: run(), arun(), plot() (a PNG, needs pydot) and plot_interactive() (an HTML page, needs pyvis).
flow.sinks() names the output nodes, those no other node consumes; flow.topological_order() the order the nodes resolve in.
Failures
Locally, each node runs in a worker thread. A node that raises is retriedsettings.retries times with a backoff; a node that exceeds settings.timeout fails. When a node is out of retries, the nodes still running are cancelled and FlowNodeError (a FlowError) is raised, naming the node:
Credentials
Keep secrets out of flow files:credentials maps a parameter to an environment variable, read only when the node runs.
Flows in Python
Flow.from_dict() builds a flow from the same structure as the YAML file, and flow.to_dict() returns it, so a flow can be generated, stored or sent over the wire: