Models

Every LLM client in Gwenflow subclasses ChatBase and exposes the same four methods, takes the same messages and returns the same ModelResponse. Changing provider is changing one line.

from gwenflow import ChatOpenAI, ChatAnthropic, ChatGoogle, ChatMistral

llm = ChatOpenAI(model="gpt-5-mini")
# llm = ChatAnthropic(model="claude-sonnet-4-6")
# llm = ChatGoogle(model="gemini-2.5-flash")
# llm = ChatMistral(model="mistral-small-2603")

response = llm.invoke("Describe Argentina in one sentence.")
print(response.text)

Providers

Client	Default model	API key	SDK
`ChatOpenAI`	`gpt-5-mini`	`OPENAI_API_KEY`	`openai`
`ResponseOpenAI` (`gwenflow.llms.openai_response`)	`gpt-5-mini`	`OPENAI_API_KEY`	`openai`, Responses API
`ChatAnthropic`	`claude-sonnet-4-6`	`ANTHROPIC_API_KEY`	`anthropic`
`ChatGoogle`	`gemini-2.5-flash`	`GOOGLE_API_KEY` or `GEMINI_API_KEY`	`google-genai`
`ChatMistral`	`mistral-small-2603`	`MISTRAL_API_KEY`	`mistralai`
`ChatAzureOpenAI`	none	`AZURE_OPENAI_API_KEY`	`openai`
`ChatDeepSeek`	`deepseek-chat`	`DEEPSEEK_API_KEY`	OpenAI-compatible
`ChatOllama`	none, `base_url` required	none	OpenAI-compatible
`ChatGwenlake`	none	`GWENLAKE_API_KEY`	OpenAI-compatible

Anthropic, Google and Mistral use their native SDKs; the others speak the OpenAI Chat Completions format. Every client accepts api_key= to pass the key directly, and timeout and retry settings for its SDK.

OpenAI and compatible servers

ChatOpenAI takes the usual sampling parameters (temperature, top_p, max_completion_tokens, seed, stop, presence_penalty, frequency_penalty, logprobs…), reasoning_effort for reasoning models and prompt_cache_key. With base_url it talks to any OpenAI-compatible server: vLLM, LM Studio, Scaleway, a gateway.

llm = ChatOpenAI(
    model="Qwen/Qwen3.6-27B",
    base_url="http://vllm:8000/v1",
    api_key="none",
    context_window="auto",   # read max_model_len from the server
)

Anthropic

from gwenflow import ChatAnthropic

llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    max_tokens=4096,
    cache_system_prompt=True,   # mark the system prompt for prompt caching
)

thinking turns on extended thinking; see Reasoning.

Google

from gwenflow import ChatGoogle

llm = ChatGoogle(model="gemini-2.5-flash")                                       # Gemini API
llm = ChatGoogle(model="gemini-2.5-flash", vertexai=True, project="my-project", location="europe-west1")  # Vertex AI

Azure OpenAI

from gwenflow import ChatAzureOpenAI

llm = ChatAzureOpenAI(
    model="gpt-4o-mini",
    azure_endpoint="https://my-resource.openai.azure.com",
    azure_deployment="gpt-4o-mini",
    api_version="2024-10-21",
)

azure_endpoint, azure_deployment and api_version fall back on AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_DEPLOYMENT and AZURE_OPENAI_API_VERSION.

Ollama and local models

from gwenflow import ChatOllama

llm = ChatOllama(model="llama3.2", base_url="http://localhost:11434/v1")

ChatOllama extracts the <think>...</think> blocks local reasoning models write inline into separate thinking parts.

Gwenlake

ChatGwenlake calls the models served by Gwenlake, at https://integrate.api.gwenlake.com/v1 by default. It reads GWENLAKE_API_KEY, and GWENLAKE_ORGANIZATION when set.

from gwenflow import ChatGwenlake

llm = ChatGwenlake(model="<a model served by Gwenlake>")

Calling a model

response = llm.invoke("Hello")                 # ModelResponse
response = await llm.ainvoke("Hello")

for chunk in llm.stream("Tell me a story"):    # one ModelResponse per delta
    if chunk.text:
        print(chunk.text, end="", flush=True)

async for chunk in llm.astream("Tell me a story"):
    ...

Call await llm.aclose() to release the async HTTP client when you are done with it.

Structured answers

Set response_format to a Pydantic model and response.parsed holds the validated object:

from pydantic import BaseModel

class Country(BaseModel):
    name: str
    capital: str
    population_millions: float

llm = ChatOpenAI(model="gpt-5-mini", response_format=Country)
country = llm.invoke("Tell me about Argentina.").parsed

On an agent, use response_model instead: it also works while tools are on offer. See Structured output.

Context window

The agent keeps its history within the model’s context window, so it needs to know it. llm.context_window_info() tells the size and where it comes from, resolved in this order:

context_window=<int>: an explicit value, for instance a vLLM server started with a smaller --max-model-len;
context_window="auto": read from the server’s /models listing (max_model_len, context_length or context_window) on OpenAI-compatible clients;
the built-in table of known models (gwenflow.llms.models.MODELS), matched without case and without a provider or organisation prefix, so Qwen/Qwen3.6-27B-AWQ finds qwen3.6-27b;
fallback_context_window=<int>: a value used only for a model found nowhere else;
128,000 tokens, with a warning logged once.

llm = ChatOpenAI(model="my-finetune", base_url="...", fallback_context_window=32768)
print(llm.context_window_info())   # ContextWindowInfo(tokens=32768, resolved=False, source='fallback')

The agent lets its history use 75 % of the window and leaves the rest for the answer. See Context management.

Addressing models by name

Flows and the command line name a model as provider/model-name. make_llm builds the client from that string, with any extra keyword argument passed through:

from gwenflow.registry import make_llm

llm = make_llm("anthropic/claude-sonnet-4-6", temperature=0)
llm = make_llm("gpt-5-mini")                    # no provider: OpenAI

The provider keys are openai, anthropic, google (or gemini), mistral, azure, deepseek, ollama and gwenlake.

Get started

Concepts

Agents

Multi-agent

Extend

Memory

Flows

Knowledge

Production

Project

Models

Providers

OpenAI and compatible servers

Anthropic

Google

Azure OpenAI

Ollama and local models

Gwenlake

Calling a model

Structured answers

Context window

Addressing models by name

​Providers

​OpenAI and compatible servers

​Anthropic

​Google

​Azure OpenAI

​Ollama and local models

​Gwenlake

​Calling a model

​Structured answers

​Context window

​Addressing models by name

Providers

OpenAI and compatible servers

Anthropic

Google

Azure OpenAI

Ollama and local models

Gwenlake

Calling a model

Structured answers

Context window

Addressing models by name