Context management

An agent that works for a long time on one task never exceeds the model’s context window. Before every request the history is measured against the window of the model actually called, and compacted in tiers, cheapest first, when it nears the limit. Without any configuration, Agent(llm=llm) already does this.

How the history is measured

The window comes from the model: an explicit context_window, the server’s listing, the built-in table, or a fallback. See Models.
The count is anchored on the provider’s own report: after each reply, the agent records the input and output tokens the provider billed, which cover the system prompt, the tool schemas and any files. Only the messages added since are estimated locally (tiktoken by default), so a tokenizer mismatch cannot drift.

usage = agent.context_usage()
print(f"{usage.used_tokens:,} of {usage.window_tokens:,} tokens ({usage.fraction:.0%})")

ContextUsage also tells the limit (limit_tokens), whether the window is known (resolved) and whether the count is anchored on the provider (anchored). on_context_usage is called with it before every request:

agent = Agent(
    name="coder",
    llm=llm,
    on_context_usage=lambda u: print(f"context: {u.fraction:.0%} of {u.window_tokens:,} tokens"),
)

When compaction happens

Compaction triggers when the history reaches max_fraction of the window (0.85 for the agent’s default buffer), and stops once it is back under a target, 70 % of that limit by default (about 60 % of the window). That gap means compaction runs in batches, and between two batches the prefix sent to the model stays identical, so the provider’s prompt cache stays valid.

The tiers

Tier	What it does	Cost
`DeduplicateReads`	keeps only the latest read of a file (coding tools), on every request	none
`ClampOversized`	keeps the head and tail of one oversized answer, tool result or tool argument	none
`ClearToolResults`	replaces old tool results by a one-line stub naming the call; the `keep_pairs` most recent are kept	none
`SummarizeOlder`	folds the oldest messages into a running summary; a recent tail is kept verbatim	one LLM call
`SlidingWindow`	drops the oldest messages, leaving a one-line receipt	none

The tiers are applied in this order, re-measuring after each, until the history fits. Whatever happens:

a tool call and its result always move together;
the first user message, the task of the current run, and any message marked with pin() (gwenflow.memory.compaction) survive every tier;
a summarizer that fails never fails the run: the sliding window takes over for that batch.

The originals of what a compaction changed are kept in agent.history.compactions[-1].removed, and each compaction emits a compact_messages span with the tiers applied and the counts before and after (never the content).

The summarizer

The summary tier calls a model. By default it is a copy of the agent’s llm; give a cheaper one with summarizer:

agent = Agent(name="assistant", llm=ChatOpenAI(model="gpt-5"), summarizer=ChatOpenAI(model="gpt-5-mini"))

The summary’s tokens are counted in the run that triggered it. The summarizer reads a transcript where every tool result is capped, and updates the previous summary rather than starting over.

Compacting by hand

agent.history.compact()   # compact now, returns a record of what was done
agent.history.reset()     # empty the history

Large tool outputs

A single grep or file read can fill a small window. max_tool_output_tokens caps one tool result: a longer one is set aside, the model gets its beginning and a reference, and a read_tool_output(ref, offset, length) tool to page through the rest.

agent = Agent(name="analyst", llm=llm, tools=tools, max_tool_output_tokens=2000)

Warning the model

With warn_near_limits=True, the system prompt of a request carries a notice when the run nears a limit, so the model concludes instead of seeing its history rewritten or being cut off:

URGENT when the history reaches warning_threshold (0.7) of its budget, or the run warning_threshold of max_turns;
CRITICAL when a compaction is imminent, or critical_remaining_turns (3) turns are left.

The notice is request-scoped: nothing is written in the history.

A custom buffer

Pass your own buffer as history to tune everything:

Buffer	Compaction
`ChatMemoryBuffer`	every tier but the summary
`ChatSummaryMemoryBuffer`	every tier, with a summarizer `llm`

Parameter	Meaning
`max_fraction`	the trigger, as a share of the window, resolved on every request (exclusive with `token_limit`)
`token_limit`	the trigger, as a fixed number of tokens
`target_fraction`, `target_tokens`	where compaction stops (by default 70 % of the limit)
`tokenizer_fn`	how to count the messages added since the last reply
`strategies`	the list of tiers, replacing the default ones
`llm`	the summarizer (`ChatSummaryMemoryBuffer`)

default_strategies(summarizer=..., deduplicate={...}, clamp={...}, clear={...}, summarize={...}, sliding={...}) builds the default tiers with per-tier options; without a summarizer it skips the summary tier.

A coding agent on a 32K window

Recommended configuration for a coding agent on a model served by vLLM with a 32K window:

from gwenflow import Agent, ChatOpenAI
from gwenflow.memory import ChatSummaryMemoryBuffer
from gwenflow.memory.compaction import default_strategies
from gwenflow.tools import EditFileTool, GrepTool, ReadFileTool, ShellTool
from gwenflow.utils.tokens import hf_tokenizer

llm = ChatOpenAI(
    model="Qwen/Qwen3.6-27B",
    base_url="http://vllm:8000/v1",
    api_key="none",
    context_window=32768,             # the server's --max-model-len
)
summarizer = ChatOpenAI(model="Qwen/Qwen3.6-27B", base_url="http://vllm:8000/v1", api_key="none")

history = ChatSummaryMemoryBuffer(
    max_fraction=0.85,                # compaction triggers at 85 % of the window (~27.8K)
    target_fraction=0.55,             # and stops at 55 % (~18K): few batches, a stable prefix
    tokenizer_fn=hf_tokenizer("Qwen/Qwen3.6-27B"),   # count like the server (pip install transformers)
    llm=summarizer,
    strategies=default_strategies(
        summarizer=summarizer,
        clear={"keep_pairs": 4, "exclude_tools": frozenset({"EditFileTool"}), "min_clear_tokens": 200},
        summarize={"keep_ratio": 0.4, "tool_result_max_chars": 400},
    ),
)

agent = Agent(
    name="coder",
    llm=llm,
    history=history,
    tools=[ReadFileTool(base_dir="."), GrepTool(base_dir="."), EditFileTool(base_dir="."), ShellTool(base_dir=".")],
    max_tool_output_tokens=2000,      # a long grep or file read is paged with read_tool_output
    warn_near_limits=True,            # the model is told to conclude when a limit is close
)

hf_tokenizer(model, **kwargs) counts with the Hugging Face tokenizer of the served model (kwargs go to AutoTokenizer.from_pretrained: trust_remote_code, token…); tiktoken under-counts for Mistral, Qwen or Gemma models.

Get started

Concepts

Models

Agents

Multi-agent

Extend

Memory

Flows

Knowledge

Production

Project

Context management

How the history is measured

When compaction happens

The tiers

The summarizer

Compacting by hand

Large tool outputs

Warning the model

A custom buffer

A coding agent on a 32K window

​How the history is measured

​When compaction happens

​The tiers

​The summarizer

​Compacting by hand

​Large tool outputs

​Warning the model

​A custom buffer

​A coding agent on a 32K window

How the history is measured

When compaction happens

The tiers

The summarizer

Compacting by hand

Large tool outputs

Warning the model

A custom buffer

A coding agent on a 32K window