Multi-modal input

A Message whose content is a list of parts carries images, audio clips and files alongside text. Each client translates the parts to its provider’s format (image_url for OpenAI, an image block for Anthropic, inline_data for Google…), so the same message works everywhere the model supports the media.

Images

from gwenflow import ChatOpenAI
from gwenflow.types import ImageContent, Message, TextContent

msg = Message(role="user", content=[
    TextContent(content="What's in this image?"),
    ImageContent.from_url("https://example.com/cat.jpg"),
])

response = ChatOpenAI(model="gpt-4o-mini").invoke([msg])
print(response.text)

ImageContent is built from a URL, a local file or raw bytes:

ImageContent.from_url("https://example.com/cat.jpg", detail="high")
ImageContent.from_path("/tmp/photo.jpg")                      # base64-encoded, media type guessed
ImageContent.from_bytes(png_bytes, media_type="image/png")

Audio

from gwenflow.types import AudioContent

msg = Message(role="user", content=[
    TextContent(content="Transcribe this clip."),
    AudioContent.from_path("/tmp/recording.wav"),
])

AudioContent.from_bytes(data, format="wav") takes raw bytes. Audio input works with gpt-4o-audio-preview and Gemini 2.5. Audio produced by a model comes back as response.audio: one AudioContent with the base64 data and, when available, the transcript.

PDFs and files

from gwenflow import ChatAnthropic
from gwenflow.types import FileContent

msg = Message(role="user", content=[
    TextContent(content="Summarise this report."),
    FileContent.from_path("/tmp/report.pdf"),
])

response = ChatAnthropic(model="claude-sonnet-4-6").invoke([msg])

A FileContent is one of:

Constructor	Sends
`FileContent.from_path(path)`	the file inline, base64-encoded
`FileContent.from_bytes(data, ...)`	bytes inline
`FileContent.from_url(url, media_type="application/pdf")`	a hosted URL (Anthropic accepts it for PDFs)
`FileContent.from_file_id(file_id)`	a file already uploaded to the provider’s Files API

PDF input works with gpt-4o, Claude and Gemini.

With an agent

An agent accepts the same list of messages as input:

response = agent.run([msg])

The parts stay in the agent’s history; the context budget counts images, audio and files with conservative estimates.

Get started

Concepts

Models

Agents

Multi-agent

Extend

Memory

Flows

Knowledge

Production

Project

Multi-modal input

Images

Audio

PDFs and files

With an agent

​Images

​Audio

​PDFs and files

​With an agent

Images

Audio

PDFs and files

With an agent