Rich output

Handing the frontend more than text

Rich output is content a tool hands to your frontend alongside the agent's reply: a receipt to download, an order card, a chart. The tool puts that content on a side channel that the model never sees. Your frontend reads it and renders it next to the reply. Use it whenever a customer should get more than text, without asking the model to copy a link or a payload into its words.

What you'll learn

• The two channels on a tool result: one for the model, one for the frontend
• How a tool event reaches the frontend and what it looks like
• What customers and guests see when a tool result is redacted
• Why rendering the payload is your frontend's job

Talk this page through with your coding agent

This page covers both sides of that hand-off: what the tool returns, and what your frontend reads out of it.

A few terms first. A tool is a Python function your agent can call during a turn, and a turn is Parlant's handling of one customer message, from receiving it to posting the reply. Everything a turn produces, Parlant records as events in the session (the stored log of one conversation).

The idea is simple. To show a download or a card, the tool returns its data on the tool result's metadata. Your frontend renders that data alongside the agent's reply. The model receives a separate description of what happened, so it never has to copy a file URL or a UI payload into its text.

Two channels on a tool result

Rich output lives on the tool result, so that is where we look first.

A tool returns a p.ToolResult, and two of its fields matter here. data is what the model reads and reasons with, so it should say what happened in words the agent can pass on. metadata is only for the frontend. The model never sees it.

Put anything the UI needs in metadata: a link, a file reference, or a small object your UI knows how to render. Parlant defines no widget format of its own; your frontend decides what a payload looks like.

Consider an invoice. The example below returns a custom invoice payload, which needs a matching renderer in your frontend. (The official widget uses a widget object with a type naming the renderer; Frontend integration shows that convention.)

import parlant.sdk as p


@p.tool
async def generate_invoice(order_id: str) -> p.ToolResult:
    url = await billing.render_invoice_pdf(order_id)   # stored in your storage

    return p.ToolResult(
        data=f"Invoice for order {order_id} is ready and shown to the customer for download.",   # the model reads this
        metadata={"invoice": {"url": url, "media_type": "application/pdf"}},   # only the frontend reads this
    )

Here, the model reads data, so it can tell the customer the invoice is ready. It never sees the URL. Your frontend reads metadata and renders a download card.

Note that the two channels don't overlap. If the agent needs to know something, it belongs in data. metadata is invisible to it.

How the frontend gets it

Once a tool has written its metadata, it reaches the browser like this.

Your frontend reads it from the same event feed it already uses for replies, GET /sessions/{id}/events. Parlant records a tool call there as an event with kind: "tool". It arrives alongside the message events that carry replies and the status events that report the turn's progress.

Every event from one turn shares the same trace_id (an id Parlant assigns to the turn). To show a tool's output next to the reply it belongs to, take the trace_id from the message event and match it against the tool events from the same stream.

The shape of a tool event

A tool event's data holds the turn's tool calls. Here's one, trimmed to the fields that matter:

{
  "kind": "tool", "source": "system", "trace_id": "trc_9f…",
  "data": {
    "tool_calls": [
      {
        "tool_id": "built-in:generate_invoice",
        "arguments": { "order_id": "5567" },
        "result": {
          "metadata": { "invoice": { "url": "…", "media_type": "application/pdf" } },
          "data": "Invoice for order 5567 is ready…"
        }
      }
    ]
  }
}

Your frontend reads toolEvent.data.tool_calls[i].result.metadata. Each entry in tool_calls records one call: the tool's id, the arguments the agent passed, and the result it returned, with the same data and metadata the tool wrote.

There are three things to get right:

• tool_calls has one entry per call, not per tool. A tool that runs three times appears three times; tell the entries apart by their arguments.
• tool_id is qualified as service:tool. SDK tools appear as built-in:generate_invoice, so match accordingly.
• Skip the __parlant__ key if it appears in result.metadata. It's Parlant's own namespace, not your tool's data.

Who is reading

Parlant checks the caller's permissions on every read. For customers and guests, result.data arrives as {"redacted": true}, because that field can hold internal detail meant for the model, not for them.

Note that the shape is kept, so check for the marker rather than for a missing field. metadata is always kept, which is exactly what your frontend needs.

Loading a turn's tool calls later

If you show the live stream, you already have these events as they arrive. Rendering history is a different problem: you have a stored message event and want the tool output that went with it.

The endpoint's filters answer exactly that question. kinds=tool keeps only tool events, and trace_id keeps only that turn's.

Note that the endpoint normally long-polls (it waits wait_for_data seconds for new events before returning), so pass wait_for_data=0 to return at once. A turn that called no tools returns an empty list:

import httpx

PARLANT_URL = "http://localhost:8800"


def tool_calls_for_trace(session_id: str, trace_id: str) -> list[dict]:
    events = httpx.get(
        f"{PARLANT_URL}/sessions/{session_id}/events",
        params={"trace_id": trace_id, "kinds": "tool", "wait_for_data": 0},
    ).raise_for_status().json()

    return [call for e in events for call in e["data"]["tool_calls"]]

The same query in the browser:

async function toolCallsForTrace(sessionId: string, traceId: string) {
  const params = new URLSearchParams({ trace_id: traceId, kinds: "tool", wait_for_data: "0" });
  const res = await fetch(`${PARLANT_URL}/sessions/${sessionId}/events?${params}`);
  const events = await res.json();
  return events.flatMap((e: any) => e.data.tool_calls ?? []);
}

Rendering is the frontend's job

Parlant delivers the payload and stops there. Rendering is yours, which in practice means your coding agent writes a renderer for each payload type.

The server's built-in chat page shows only text and status. The official chat widget renders tool metadata through a small registry of renderers, one per payload type, so you can register a component for invoice and be done.

Learn more about the widget

Frontend integration shows the registration.

Note that tool metadata is hidden from the agent and not moderated. Don't put agent-relevant facts there; use data.

info

For frontend-only data outside a tool call, post an event with kind: "custom" on the session. It's a free-form event Parlant stores and delivers but never interprets: same feed, no tool needed.