Compacting long sessions
Keeping long conversations affordable
Compaction is how Parlant keeps a long conversation affordable. Every model call carries the session's history, so cost grows with every message. When the history grows past a threshold, Parlant summarizes the earlier part into one durable summary and continues on top of it. Your agent keeps the facts that matter and pays less per turn. It is on by default and needs no setup.
- When compaction runs, and how the threshold depends on effort
- What the summary keeps at each detail level
- How to tune thresholds and detail when your conversations need it
- How to read what compaction did when a session surprises you
Without compaction, every model call in a turn carries the whole history, so the cost of every call grows with the session's length. Models also attend less reliably to details buried in a huge transcript. And a long enough session eventually overflows the context window altogether.
Once Parlant has compacted a session, every prompt it builds, for responding, matching, reviewing, and pruning, sees the summary plus only the events that came after it.
The rest of this page explains when compaction happens, what the summary keeps, and how to tune it when your conversations give you a reason to.
When it happens
Compaction runs after the turn's reply has already been sent, in the finalize stage, so your customer never waits on it. At that point, Parlant estimates the token count of the history since the last compaction, including the reply it just sent, and compares it with a threshold.
The threshold depends on effort
The threshold depends on the turn's effort, the level from MIN to MAX that decides how much model work a turn gets. Your agent sets the baseline, and a matched rule can raise it for one turn:
| Effort | Compacts at about | Summary detail |
|---|---|---|
MIN, LOW | 24,000 tokens | Low |
MEDIUM | 32,000 tokens | Medium |
HIGH | 48,000 tokens | High |
MAX | 64,000 tokens | High |
Higher effort tolerates a longer raw history, and preserves more when it does compact. This is what effort means in general: spend more to lose less.
Note that the effort Parlant consults is the turn's resolved level. So if a rule raised the effort for this turn, that changes which row of the table applies.
The compaction marker
In the session, you will see a Compacting session status, followed by a system message carrying the summary. That message is the compaction marker: the point in the history from which Parlant reads the summary instead of the events that came before it.
You can identify the marker by its metadata (the free-form data every event carries), where metadata.__parlant__.source == "compacter". If your frontend should not show summaries to customers, it can filter on that metadata. The official widget already does.
If the summary generation fails, Parlant logs the error and the turn is unaffected, since the reply already went out. The history is still over the threshold, though, so Parlant tries again at the end of the next turn.
What survives
Parlant writes the summary with your agent's full instructions and rule inventory in view. This way, the summarizer knows which details your agent is obligated to act on, and must not drop.
Parlant instructs it to preserve facts, requests, commitments, tool results, confirmations and denials, unresolved questions, and policy-sensitive state. It also instructs it to distinguish what was confirmed from what was merely assumed, and never to invent anything or include the agent's private reasoning.
Detail levels
The detail level shapes how aggressively the summarizer compresses:
- Low keeps durable facts, active goals, important tool results, unresolved tasks, and policy-sensitive state.
- Medium adds major decision points, and enough chronology to explain how the session got here.
- High keeps precise identifiers, preferences, summarized tool calls and results, and the latest state of every active request.
For example, if a customer gave an order number early in a long session, a high detail summary keeps the number itself. A low one may only keep the fact that an order is being discussed.
Note that compaction never deletes history. Old events and earlier markers stay in the session store. Only what Parlant loads into its working view (the part of the history it puts in front of the model) changes. Compactions also chain: each new summary folds in the last one.
Compaction is about one session's history. What your agent should remember about a customer across sessions is a different mechanism, built in Cross-session memory.
Tuning it
Our recommendation is to keep the defaults unless your conversations show a reason to change them. If prompts grow too large, compact earlier. If summaries lose details the agent still needs, keep more history or increase the summary detail. Then test the same conversation again.
The compaction policy
The compaction policy is a per-effort table. For each effort level, a CompactionPolicy pairs the token_threshold that triggers compaction with the detail_level of the summary. It lives on the compacter, the component that writes summaries.
Your coding agent replaces it through initialize_container, a function you pass to p.Server that runs once at startup with the container (the registry that maps each server component to the implementation serving it). Resolving Compacter there gives you the same instance the engine uses. Pass a complete mapping, one entry per effort level:
import parlant.sdk as p
from parlant.core.engines.compass.compacter import (
Compacter,
CompactionDetail,
CompactionPolicy,
)
async def initialize_container(container: p.Container) -> None:
container[Compacter].set_policy( # the running compacter; the change sticks
{
p.Effort.MIN: CompactionPolicy(token_threshold=16_000, detail_level=CompactionDetail.LOW),
p.Effort.LOW: CompactionPolicy(token_threshold=16_000, detail_level=CompactionDetail.LOW),
p.Effort.MEDIUM: CompactionPolicy(token_threshold=24_000, detail_level=CompactionDetail.MEDIUM),
p.Effort.HIGH: CompactionPolicy(token_threshold=32_000, detail_level=CompactionDetail.HIGH),
p.Effort.MAX: CompactionPolicy(token_threshold=48_000, detail_level=CompactionDetail.HIGH),
}
)
async with p.Server(initialize_container=initialize_container) as server:
...
Choosing thresholds and detail
Lower thresholds compact earlier. You get cheaper long sessions and more frequent summarization calls, but you lose verbatim history sooner.
Higher thresholds keep more raw history around, which helps reference-heavy conversations, at a per-turn cost that grows linearly with the history.
Detail level is the fidelity knob within a compaction. If your sessions carry many identifiers and in-flight requests, prefer high detail, even at lower efforts.
Note that the Compacter import comes from parlant.core, which is internal and may move between releases, unlike the p. names.
Parlant chooses the summarization model through model routing, under the compaction route (the name for this kind of work in the models mapping), so it can run on a different model than the responder. Model routing covers routes, and Overriding engine behavior covers the initialization hook.
Debugging it
The session hasn't compacted. Parlant prints Compaction check: lines in the debug log, with the estimated tokens, the active threshold, and the effort that selected it. Remember that the check runs after the reply, so a session crossing the threshold mid-turn compacts at the end of that turn.
The agent forgot something after compaction. Find the marker event and read the summary. Either the detail did not survive, which a higher detail level or threshold may help preserve, or it was never in the history to begin with. The full compaction prompt is in the turn's diagnostic report; Debugging issues shows how to download it.
Compaction failed. Look for Session compaction failed after response generation in the log. The turn is unaffected, and Parlant retries at the end of the next one.