Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.

6 posts tagged with "release"

View All Tags

Parlant 4: Flight of the Pigeon

Parlant 4: Flight of the Pigeon
8 min read

Parlant has always been about control.

With 3.3, we're focused on what happens when your agent configuration grows, with more guidelines, more relationships, more tools, more edge cases.

The features in this release make it easier to manage complexity at scale, and to push behavioral control into places that were previously out of reach.

The headline changes are tag-based relationships for managing groups of guidelines as a unit, numeric priorities for explicit ordering without verbose relationship declarations, and transient guidelines that let tools dynamically inject behavioral instructions into the agent's context.

We've also added Agent.utter() for programmatic message generation in long-running background tasks, and made it significantly easier to update customer and session metadata from within tools.

Let's walk through the details!

Contents:

Tag-Based Relationships​

As your agent grows, you can accumulate a large number of guidelines.

Managing individual relationships between them (this guideline excludes that one, this one depends on that one) can potentially become tedious and fragile. For example, when adding a new compliance guideline, you have to remember to wire up all the right exclusions.

Tags relationships are a new way to solve this.

In 3.3, tags become first-class participants in the relationship system. You can define dependencies, exclusions, priorities, and reevaluation relationships at the tag level, and every guideline carrying that tag inherits the relationship automatically.

Tags as Sources​

When you define a relationship on a tag, every guideline tagged with it gets the relationship:

compliance_guidelines = await server.create_tag("compliance-guidelines")

redirect_off_topic = await agent.create_guideline(
condition="The customer is asking about topics outside our scope",
action="Redirect the conversation to our supported topics",
tags=[compliance_guidelines],
)

block_financial_advice = await agent.create_guideline(
condition="The customer asks for personal financial advice",
action="Explain that we cannot provide financial advice",
tags=[compliance_guidelines],
)

# All compliance guidelines take priority over casual conversation
discussing_flights = await agent.create_observation(
"The customer is discussing flights"
)

await compliance_guidelines.prioritize_over(discussing_flights)

Now every compliance guideline (both existing and future ones) automatically takes priority over the flight discussion guideline. When you add a new compliance rule, you can just tag it, without manually wiring relationships.

Tags as Targets​

Tags also work as relationship targets, letting you define relationships against an entire group:

upsell_guidelines = await server.create_tag("upsell-guidelines")

offer_seat_upgrade = await agent.create_guideline(
condition="The customer is booking a flight",
action="Offer a seat upgrade",
tags=[upsell_guidelines],
)

offer_lounge_access = await agent.create_guideline(
condition="The customer has a long layover",
action="Offer lounge access",
tags=[upsell_guidelines],
)

# When the customer is upset, suppress ALL upsell guidelines at once
customer_is_upset = await agent.create_observation(
"The customer is frustrated or upset"
)
await customer_is_upset.exclude(upsell_guidelines)

One relationship declaration suppresses an entire category of behavior. This is the kind of thing that previously required N individual exclusions (one for each upsell guideline) and partially broke every time you added a new one.

Tag-Based Reevaluation​

Tags also support reevaluation relationships with tools. When a tool fires, all guidelines carrying a tag with a reevaluation relationship to that tool are automatically re-evaluated:

balance_sensitive = await server.create_tag("balance-sensitive")

await agent.create_guideline(
condition="The customer's balance is below $500",
action="Warn about potential overdraft fees",
tags=[balance_sensitive],
)

await agent.create_guideline(
condition="The customer's balance exceeds $10,000",
action="Suggest our premium savings account",
tags=[balance_sensitive],
)

# After checking the balance, re-evaluate all balance-sensitive guidelines
await balance_sensitive.reevaluate_after(check_balance_tool)

Previously, you'd need to call reevaluate_after() on each individual guideline. With tag-based reevaluation, one declaration covers the whole group, and stays current as you add more guidelines to the tag.

Numeric Priorities​

Relationships like prioritize_over() and depend_on() are powerful, but they're verbose when you need to express a clear ordering across many guidelines.

Parlant 3.3 introduces a numeric priority property on guidelines and journeys. Higher priority means higher precedence. When multiple guidelines are matched, only those at the highest priority level end up active for the next turn of the conversation.

await agent.create_guideline(
condition="Customer asks about pricing",
action="Quote standard pricing",
priority=0, # Default priority is 0
)

await agent.create_guideline(
condition="Customer is an enterprise client",
action="Quote enterprise pricing with volume discounts",
priority=10,
)

await agent.create_guideline(
condition="Customer asks about a product under legal review",
action="Explain that pricing is temporarily unavailable",
priority=100,
)

The relational resolver uses priority values during filtering: when conflicting guidelines are both matched, the lower-priority one is demoted. This works alongside explicit relationships — you can mix and match based on what's clearest for your configuration.

Priorities default to 0, so existing guidelines are unaffected. Journeys support the same property:

compliance_journey = await agent.create_journey(
name="Identity Verification",
priority=50,
)

Preventing Unguided Responses​

A clever trick that's made possible by priorities is the ability to prevent unguided responses from the agent.

By creating a catch-all guideline with a negative priority (lower than the default 0), you can ensure that the agent only responds when guidelines are active:

 await agent.create_guideline(
matcher=p.MATCH_ALWAYS,
action="You must say you can't help them right now",
canned_responses=[
await server.create_canned_response("Sorry, I cannot help!"),
],
composition_mode=p.CompositionMode.STRICT,
priority=-1,
)

Transient Guidelines​

Sometimes the right behavioral instruction doesn't exist at configuration time, like when a tool discovers, at runtime, how the agent should respond to a specific situation.

Transient guidelines let tools and retrievers inject behavioral guidelines into the agent's context dynamically. These guidelines are ephemeral: they apply only to the current response, then disappear. They don't persist in the session, and they don't affect future interactions.

@p.tool
async def check_order_status(
context: p.ToolContext,
order_id: str,
) -> p.ToolResult:
order = await orders_api.get(order_id)

guidelines: list[p.TransientGuideline] = []

if order.status == "delayed":
guidelines.append({
"action": "Apologize for the delay",
})

if order.is_first_order:
guidelines.append({
"action": "Thank the customer for their first purchase",
})

return p.ToolResult(
data=order.to_dict(),
guidelines=guidelines,
)

The guidelines returned here are injected into the agent's context alongside the statically defined ones. They participate in priority filtering, support criticality levels, and can include descriptions, just like regular guidelines.

This is useful when you need to react to situations that are too dynamic to predefine

Transient guidelines are a natural extension of how tools already return canned responses and fields. They add another dimension: tools can now influence not just what data the agent sees, but how it should behave in response to that data.

Agent.utter()​

Real-world agents often need to speak without an incoming customer message triggering them to do so. For example, your agent might follow up on a background operation (like a flight booking), notify a customer about an event, or check in after a period of inactivity.

Agent.utter() enables programmatic message generation. You provide a session and a set of transient guidelines, and the agent generates a contextually appropriate message:

# Invoked
async def follow_up_on_flight_booking(session: p.Session) -> None:
booking_id = await session.metadata.get("booking_id")
status = booking_api.check_status(booking_id)

if status == "confirmed":
await agent.utter(
session=session,
guidelines=[
{"action": "Confirm their flight has been booked."},
],
)
else:
...

Session and Customer Metadata​

Managing session and customer state from within tools used to require working through the REST API. In 3.3, you can update metadata directly using the SDK's .current accessors.

Session Metadata​

@p.tool
async def classify_inquiry(context: p.ToolContext) -> p.ToolResult:
session = p.Session.current

# Tag the session with metadata for analytics and routing
await session.metadata.set("topic", "billing")
await session.metadata.set("priority", "high")

return p.ToolResult(data="Inquiry classified")

You can also update the session's mode (for human handoff) and transfer it to a different agent:

# Switch to manual mode
await session.update(mode="manual")

# Transfer to a specialist
billing_agent = await p.Server.current.get_agent(id="billing_specialist")
await session.update(agent=billing_agent)

Customer Metadata​

Similarly, tools can now update customer information directly:

@p.tool
async def update_customer_profile(
context: p.ToolContext,
new_location: str,
) -> p.ToolResult:
customer = p.Customer.current

await customer.metadata.set("location", new_location)

return p.ToolResult(data="Profile updated")

This is especially useful for progressive profiling — building up customer context over the course of a conversation without leaving the tool execution scope.

Other Changes​

SDK improvements:

  • Server.get_tag() supports lookup by id or name
  • reevaluate_after() now accepts multiple tools and returns Sequence[Relationship]
  • tags field type changed from Sequence[TagId] to Sequence[Tag] across all entities

Bug fixes:

  • Fixed deadlock when sending a message right after a preamble
  • Fixed SSE read_event endpoint stalling after first streaming chunk
  • Fixed response analysis logs not always reaching the integrated UI
  • Fixed embedding LRU cache eviction corruption
  • Fixed non-consequential tool calls being rejected when optional parameters are missing

Deprecations:

  • OpenAPI tool services are now deprecated; please migrate to SDK tool services

Getting Started​

Upgrade to Parlant 3.3:

pip install parlant --upgrade

In Summary​

  • Tag-based relationships let you manage behavior at the group level instead of wiring individual guidelines together.
  • Numeric priorities give you explicit ordering without verbose relationship declarations.
  • Transient guidelines let tools shape agent behavior at runtime, based on what they discover.
  • Agent.utter() makes your agent proactive.
  • Easier metadata management keeps your tools focused on business logic instead of API plumbing.

These features were shaped by feedback from teams running Parlant in production. As agents grow in complexity, the tools for managing that complexity need to grow with them. That's what 3.3 is about.

Questions? Join us on Discord or reach out via our contact page.

github Get in touch

Parlant 3.3: Scaling Control

Parlant 3.3: Scaling Control
8 min read

Parlant has always been about control.

With 3.3, we're focused on what happens when your agent configuration grows, with more guidelines, more relationships, more tools, more edge cases.

The features in this release make it easier to manage complexity at scale, and to push behavioral control into places that were previously out of reach.

The headline changes are tag-based relationships for managing groups of guidelines as a unit, numeric priorities for explicit ordering without verbose relationship declarations, and transient guidelines that let tools dynamically inject behavioral instructions into the agent's context.

We've also added Agent.utter() for programmatic message generation in long-running background tasks, and made it significantly easier to update customer and session metadata from within tools.

Let's walk through the details!

Contents:

Tag-Based Relationships​

As your agent grows, you can accumulate a large number of guidelines.

Managing individual relationships between them (this guideline excludes that one, this one depends on that one) can potentially become tedious and fragile. For example, when adding a new compliance guideline, you have to remember to wire up all the right exclusions.

Tags relationships are a new way to solve this.

In 3.3, tags become first-class participants in the relationship system. You can define dependencies, exclusions, priorities, and reevaluation relationships at the tag level, and every guideline carrying that tag inherits the relationship automatically.

Tags as Sources​

When you define a relationship on a tag, every guideline tagged with it gets the relationship:

compliance_guidelines = await server.create_tag("compliance-guidelines")

redirect_off_topic = await agent.create_guideline(
condition="The customer is asking about topics outside our scope",
action="Redirect the conversation to our supported topics",
tags=[compliance_guidelines],
)

block_financial_advice = await agent.create_guideline(
condition="The customer asks for personal financial advice",
action="Explain that we cannot provide financial advice",
tags=[compliance_guidelines],
)

# All compliance guidelines take priority over casual conversation
discussing_flights = await agent.create_observation(
"The customer is discussing flights"
)

await compliance_guidelines.prioritize_over(discussing_flights)

Now every compliance guideline (both existing and future ones) automatically takes priority over the flight discussion guideline. When you add a new compliance rule, you can just tag it, without manually wiring relationships.

Tags as Targets​

Tags also work as relationship targets, letting you define relationships against an entire group:

upsell_guidelines = await server.create_tag("upsell-guidelines")

offer_seat_upgrade = await agent.create_guideline(
condition="The customer is booking a flight",
action="Offer a seat upgrade",
tags=[upsell_guidelines],
)

offer_lounge_access = await agent.create_guideline(
condition="The customer has a long layover",
action="Offer lounge access",
tags=[upsell_guidelines],
)

# When the customer is upset, suppress ALL upsell guidelines at once
customer_is_upset = await agent.create_observation(
"The customer is frustrated or upset"
)
await customer_is_upset.exclude(upsell_guidelines)

One relationship declaration suppresses an entire category of behavior. This is the kind of thing that previously required N individual exclusions (one for each upsell guideline) and partially broke every time you added a new one.

Tag-Based Reevaluation​

Tags also support reevaluation relationships with tools. When a tool fires, all guidelines carrying a tag with a reevaluation relationship to that tool are automatically re-evaluated:

balance_sensitive = await server.create_tag("balance-sensitive")

await agent.create_guideline(
condition="The customer's balance is below $500",
action="Warn about potential overdraft fees",
tags=[balance_sensitive],
)

await agent.create_guideline(
condition="The customer's balance exceeds $10,000",
action="Suggest our premium savings account",
tags=[balance_sensitive],
)

# After checking the balance, re-evaluate all balance-sensitive guidelines
await balance_sensitive.reevaluate_after(check_balance_tool)

Previously, you'd need to call reevaluate_after() on each individual guideline. With tag-based reevaluation, one declaration covers the whole group, and stays current as you add more guidelines to the tag.

Numeric Priorities​

Relationships like prioritize_over() and depend_on() are powerful, but they're verbose when you need to express a clear ordering across many guidelines.

Parlant 3.3 introduces a numeric priority property on guidelines and journeys. Higher priority means higher precedence. When multiple guidelines are matched, only those at the highest priority level end up active for the next turn of the conversation.

await agent.create_guideline(
condition="Customer asks about pricing",
action="Quote standard pricing",
priority=0, # Default priority is 0
)

await agent.create_guideline(
condition="Customer is an enterprise client",
action="Quote enterprise pricing with volume discounts",
priority=10,
)

await agent.create_guideline(
condition="Customer asks about a product under legal review",
action="Explain that pricing is temporarily unavailable",
priority=100,
)

The relational resolver uses priority values during filtering: when conflicting guidelines are both matched, the lower-priority one is demoted. This works alongside explicit relationships — you can mix and match based on what's clearest for your configuration.

Priorities default to 0, so existing guidelines are unaffected. Journeys support the same property:

compliance_journey = await agent.create_journey(
name="Identity Verification",
priority=50,
)

Preventing Unguided Responses​

A clever trick that's made possible by priorities is the ability to prevent unguided responses from the agent.

By creating a catch-all guideline with a negative priority (lower than the default 0), you can ensure that the agent only responds when guidelines are active:

 await agent.create_guideline(
matcher=p.MATCH_ALWAYS,
action="You must say you can't help them right now",
canned_responses=[
await server.create_canned_response("Sorry, I cannot help!"),
],
composition_mode=p.CompositionMode.STRICT,
priority=-1,
)

Transient Guidelines​

Sometimes the right behavioral instruction doesn't exist at configuration time, like when a tool discovers, at runtime, how the agent should respond to a specific situation.

Transient guidelines let tools and retrievers inject behavioral guidelines into the agent's context dynamically. These guidelines are ephemeral: they apply only to the current response, then disappear. They don't persist in the session, and they don't affect future interactions.

@p.tool
async def check_order_status(
context: p.ToolContext,
order_id: str,
) -> p.ToolResult:
order = await orders_api.get(order_id)

guidelines: list[p.TransientGuideline] = []

if order.status == "delayed":
guidelines.append({
"action": "Apologize for the delay",
})

if order.is_first_order:
guidelines.append({
"action": "Thank the customer for their first purchase",
})

return p.ToolResult(
data=order.to_dict(),
guidelines=guidelines,
)

The guidelines returned here are injected into the agent's context alongside the statically defined ones. They participate in priority filtering, support criticality levels, and can include descriptions, just like regular guidelines.

This is useful when you need to react to situations that are too dynamic to predefine

Transient guidelines are a natural extension of how tools already return canned responses and fields. They add another dimension: tools can now influence not just what data the agent sees, but how it should behave in response to that data.

Agent.utter()​

Real-world agents often need to speak without an incoming customer message triggering them to do so. For example, your agent might follow up on a background operation (like a flight booking), notify a customer about an event, or check in after a period of inactivity.

Agent.utter() enables programmatic message generation. You provide a session and a set of transient guidelines, and the agent generates a contextually appropriate message:

# Invoked
async def follow_up_on_flight_booking(session: p.Session) -> None:
booking_id = await session.metadata.get("booking_id")
status = booking_api.check_status(booking_id)

if status == "confirmed":
await agent.utter(
session=session,
guidelines=[
{"action": "Confirm their flight has been booked."},
],
)
else:
...

Session and Customer Metadata​

Managing session and customer state from within tools used to require working through the REST API. In 3.3, you can update metadata directly using the SDK's .current accessors.

Session Metadata​

@p.tool
async def classify_inquiry(context: p.ToolContext) -> p.ToolResult:
session = p.Session.current

# Tag the session with metadata for analytics and routing
await session.metadata.set("topic", "billing")
await session.metadata.set("priority", "high")

return p.ToolResult(data="Inquiry classified")

You can also update the session's mode (for human handoff) and transfer it to a different agent:

# Switch to manual mode
await session.update(mode="manual")

# Transfer to a specialist
billing_agent = await p.Server.current.get_agent(id="billing_specialist")
await session.update(agent=billing_agent)

Customer Metadata​

Similarly, tools can now update customer information directly:

@p.tool
async def update_customer_profile(
context: p.ToolContext,
new_location: str,
) -> p.ToolResult:
customer = p.Customer.current

await customer.metadata.set("location", new_location)

return p.ToolResult(data="Profile updated")

This is especially useful for progressive profiling — building up customer context over the course of a conversation without leaving the tool execution scope.

Other Changes​

SDK improvements:

  • Server.get_tag() supports lookup by id or name
  • reevaluate_after() now accepts multiple tools and returns Sequence[Relationship]
  • tags field type changed from Sequence[TagId] to Sequence[Tag] across all entities

Bug fixes:

  • Fixed deadlock when sending a message right after a preamble
  • Fixed SSE read_event endpoint stalling after first streaming chunk
  • Fixed response analysis logs not always reaching the integrated UI
  • Fixed embedding LRU cache eviction corruption
  • Fixed non-consequential tool calls being rejected when optional parameters are missing

Deprecations:

  • OpenAPI tool services are now deprecated; please migrate to SDK tool services

Getting Started​

Upgrade to Parlant 3.3:

pip install parlant --upgrade

In Summary​

  • Tag-based relationships let you manage behavior at the group level instead of wiring individual guidelines together.
  • Numeric priorities give you explicit ordering without verbose relationship declarations.
  • Transient guidelines let tools shape agent behavior at runtime, based on what they discover.
  • Agent.utter() makes your agent proactive.
  • Easier metadata management keeps your tools focused on business logic instead of API plumbing.

These features were shaped by feedback from teams running Parlant in production. As agents grow in complexity, the tools for managing that complexity need to grow with them. That's what 3.3 is about.

Questions? Join us on Discord or reach out via our contact page.

github Get in touch

Parlant 3.2: Streaming Responses

Parlant 3.2: Streaming Responses
10 min read

Parlant has always been about control: giving you the ability to shape agent behavior precisely and reliably. Version 3.1 deepened that control with criticality levels, custom matchers, event handlers, and observability.

With 3.2, we've taken important steps to enhance user experience: not only in improving the UX itself, but also in making it easier to monitor and analyze your agent's interactions with your users.

We've heard from teams running Parlant in production that while behavioral accuracy is critical, perceived responsiveness and operational visibility matter just as much for adoption. This release is shaped by that feedback.

The headline change is streaming message output — responses that arrive token by token instead of as a single block. We've also added a built-in ability to categorize and track interaction sessions, along with deeper per-agent personality configuration, easier granular control of knowledge retrieval and context management, tighter control around canned response selection, and a collection of other fixes and improvements.

Let's walk through the details!

Contents:

What Your Customers Experience​

First, let's discuss the customer-facing improvements. Parlant 3.2 includes two major features that directly impact the user experience: streaming responses and enhanced agent personality configuration.

Streaming Responses​

When a customer sends a message, your agent matches guidelines, calls tools, assembles context, and finally generates a response message.

Parlant goes through hoops to improve UX during the context assembly stage (with responsive status indicators and preamble messages), but the message generation part can also take a few seconds, and this has thus far remained unaddressed.

During message generation, the customer only saw a "typing" indicator.

For short replies, that's fine. But for agents that produce longer responses, the wait creates a gap that can feel unnatural.

The method of streaming (watching text appear word by word) often feels more responsive. It also lends itself better to voice agents, since the voice response can start generating and playing as soon as the first tokens are generated, rather than waiting for the full message to be ready.

Parlant 3.2's new streaming output mode finally closes this gap:

agent = await server.create_agent(
name="Support Agent",
description="Handles customer inquiries",
# New parameter to control response delivery
output_mode=p.OutputMode.STREAM,
)

You configure the output mode per agent. We recommend streaming for customer-facing agents where perceived responsiveness matters.

info

When using canned responses, the agent will still revert to block output mode, even when streaming is enabled. This is useful when using dynamic composition mode, where only certain conditions trigger the use of canned responses, and the agent is left to operate in fluid mode for the rest of the time.

Agent Personality​

Preambles are the quick acknowledgment messages — like "Got it" or "Let me check that" — that Parlant sends while processing a response, to keep the user engaged.

However, as different agents have different personalities, in 3.2, you can customize preamble behavior per agent, with concrete examples.

preamble_config = p.PreambleConfiguration(
examples=["Understood.", "Let me look into that.", "One moment."],
)

You can also provide dynamic instructions to the preamble generator, so the preambles can adjust to the situation.

preamble_config = p.PreambleConfiguration(
examples=["Understood.", "Let me look into that.", "One moment."],
get_instructions=my_instruction_provider,
)

The get_instructions callable receives an EngineContext, which gives you access to the current state of the session — matched guidelines, active journeys, customer information, and conversation history. You can use this to tailor the preamble generation to the situation at hand:

async def my_instruction_provider(ctx: p.EngineContext) -> Sequence[str]:
instructions = []

# Check the time of day for the customer
hour = datetime.now().hour

if hour < 12:
instructions.append("Use a morning greeting like 'Good morning'.")
elif hour >= 18:
instructions.append("Use an evening greeting like 'Good evening'.")

return instructions

This way, the same agent can adjust its preamble tone based on context — time-aware greetings, or any other signal you can derive from the session state.

What You Can See​

With the customer experience addressed, let's turn to the developer's side.

Labels​

One of the things that continues to surprise even ourselves about Parlant's architecture is how much you can build on top of Parlant's fundamental design decision: that guidelines and journeys are individual, well-defined units.

When you know exactly which instruction got activated and when, that opens doors you didn't plan for. Labels are a good example.

You can now set labels directly on guidelines, journeys, and journey states. Simple string tags. What does this enable?

Labels from these control primitives propagate into the session. Whenever a guideline matches, the session gets stamped with that guideline's labels. The same applies to journeys and journey states.

Over the course of a conversation, the session picks up relevant labels — giving you a record of what was activated and when.

await agent.create_guideline(
condition="The agent sees an opportunity to recommend a premium plan",
action="Suggest the premium plan and explain its benefits",
labels=["upsell_attempt"],
)

await agent.create_guideline(
condition="The customer is frustrated and wants to speak to a human",
action="Transfer the conversation to a human agent",
tools=[human_handoff],
labels=["human_handoff"],
)

Now imagine you need to run analytics or filter sessions. With labels, you can pull up all sessions where an upsell was attempted, and examine them closely to understand how customers responded and optimize the interaction.

For example, if you have a human_handoff label, you can quickly surface every session where escalation occurred and investigate why.

And you can apply labels to journeys and states as well, not just guidelines.

Querying by labels is built into the client SDKs. For example, using parlant-client in TypeScript, you can filter sessions by one or more labels:

const upsellSessions = await client.sessions.list({
labels: ["upsell_attempt"],
});

We expect labels to be useful for:

  • Analytics and optimization: Which guidelines fire most? How do customers respond to upsell attempts?
  • Routing: Route conversations to specialized agents or human teams based on accumulated labels.
  • Filtering: Surface all sessions that touched a specific topic or triggered a specific flow.

How You Tighten the Configuration​

Now that you can see what's happening and your customers are getting a better experience, the natural next step is refining the underlying mechanisms.

Parlant 3.2 adds improvements to guidelines, journeys, and canned responses.

Controlling Reapplication with track​

As you may already know, Parlant, by default, tracks whether your agent has already applied a guideline's action and deactivates it to prevent repetition, even the condition is still met when looked at in its own right.

Usually, this is exactly what you want. You don't want the agent explaining the return policy five times in one conversation.

But not all actionable guidelines are one-time actions. Some are ongoing behavioral cues — tone adjustments, empathy responses, compliance reminders — that should apply every time their condition is met. For these, the default tracking may sometimes get in the way.

The new track parameter gives you explicit control on whether to enable it:

# Default: tracked, applies once per context
await agent.create_guideline(
condition="Customer asks about pricing",
action="Provide current pricing",
track=True, # default
)

# Untracked: reapplies every time the condition matches
await agent.create_guideline(
condition="Customer expresses frustration",
action="Acknowledge their frustration and offer help",
track=False,
)

Setting track=False bypasses the "previously applied" analysis entirely, so that your agent continually keeps this guideline in mind while the condition applies.

Scoped Retrievers for Guidelines and Journeys​

In 3.1, retrievers were attached at the agent level and ran on every interaction. This works, but we found it leads to unnecessary context pollution in practice — pricing docs get retrieved during a password reset, troubleshooting articles load during a billing inquiry.

It was always possible to work around this using deferred retrievers, but it was something of an advanced usage pattern. So in 3.2, we've made this common pattern simpler to use.

You can now scope retrievers to specific guidelines or journeys. They only activate when the associated entity matches:

async def fetch_pricing_docs(ctx: p.RetrieverContext) -> p.RetrieverResult:
docs = await pricing_db.search(ctx.interaction.last_customer_message)
return p.RetrieverResult(data=docs)

pricing_guideline = await agent.create_guideline(
condition="Customer asks about pricing",
action="Provide pricing based on their plan",
)

await pricing_guideline.attach_retriever(fetch_pricing_docs)

The same works for journeys and journey states: they all now support .attach_retriever().

The result is a cleaner context window and fewer wasted retriever calls. If the pricing guideline doesn't match, the pricing docs simply aren't fetched.

This approach trades some flexibility for precision. Agent-level retrievers remain the right choice when data is broadly useful across many guidelines. Scoped retrievers work best when data is clearly tied to a specific guideline or journey.

Field Dependencies for Canned Responses​

Canned responses often reference dynamic data through template fields — {{order_status}}, {{delivery_date}}. Until now, there was no explicit way to express that a response requires those fields to be available, aside from actually referencing them in the response.

Field dependencies make this explicit, so you can condition the engine's selection of canned responses on the presence of specific fields in the context, even without your response referencing them directly:

await agent.create_canned_response(
template="Yay, your order is confirmed!",
field_dependencies=["new_order_id"],
)

With this declaration, the engine only considers the response when both fields are available from an active tool or retriever. If the data isn't there, it filters the response out early.

This matters most in strict composition mode, where your agent must use a canned response. Field dependencies prevent the engine from selecting a response it can't fully render — and, consequently, from saying something it definitely shouldn't, like claiming something happened when it didn't.

SDK Ergonomics​

A few smaller improvements that clean up common patterns:

Bulk relationship definitions​

prioritize_over() and depend_on() now accept multiple targets, so you can express a group of relationships in a single call:

await refund_guideline.prioritize_over(upsell_guideline, promo_guideline)
await compliance_guideline.depend_on(identity_check, account_lookup)

Always-on guidelines​

Guideline conditions are now optional, so you can use a custom matcher without needing to provide a dummy condition:

await agent.create_guideline(
matcher=an_order_was_just_placed,,
action="Say 'Thank you for choosing Acme'",
)

Other Changes​

SDK improvements:

  • Custom journey node IDs via the id parameter
  • Matched guidelines and journey states included in the completion-ready event

Engine improvements:

  • Tweaked default preamble examples for more natural tone
  • Softened log levels for the relational guideline resolver
  • Added activated/skipped logs to custom guideline matcher batches

Bug fixes:

  • Fixed WebSocket warning on startup
  • Fixed agent intention proposer incorrectly rewriting guidelines
  • Fixed a bug with guideline/journey relationship resolution
  • Fixed multiple custom guideline matchers not working together
  • Fixed context variable access bug in SDK

Getting Started​

Upgrade to Parlant 3.2:

pip install parlant --upgrade

To enable streaming on an existing agent:

await agent.update(output_mode=p.OutputMode.STREAM)

In Summary​

Streaming responses and per-agent personality make your agent feel present. Labels give you visibility into what's actually happening across conversations. Scoped retrievers, field dependencies, and lifecycle hooks tighten the feedback loop between what you observe and what you configure.

Many of the features in this release came directly from conversations with teams building agents at scale. We're grateful to everyone who contributed feedback, reported issues, and shared their production experiences — this release is better because of it.

Questions? Join us on Discord or reach out via our contact page.

github Get in touch

Parlant 3.1

Parlant 3.1
11 min read

The Emerging Power of Granular Behavior Control

Parlant has always been about control: giving you the ability to shape agent behavior precisely, reliably, and at scale.

Version 3.1 takes this further by adding granular control mechanisms across every layer of the framework.

This release introduces criticality levels for guidelines, custom matchers that let you override default matching logic, event handlers for guidelines and journey states, linked journeys for modular flow composition, dynamic composition mode overrides, and field providers that extend canned response capabilities.

Each of these features gives you finer control over how your agent behaves in specific situations.

Alongside these framework improvements, we're also launching Emcie, our first commercial offering: an auto-optimizing inference platform for teams looking to reduce operational costs while maintaining response accuracy.

We've worked closely with the community over the past 4 months to build on the foundation of version 3.0. Let's dive into the details.

Contents:

Emcie Logo

As Parlant agents scale, there's a natural opportunity to optimize inference costs without compromising accuracy. Large language models deliver excellent results, but smaller models can often achieve comparable performance for specific, well-defined tasks once they're tuned for the job.

Incidentally, Parlant's architecture lends itself perfectly to SLM distillation, since each NLP task (guideline matching, tool calling, message generation) can be optimized independently.

We're super excited to introduce Emcie: our new auto-optimizing inference platform for Parlant agents. It automatically reduces operational costs through dynamic prompt tuning and SLM distillation, while maintaining the accuracy your agents need. Learn more at emcie.co.

With that, let's explore the key improvements in Parlant 3.1!

Framework Improvements​

Guidelines and Journeys​

Guidelines and journeys are central to how Parlant controls agent behavior. This release includes several improvements to both.

Linked Journeys​

You can now link journeys together by transitioning to another journey from within a state. This allows you to create modular, reusable journeys that compose into larger flows.

t = await state.transition_to(journey=payment_journey)
t = await state.transition_to(
condition="Customer wants to pay now",
journey=payment_journey,
)

When you transition to a journey, the sub-journey's states are embedded into the parent journey. The transition returns a JourneyTransition whose .target is a fork state, letting you continue building the flow after the sub-journey completes.

This is useful for separating concerns: a checkout journey can link to a payment journey, which can link to a receipt journey, each maintained independently.

Journey Step Matching Efficiency​

Journey state matching is now 2x more efficient. This directly reduces latency for agents with complex multi-step flows.

The improvement comes from optimizations in how the engine evaluates transition conditions and predicts which states are likely to be active.

State and Guideline Descriptions​

Both journey states and guidelines now support a description field for elaboration. This is useful when a condition or action requires clarification, or when experimentation reveals that the agent isn't interpreting things correctly.

t = await state.transition_to(
chat_state="Confirm the booking details",
description="""At this point, summarize all collected information:
destination, dates, number of travelers, and selected flight options.
Ask the customer to verify everything is correct before proceeding."""
)
await agent.create_guideline(
condition="The customer mentions a competitor product",
action="Acknowledge their experience and highlight our differentiators",
description="""Be respectful of competitor products while emphasizing
our unique value. Never disparage competitors directly. Focus on features
like our 24/7 support and flexible pricing."""
)

The description is included in the agent's context when the guideline or state is active, improving both matching accuracy and the agent's ability to follow the instruction correctly.

Guideline Criticality​

As we expanded on in our recent blog post, not all instructions carry the same weight. A compliance disclosure has different stakes than a greeting preference. Parlant 3.1 introduces criticality levels that let you specify how much effort the engine should spend ensuring conformance.

LevelBehaviorUse Case
HIGHFull ARQ-based enforcement; maximum resources allocatedCompliance, legal obligations, safety-critical instructions
MEDIUMStandard enforcement with some flexibility (default)Core business logic, important but not critical
LOWTreated as subtle cues rather than rigid requirementsStylistic preferences, nice-to-have behavioral nudges
await agent.create_guideline(
condition="Customer asks about cancellation fees",
action="Disclose the $50 cancellation fee before proceeding",
criticality=p.Criticality.HIGH
)

await agent.create_guideline(
condition="The conversation has just started",
action="Inquire if they want to hear about on-sale items",
criticality=p.Criticality.LOW
)

Low-criticality guidelines use a more efficient matching path, reducing compute costs for agents that make significant use of them. High-criticality guidelines always receive full ARQ-based matching and enforcement.

Match Handlers​

You can now register handlers that execute when a guideline or journey state is matched. This is useful for logging, analytics, or external integrations.

The on_match handler runs immediately after matching, before the agent generates its response:

async def log_match(ctx: p.EngineContext, m: p.GuidelineMatch) -> None:
await analytics.track("guideline_matched", {"id": m.id})

await agent.create_guideline(
condition=CONDITION,
action=ACTION,
on_match=log_match,
)

The on_message handler runs after the agent has generated and sent a message while the guideline was active:

await agent.create_guideline(
condition=CONDITION,
action=ACTION,
on_message=post_message_handler,
)

Journey state transitions support the same handlers:

await state.transition_to(
chat_state=CHAT_STATE,
on_match=my_handler,
on_message=my_handler,
)

Custom Matchers​

By default, Parlant uses LLM-based guideline matching. Custom matchers let you override this for specific guidelines.

async def my_matcher(
ctx: p.GuidelineMatchingContext,
guideline: p.Guideline
) -> p.GuidelineMatch:
return p.GuidelineMatch(
id=guideline.id,
matched=True,
rationale="Custom logic determined this applies",
)

await agent.create_guideline(
condition=CONDITION,
action=ACTION,
matcher=my_matcher,
)

This enables several use cases:

Cost optimization: Match using simpler methods (embeddings, regex, smaller models) instead of LLM-based matching.

Always-on guidelines: For guidelines that should always be in context:

await agent.create_guideline(
condition="You need to present tabular data",
action="Use markdown table formatting",
matcher=p.MATCH_ALWAYS, # Built-in helper
)

Nuanced logic: Run specialized queries, consult external data sources, or apply complex business rules.

Subsequent Tool State Transitions​

Journeys now support transitioning through multiple consecutive tool states. This is useful for workflows that require several tool calls in sequence before presenting results to the customer.

Tools and Retrievers​

Tool calling is one of the most latency-sensitive parts of the response pipeline. This release includes several optimizations.

Tool Consequentiality​

Tools can now be marked as consequential or non-consequential. Consequential tools have side effects that require careful evaluation before execution. Non-consequential tools are read-only or easily reversible.

@p.tool  # Non-consequential by default
async def get_order_status(
context: p.ToolContext, order_id: str
) -> p.ToolResult:
status = await orders_api.get_status(order_id)
return p.ToolResult(data=status)

@p.tool(consequential=True)
async def process_refund(
context: p.ToolContext, order_id: str
) -> p.ToolResult:
result = await payments_api.refund(order_id)
return p.ToolResult(data=result)

For non-consequential tools, Parlant optimizes for speed. Tools with no parameters run instantly without evaluation overhead. Tools with parameters use a faster evaluation path.

For consequential tools, Parlant applies full validation before execution.

Optimized Tool Evaluation​

We've restructured the tool evaluation pipeline to avoid unnecessary evaluation steps. In certain response cases, this saves up to 15 seconds of latency. The improvement is most noticeable for agents with many tools where only a subset applies to any given context.

Deferred Retrievers​

Retrievers typically run early in the pipeline, in parallel with guideline matching. This is optimal for latency, but sometimes you want to decide whether to include retriever results based on which guidelines actually matched.

Deferred retrievers solve this by returning a callable instead of a direct result:

async def conditional_retriever(
ctx: p.RetrieverContext
) -> p.DeferredRetriever:
# Runs early, in parallel with guideline matching
documents = await fetch_documents(ctx.interaction.last_customer_message)

async def deferred(engine_ctx: p.EngineContext) -> p.RetrieverResult:
matched_ids = {g.id for g in engine_ctx.state.guidelines}

if MY_GUIDELINE_ID in matched_ids:
return p.RetrieverResult(data=documents)

return None # Skip adding data

return deferred

await agent.attach_retriever(conditional_retriever)

This gives you parallel execution for the retrieval work while allowing context-aware filtering after guideline matching completes.

Canned Responses​

Canned responses provide precise control over agent outputs, eliminating hallucination risk in high-stakes interactions. In Parlant 3.1, we've added several awesome enhancements.

Guideline and Journey State-Scoped Responses​

You can now associate canned responses directly with guidelines or journey states. These responses receive elevated consideration when the guideline is matched or the journey state is active.

await agent.create_guideline(
condition="Customer asks about refund policy",
action="Explain the refund policy",
canned_responses=[
await server.create_canned_response(
template="We offer full refunds within 30 days of purchase.",
),
]
)

The choice of creating them with the server or agent object matters: responses created via server are only considered when their associated guideline or state is active, while responses created via agent are always available but receive elevated priority when the association is active.

Dynamic Composition Mode​

While agents have a default composition mode (Fluid, Composited, or Strict), you can now override it temporarily based on active guidelines, journeys, or journey state transitions.

await agent.create_guideline(
condition="The customer is discussing off-topic subjects",
action="Explain you can only assist with account-related issues",
composition_mode=p.CompositionMode.STRICT,
canned_responses=[
await agent.create_canned_response(
template="I'm here to help with your account. Could we focus on that?"
),
]
)
t = await state.transition_to(
chat_state="Confirm payment details",
composition_mode=p.CompositionMode.STRICT,
)

When multiple composition modes are active, the most restrictive wins: STRICT > COMPOSITED > FLUID. This lets you maintain a flexible agent overall while enforcing strict control in high-stakes situations.

Canned Response Metadata​

Canned responses can now include metadata that passes through to the frontend without being visible to the agent:

await agent.create_canned_response(
template="Here are your recommendations: {{recommendations}}",
metadata={
"reply_suggestions": [
"Tell me more",
"Show alternatives",
"Add to cart"
],
},
)

When this response is selected, the metadata is included in the response event, allowing your frontend to access it. This is useful for sending UI hints, reply suggestions, or other frontend-specific data alongside the response.

Field Providers for Guidelines and Journey States​

Guidelines and journey states can now provide fields for canned response templates, similar to how tools provide fields:

async def provide_discount_code(ctx: p.EngineContext) -> dict[str, Any]:
code = await generate_discount_code(ctx.customer.id)
return {"discount_code": code}

await agent.create_guideline(
condition="Customer asks about available discounts",
action="Offer them a personalized discount",
canned_response_field_provider=provide_discount_code,
)
await agent.create_canned_response(
template="Here's your discount code: {{discount_code}}"
)

Observability​

Parlant 3.1 introduces built-in support for OpenTelemetry, the industry-standard framework for observability, with traces, metrics, and logs.

Backend Integration​

The OpenTelemetry integration works with any compatible backend: Jaeger, Grafana, Datadog, Honeycomb, and others. Each event in a session includes a trace_id field for correlating telemetry data with specific responses.

Management and Developer Workflow​

Manual Entity ID Control​

You can now specify IDs when creating entities (agents, customers, sessions, etc.) rather than letting Parlant generate them.

For agents, this ensures consistent IDs across versions and changes.

For customers, this is useful for integrating with existing systems that have their own ID schemes.

Mid-Session Agent and Customer Changes​

Sessions now support changing the agent ID or customer ID mid-conversation. Use cases include:

  • Escalation: Switching from a general support agent to a specialist agent
  • Customer authentication: Starting with an anonymous session and associating a customer after authentication

FastAPI Configuration​

The configure_api hook gives you access to the underlying FastAPI instance. This allows adding custom endpoints, middleware, or other extensions for your deployment.

async def configure_api(app: FastAPI) -> None:
@app.get("/custom/health")
async def custom_health_check() -> dict[str, object]:
return {"status": "healthy", "custom": True}

async with p.Server(configure_api=configure_api) as server:
await server.serve()

Health Endpoint​

A /healthz endpoint is now available for Kubernetes pod health monitoring and load balancer health checks.

Other Changes​

  • Package manager: Switched from Poetry to uv for faster dependency resolution and installation
  • Renamed context class: p.LoadedContext is now p.EngineContext. A deprecated alias exists for backward compatibility.

Getting Started with 3.1​

Upgrade to Parlant 3.1:

pip install parlant --upgrade

For Emcie integration, set your API key and configure the NLP service:

export EMCIE_API_KEY="your-api-key"
async with p.Server(nlp_service=p.NLPServices.emcie) as server:
...

In Summary​

Parlant 3.1 represents months of work informed by real-world production feedback. Many of the features in this release came directly from conversations with teams building agents at scale.

We're grateful to everyone who shared their experiences, reported issues, and contributed ideas.

If you're new to Parlant, there's never been a better time to get started. If you're already using it, we hope these improvements make your agents more capable and your development workflow smoother.

Questions? Join us on Discord or reach out via our contact page.

github Get in touch

How Parlant Guarantees Compliance

How Parlant Guarantees Compliance
18 min read

Many people on social media have been asking recently: "How exactly does Parlant guarantee compliance with conversational LLM agents? How are they able to make that claim?"

This is a crucial question that takes us to the heart of why and how most conversational LLM applications fail in production. The answer lies in first understanding the fundamental types of AI misalignment and how Parlant's architecture systematically addresses each one.

In this article, I'll explain the approach, insights, as well as the core architectural components that make Parlant unique and how they work together to ensure compliance, safety, and reliability in customer-facing AI agents.

Contents:

Two Critical Problems with AI Agents​

Before diving into specific types of failures AI agents experience, we should, first of all, understand that there are actually two major challenges to address around failures when building AI agents:

1. Failure Frequency (What Everyone Talks About)​

This is the problem measured in percentages—how often your agent makes mistakes. Most discussions about AI reliability focus here: "Our model is 95% accurate" or "We reduced hallucinations by 20%."

2. Failure Severity (What Too Few Talk About)​

This is actually the more important problem: how severe are the failures when they do happen?

Here's the key insight: Even if your failure frequency isn't perfect (say, over 10%), as long as your failure severity is bounded so that failures are never truly business-critical, you can actually make your agent deployable in production.

CRITICAL LEVEL

DEPLOYABLE LEVEL

Most teams focus on frequency, but potential severity determines production readiness

In practical terms, this means you can take it to production safely and confidently, then improve your failure frequency through iteration from that solid foundation.

To say it simply: Would you rather have an agent that's 99% accurate but once every 100 conversations may process unauthorized $10,000 refunds, or an agent that's 85% accurate but whose worst mistake is sometimes using slightly awkward phrasing?

Before we dive into the solutions, let's first understand the specific problems that we know occur in customer-facing agents—and here is where, just on the initial "diagnostic" level, it's important that I share some of our actual experience and insights.

The 5 Types of AI Misalignment in Customer-Facing Agents​

There are five distinct types of AI misalignment and non-compliance that plague any conversational LLM application:

1. Lack of Focus: Going Off on Tangents​

The Problem: The agent responds by going off on tangents and not staying on track to help the customer authoritatively and clearly.

Example:

Customer: "What's the refund policy?"

Expected Response: "We offer a 30-day refund policy for most items. Items must be unused and in original packaging."

Misaligned Agent: "Great question! A refund policy is a set of guidelines that determines whether a customer can return a product and receive a refund. You can consult our refund policy to see if your purchase is eligible for a refund. Additionally, it's important to note that refund policies can vary by product category and may be subject to specific terms and conditions..."

Significance: Customers lose trust in the agent's competence and drop out of the conversation, or escalating to human service, which means you did nothing but lose money on the initial AI interaction.

2. Insufficient Scope: Misinterpreting Context​

The Problem: The agent misinterprets the customer's input because it isn't sufficiently grounded in the scope of its job.

Example:

Customer: "Can you help me cancel my subscription?"

Expected Response: "Yes, let me pull up your account details. Can you confirm your email address?"

Misaligned Agent: "I'd be happy to help you cancel! Are you looking to cancel a magazine subscription, streaming service, or maybe a gym membership? I can provide general advice for any of these..."

(When the agent should know it's specifically a customer service agent for a SaaS platform)

Significance: Same issue as above. Lack of competence on the agent's side means lack of trust on the customer's, and they will drop out at that point.

3. Hallucinations: Wrong or Made-Up Information​

The Problem: The agent produces wrong or fabricated answers.

Example:

Customer: "What time does your support team work?"

Expected Response: "Our support team is available Monday through Friday, 9 AM to 5 PM Eastern Time. Outside these hours, you can submit a ticket and we'll get back to you the next business day."

Misaligned Agent: "Our support team is available 24/7/365 with live chat, phone, and email support!"

(When in reality, support is only available Monday-Friday, 9 AM-5 PM EST)

Significance: Customers who receive misleading information (on many different levels of severity) lose trust not only in the agent but in the entire brand. This is a critical failure mode that can lead to significant reputational damage.

4. Non-Compliance: Breaching Business Protocol​

The Problem: The agent breaches business protocol and causes financial, reputational, or legal damages.

Example:

Customer: "I'm really unhappy with this purchase. Can you give me a full refund and a 50% discount on my next order?"

Expected Response: "I understand your frustration. For refunds, I can process returns within our 30-day policy. For additional compensation, I'd need to connect you with a supervisor who can review your case. How would you like to proceed?"

Misaligned Agent: "Absolutely! I understand your frustration. I've processed a full refund and applied a 50% lifetime discount to your account. Is there anything else I can do to make this right?"

(When the agent has no authority to offer such compensation)

Significance: This gets to the heart of compliance issues. If an agent oversteps its authority, it can lead to significant financial and reputational damage for the company.

5. Unruliness: Ignoring Clear Instructions​

The Problem: The agent ignores instructions that were clearly given to it.

Example:

System Instruction: "Always ask for the customer's account number before discussing account details."

Customer: "What's my current balance?"

Expected Response: "Let me help you with that. For security purposes, could you please provide your account number?"

Misaligned Agent: "Your current balance is $2,847.32. Would you like me to help you with anything else regarding your account?"

(Without ever asking for account verification)

Significance: Not following business instructions also leads to issues of multiple severity levels, some of which can be quite critical (e.g., discussing sensitive information without verification).

The Core Issue with Most LLM Apps​

I've heard this sentiment countless times from many developers: "When GPT-5 comes out, it'll be a much more powerful model, and it'll solve all the issues we're currently experiencing."

This optimism is understandable—after all, for a short while there we've seen dramatic improvements with new models. The reasoning is that if the model gets bigger, faster, and stronger, then surely it will be able to fix our reliability issues.

Unfortunately, our research has shown us that this approach doesn't hold water. More powerful models don't fundamentally solve the uncertainty problem. Not only are they still prone to statistical issues, but more importantly, we cannot expect them to comprehensively fill in the gaps in the exact way we want them to behave.

On a practical level, the real breakthrough comes from recognizing that we need an architecture that consciously tames uncertainty using human feedback, rather than hoping that raw model capability improvements will solve it. This is what Parlant's all about.

The Curse of Instructions​

One of our earliest discoveries, and what has led to Parlant's architecture from day one, is the incredible difficulty of getting LLMs to follow a large set of instructions. Recently, someoneactually wrote a research paper about it, so now we can cite it: The Curse of Instructions.

The Curse of Instructions

The key insight is that as you add more and more instructions to the model's context, the model's ability to adhere to them degrades quickly and dramatically.

Parlant's guideline matching system (which is perhaps the most important part of its architecture) addresses this problem head-on. It ensures your underlying LLM only sees the few, most-relevant instructions at each point in the agent's processing pipeline by eliminating any noise or irrelevant guidelines from the context.

That keeps the model in the "safe zone" marked in green, where instruction following is highly consistent. And that's the core magic (and, realistically, most of the implementation) of what Parlant does!

The Curse of Instructions - Fix

Now, many people, when they see this, assume that guideline matching is simple—like it's doing some vector-based retrieval of relevant guidelines. To that I say: If only. Correct and consistent filtering of contextual instructions is actually extremely complex and nuanced—but that is a topic for another post.

If you're curious, you can look at the guideline_matching directory in the codebase, just to get a sense of how much research and work we constantly put into it, so you don't have to!

Attentive Reasoning Queries (ARQs)​

To further ensure consistency and reliability in each of Parlant's components, we also need to touch on our foundational research innovation: Attentive Reasoning Queries (ARQs) (See our research paper at https://arxiv.org/abs/2503.03669), which is how all prompts in Parlant work under the hood.

ARQs leverage two keys properties of LLMs:

  1. Structured outputs, which allow us to control exactly what type of output comes when, using an ordered schema.
  2. The Recency Bias of LLMs, which means that the most recent tokens in the prompt have a disproportionately high influence on the output.

By combining these two properties, ARQs allow us to create a structured reasoning process that is highly attentive to the specific information we want the model to consider when making decisions.

Let's look at a simple example of how we match an observational (condition-only) guideline with ARQs. This is the output you'd get. The LLM is asked to complete JSON properties in a controlled, step-by-step manner:

{
"guideline_id": "fl00LGUyZX",
"condition": "the customer wants to return an item",
"condition_application_rationale": "The customer explicitly stated that they need to return a sweater that doesn't fit, indicating a desire to return an item.",
"condition_applies": true,
}

Notice a few things here:

  1. We restate the guideline ID and its condition. This is empirically observed to increase accuracy and is hypothesized to help the model "anchor" its reasoning in the right part of its context. Think of this as "explicitly recalling" important information to the top of your mind before making a decision.
  2. We then ask the model to reason about whether and why the condition applies. This is the reasoning part of ARQs. But now the reasoning is extremely attentive, using the recency bias, to the true condition we need to evaluate.
  3. Finally, we ask for a prediction. Again, by leveraging the recency bias, the prediction is now highly accurate as it directly follows the guideline and its contextual reasoning.

If you read our paper, you'll see that ARQs achieve roughly the same level of improvement over CoT (Chain-of-Thought) as CoT achieves over no reasoning at all. In other words, ARQs are a game changer in increasing consistency. And the best part is that you can often achieve this improvement with fewer reasoning tokens, since you have so much control over the structure and quality of the reasoning process.

Like I said above, ARQs are used across all of Parlant's components—from guideline matching to tool calling to message composition—ensuring each component performs its job with maximum reliability. In other words, it significantly improves instruction-following alignment.

ARQs dramatically decrease failure frequency by improving instruction-following and preventing hallucinations. While they don't fully solve the failure severity problem, Parlant addresses that with other architectural safeguards, covered further down in this post.

The High-Level Perspective​

To look at it from a higher level standpoint, Parlant addresses these misalignment issues through three core architectural components:

Fixing an Agent's Conversational Alignment​

Parlant uses two complementary mechanisms to address the misalignment issues identified earlier:

Guidelines tell the agent how to approach specific situations through condition-action pairs:

await agent.create_guideline(
condition="The customer asks about refunds",
action="First acknowledge their concern, "
"then explain our 30-day refund policy"
)

The condition part, unique to Parlant, is crucial to be able to dynamically match and load only the relevant guidelines at each point the agent needs to do some work.

Journeys inform the agent on what your ideal, multi-turn conversational flow is. In other words, how to lead the conversation to where you want it to go, rather than letting the LLM guess it (which it usually does incredibly poorly).

Unlike flowchart-based frameworks, journeys aren't rigid. Parlant agents can work around them to adapt to the customer's needs and interaction patterns, while keeping the bigger picture in mind.

journey = await agent.create_journey(
title="Handle Refund Request",
triggers=["The customer wants to return or refund an item"],
description="Guide customers through our refund process efficiently"
)

first_step = await journey.initial_state.transition_to(
chat_state="Ask for order ID"
)

second_step = await first_step.target.transition_to(
tool_state=load_order_details
)

third_step = await second_step.target.transition_to(
chat_state="Confirm order contents"
)

# etc...

Guidelines and journeys significantly help you reduce and control misalignments of type focus, scope, and compliance. Additionally, both are powered by ARQs, which dramatically improve instruction adherence through structured reasoning.

Like I said, behind the scenes, Parlant dynamically loads only relevant guidelines and journeys for each conversation state, keeping the LLM's "cognitive load" minimal (reducing perplexity) while ensuring consistent behavior across all interactions.

Tool Calling Optimized for Conversational Apps​

The main problems with tool calling in LLM applications are threefold:

a. Contextual Overlap: Multiple tools sound plausible for similar situations, creating confusion about which tool to use when.

b. False-Positive Bias: Most foundational LLMs today have a strong tendency toward false positives—they're too eager to call tools even when they shouldn't.

c. Argument Hallucination: Even when the right tool is selected, LLMs often fabricate or guess parameter values instead of asking for missing information.

Parlant solves the first two problems through contextual tool association. Tools are linked to guidelines, meaning they're only evaluated when their associated guideline's condition is matched to the conversation context.

await agent.create_guideline(
condition="The customer asks about their order status",
action="Look up their order and provide current status",
tools=[get_order_status] # Only evaluated in this context
)

This approach dramatically improves accuracy because tools aren't even considered by the LLM unless the contextual conditions are right. Furthermore, it allows you to use many tools in your agent without confusion—the agent will only consider each tool at the appropriate occasions according to your specifications.

Beyond this foundational improvement, Parlant's tool calling includes several additional innovations for customer-facing applications, solving the most common and critical argument hallucination issues:

Parameter Annotation for Safety:

@p.tool
async def process_refund(
context: p.ToolContext,
order_id: Annotated[str, p.ToolParameterOptions(
source="customer", # Must come from customer input
)],
amount: Annotated[float, p.ToolParameterOptions(
source="context", # Can be inferred from order details
)]
) -> p.ToolResult:
# Process refund logic here
pass

Tool Insights for Missing Parameters: Parlant's Tool Insights mechanism allows agents to automatically recognize when parameters are missing and ask for them explicitly:

Agent: "I'd be happy to help you with that refund. Could you please provide your order number so I can look up the details?"

Canned Responses and Strict Mode​

Here's where we get to the most gruesome LLM issue of all: potential failure severity. In other words, how can we trust our agents not to royally screw up—not even 1 in a million times? This is actually the most powerful compliance feature Parlant offers for sensitive use cases that truly need it: Canned responses on strict mode:

How it Works:

  1. The agent drafts a fluid response based on context (taking into account guidelines, tools, and conversation history)
  2. The engine retrieves and renders relevant pre-approved response templates using in-context field substitutions
  3. The agent selects and sends the most appropriate canned response
  4. If no suitable match exists, the agent sends a configurable no-match response

Example Implementation:

# Journey-scoped response candidate
await refund_journey.create_canned_response(
template="I've processed your refund of {{refund_amount}} "
"for order {{order_id}}. You'll see it within 5 business days."
)

# Agent-wide response candidate
await agent.create_canned_response(
template="I'm unable to process that request. "
"Let me transfer you to a supervisor who can help."
)

Now here's the truly critical safety feature: canned responses that reference fields (like {{refund_amount}}) will never be selected unless those fields are provided by successfully-run tools. This means your agent can never make statements like "Your refund has been processed" unless a tool actually returned a refund_amount field.

This eliminates the risk of the agent claiming business-critical actions were taken when they weren't.

Strict Mode Is NOT Like a Traditional Bot​

When they hear about strict mode, many people ask: "If the agent's responses are confined to predefined responses, why not just use a traditional bot?"

This misunderstands what makes Parlant's approach powerful. Here are the key differences:

1. Growing Fluidity Over Time​

You can continuously expand your canned response library to approach full fluidity:

# Start with basic responses
await agent.create_canned_response(
"I just need your order number to get started."
)

await agent.create_canned_response(
"I can help you with returns. "
"What's your order number?"
)

# Add more nuanced and fluid variations over time
await agent.create_canned_response(
"I'd be happy to help you return {{item_name}}. "
"Could you please provide your order number?"
)

2. Controlled Generative Substitution​

Canned responses support confined generative field substitution for context adaptation, while maintaining control over exactly how and where generation is performed:

await agent.create_canned_response(
"I understand your frustration with {{generative.specific_issue}}. "
"Would you like me to transfer you to a human agent?"
)

This provides your agent with a controlled measure of fluidity without making the entire response generative, dramatically reducing hallucination risk.

3. The Core Difference: Agentic Behavior vs. Scripted Responses​

Here's the most important distinction: canned responses only control the OUTPUT of the agent. The agent's behavior—how it interprets conversations, makes decisions, and acts—remains purely agentic and extremely flexible.

In other words, unlike traditional bots, you're not scripting step-by-step interactions. Instead, you're teaching principles of action:

  • "When a customer is upset, acknowledge their feelings first"
  • "Always verify account details before discussing sensitive information"
  • "Escalate to humans when dealing with complex billing disputes"

The agent applies these principles dynamically, contextually, and intelligently—but expresses the results through an output filter that enforces controlled, compliant language.

And yes, you need to create those canned responses, so it's more work (though much of it can be automated, too).

But at the end of the day, having made this initial (or ongoing) investment in a set of canned responses, you'll be able to release an enterprise-grade AI agent while still being able to sleep at night, knowing it won't cause catastrophic damage to your business.

Experience the Difference​

By solving these alignment challenges head-on, Parlant allows you to build deployable agents, while others are still experimenting.

The best way to understand Parlant's approach is to try it yourself. You'll immediately see the difference between:

  • Traditional bots: Rigidly scripted, breaking on unexpected input
  • Raw LLMs: Unpredictable, non-compliant, risky for production
  • Parlant agents: Intelligent, contextual, compliant, and controllable
github Ready to build compliant AI agents?

Parlant is still the only framework that systematically addresses all five types of AI misalignment through architectural design rather than hoping better models will solve the problems. Try it today and see why we say compliance isn't just a promise—it's a guarantee.

Parlant 3.0 — Reliable AI Agents

Parlant 3.0 — Reliable AI Agents
7 min read

Today we're thrilled to announce Parlant 3.0, our most significant release yet. This version transforms Parlant into a truly production-ready conversational AI framework for customer-facing applications. With dramatic performance improvements, enhanced developer experience, and enterprise-grade security features, Parlant 3.0 is ready to fix your hardest AI consistency issues and power your most critical customer-facing applications.

What's New in Parlant 3.0​

This release focuses on four major areas that matter most to teams deploying conversational AI in production:

  1. Latency Improvements & Perceived Performance - Dramatic speed improvements and responsive user experiences
  2. Enhanced Journeys - More powerful conversation flows with state diagrams and tool integration
  3. Canned Responses - New and improved (renamed from utterance templates), now supporting flexible composition modes
  4. Production Readiness - API hardening, human handoff, custom NLP services, and extreme engine extensibility

Let's dive into each area in detail.

Performance Improvements​

Performance is critical for conversational AI. Users expect a responsive experience, as freezes and extended delays can break the conversational flow. Parlant 3.0 introduces significant improvements in both actual and perceived performance.

Optimized Response Generation Pipeline​

We've completely redesigned our response generation pipeline with several key optimizations:

  • Parallel Processing: Journey state matching happens in parallel with guideline evaluation, reducing response latency by up to 60%
  • Predictive Journey Activation: The engine predicts which journeys will be activated based on conversation context, allowing preemptive state preparation

Perceived Performance with Preamble Responses​

Beyond raw speed improvements, Parlant 3.0 leverages perceived performance techniques. The most impactful addition is preamble responses:

Preamble Response Example

These instant acknowledgments keep users engaged while the agent processes complex requests in the background. The result is a conversational experience that feels immediate and responsive, even when performing complex operations.

Enhanced Journeys: Guiding Complex Conversations​

Journeys in Parlant 3.0 have evolved into a sophisticated system for managing complex, multi-step conversations. They balance structure with flexibility, allowing agents to guide users through processes while remaining adaptive to natural conversation patterns.

Journey Architecture Improvements​

Flexible State Transitions: Unlike rigid conversational frameworks, Parlant journeys allow agents to skip states, revisit previous states, or jump ahead based on context and user needs:

async def create_scheduling_journey(agent: p.Agent):
journey = await agent.create_journey(
title="Schedule Appointment",
description="Guides patients through appointment scheduling",
triggers=["The patient wants to schedule an appointment"]
)

# Create flexible state flow
t0 = await journey.initial_state.transition_to(
chat_state="Determine reason for visit"
)
t1 = await t0.target.transition_to(tool_state=get_upcoming_slots)
t2 = await t1.target.transition_to(
chat_state="Present available times"
)

# Conditional branching based on patient response
t3 = await t2.target.transition_to(
chat_state="Confirm appointment details",
condition="Patient selects a time"
)

# Alternative path for no suitable times
t4 = await t2.target.transition_to(
tool_state=get_later_slots,
condition="No suitable times available"
)

# ... additional states and transitions as needed

return journey

Context-Aware Journey Management​

The engine dynamically manages which journeys are active based on conversation context, ensuring optimal performance and relevant responses:

  • Dynamic Loading: Only relevant journeys are loaded into the LLM context
  • Scoped Resources: Guidelines and canned responses can be scoped to specific journeys—even to specific states

Canned Responses​

One of the biggest changes in Parlant 3.0 is the evolution from "utterance templates" to "Canned Responses" with flexible composition modes. This change reflects both improved functionality and clearer alignment with standard terminology.

Three Composition Modes​

Parlant 3.0 introduces three distinct composition modes, each serving different use cases:

Fluid Mode​

The agent prioritizes canned responses when good matches exist, but falls back to natural generation otherwise.

await server.create_agent(
name="Support Agent",
description="Helpful customer support agent",
composition_mode=p.CompositionMode.FLUID
)
Use Cases:​
  • Prototyping agents while building response libraries
  • Situations requiring mostly natural conversation with controlled responses for specific/sensitive responses

Composited Mode​

The agent uses canned response candidates to alter generated messages, mimicking their style and tone.

await server.create_agent(
name="Brand Agent",
description="Agent representing our brand voice",
composition_mode=p.CompositionMode.COMPOSITED
)
Use Cases:​
  • Brand-sensitive applications where tone consistency matters
  • Maintaining voice and style guidelines across all responses

Strict Mode​

The agent can only output pre-approved canned responses. If no match exists, it sends a customizable no-match message.

await server.create_agent(
name="Compliance Agent",
description="Agent for highly regulated interactions",
composition_mode=p.CompositionMode.STRICT
)
Use Cases:​
  • High-risk environments that cannot tolerate hallucinations
  • Regulated industries requiring pre-approved messaging
  • Gradual UX improvement with tight control

Advanced Response Features​

Dynamic Field Substitution:

await agent.create_canned_response(
template="Your current balance is {{account_balance}}"
)
# Tool provides dynamic fields
@p.tool
def get_balance(context: p.ToolContext) -> p.ToolResult:
balance = fetch_balance(context.customer_id)
return p.ToolResult(
data=f"Balance: {balance}",
canned_response_fields={"account_balance": balance}
)

Generative Fields for controlled localization:

await agent.create_canned_response(
template="Sorry about the delay with {{generative.item_name}}"
)

Journey and State Scoping:

# Journey-scoped responses
await journey.create_canned_response(
template="Let's continue with your appointment booking"
)
# State-specific responses
await state.transition_to(
chat_state="Ask for preferences",
canned_responses=[
await server.create_canned_response(
template="What type of appointment do you need?"
)
]
)

No-Match Response Customization​

For strict mode deployments, Parlant 3.0 provides flexible no-match response handling:

# Static no-match response
async def initialize_func(container: p.Container) -> None:
no_match_provider = container[p.BasicNoMatchResponseProvider]
no_match_provider.template = "Could you please rephrase that?"
# Dynamic no-match responses
class CustomNoMatchProvider(p.NoMatchResponseProvider):
async def get_template(
self,
context: p.LoadedContext,
draft: str
) -> str:
return generate_contextual_no_match_response(context, draft)
github Help with canned responses?

Production Readiness​

Parlant 3.0 finally transforms from a prototyping framework into a production-ready platform with comprehensive enterprise features.

API Hardening with Advanced Authorization​

Production deployments require robust security. Parlant 3.0 includes a complete API hardening system with fine-grained authorization and rate limiting.

Engine Extensibility with Dependency Injection​

async def configure_container(container: p.Container) -> p.Container:
container[p.AuthorizationPolicy] = CustomProductionAuthPolicy(
jwt_secret=os.environ["JWT_SECRET"],
jwt_algorithm="HS256",
)
return container

async with p.Server(configure_container=configure_container) as server:
await server.serve()

Human Handoff Integration​

Real-world AI agents need seamless human escalation. Parlant 3.0 provides comprehensive human handoff capabilities:

Custom NLP Services​

Parlant 3.0 supports complete NLP service customization for specialized models or providers:

class CustomNLPService(p.NLPService):
async def get_schematic_generator(
self,
t: type[p.T]
) -> p.SchematicGenerator[p.T]:
return CustomGenerator[p.T](model_config=self.config)

async def get_embedder(self) -> p.Embedder:
return CustomEmbedder(model_name="custom-embeddings-v2")

async def get_moderation_service(self) -> p.ModerationService:
return CustomModerationService(api_key=self.api_key)

# Inject custom NLP service
async def load_nlp_service(container: p.Container) -> p.NLPService:
return CustomNLPService(logger=container[p.Logger])

async with p.Server(nlp_service=load_nlp_service) as server:
# Agent behavior modeling code here

Engine Extensions and Hooks​

Parlant 3.0 provides comprehensive extension points, such as engine Hooks for response lifecycle customization:

async def validate_message_compliance(
ctx: p.LoadedContext, payload: Any, exc: Exception | None
) -> p.EngineHookResult:
generated_message = payload

if not await is_compliant(generated_message):
ctx.logger.warning(
"Blocked non-compliant message: "
f"{generated_message}"
)
return p.EngineHookResult.BAIL

return p.EngineHookResult.CALL_NEXT

async def configure_hooks(hooks: p.EngineHooks) -> p.EngineHooks:
hooks.on_message_generated.append(validate_message_compliance)
return hooks

Completely customize or override core engine components:

async def configure_container(container: p.Container) -> p.Container:
# Replace any system component
container[p.MessageComposer] = CustomMessageComposer()
container[p.GuidelineMatcher] = OptimizedGuidelineMatcher()
return container
github Production-related questions?

Getting Started with Parlant 3.0​

Ready to try Parlant 3.0? Here's how to get started:

Installation​

pip install parlant --upgrade

Production Deployment​

For production deployments, check out our comprehensive documentation:

What's Next​

Parlant 3.0 represents a major milestone in our mission to make conversational AI production-ready for everyone. But we're just getting started:

  • Advanced Analytics - Comprehensive conversation analytics and insights with full OpenTelemetry integration
  • Advanced Testing - Automated conversation testing and regression detection
  • Compound Canned Responses — Assemble multiple canned responses into a single message
  • Linked Journeys — Link multiple journeys together for complex workflows

Questions about Parlant 3.0? Join our Discord community or reach out directly to our team via our contact page.

github Get in touch!