Skip to main content

Ensuring Coherence

Ensuring Coherence

Parlant helps you ensure that your guidelines work together cleanly and are followed reliably. Let's dive into how it validates guideline consistency and provides visibility into your agent's decision-making process.

note

To learn about protecting agents from inappropriate or otherwise problematic user inputs, check out the section on Guardrails & Moderation.

In this section, you'll learn:

  1. What guideline coherence means and why it matters
  2. Where and how coherence checks happen
  3. How Conformal Guidance makes guidelines more reliable
  4. How to use coherence artifacts to troubleshoot and improve behavior

Understanding Coherence

Coherence in Parlant works at two key points in the system, each serving a distinct purpose.

Guideline Management

First, at the guideline management level, Parlant prevents conflicting or contradictory guidelines from being added to your system—think of it like dependency resolution in package management, where you wouldn't want conflicting package versions or incompatible dependencies breaking your system.

When you add new guidelines, Parlant automatically checks them against your existing set to catch potential conflicts early.

This early detection is important because conflicting guidelines can lead to unpredictable agent behavior—which, incidentally, can also be hard to understand and troubleshoot.

In other words, instead of discovering conflicts through unexpected responses in live conversations, Parlant lets you catch and address many of them during the design phase—ensuring your agent's behavior aligns more closely with your intentions.

Message Generation

Second, during message generation, Parlant ensures guidelines are followed consistently in real-time conversations through our Conformal Guidance technique.

Rather than simply adding guidelines to prompts and hoping for the best, Parlant employes techniques such as echoing and self-critique to ensure the LLM actually follows the guidelines.

This approach (research paper on the way) leads to more consistent and natural responses while maximizing guideline adherence.

Together, these coherence mechanisms create a system that's both well-defined and reliably executed.

The Guidline Evaluation Process

To ensure your agent stays consistent, Parlant requires you to go through an evaluation step before adding any new guidelines to your agent.

Think of it like a code review, but for your agent's behavioral instructions. This process protects your agent's internal consistency and helps maintain a coherent worldview.

When Parlant identifies potential conflicts, it provides specific feedback about the nature of the contradiction. This allows you to either:

  1. Modify your new guidelines to align with existing ones
  2. Update existing guidelines to accommodate your new requirements
  3. Refine both to create a coherent set of instructions

This approach ensures your agent maintains consistent, predictable behavior as your set of guidelines grows and evolves.

Evaluation Types

Currently, two types of guideline evaluations are supported in Parlant:

  1. Coherence Checks
  2. Connection Propositions.

When creating evaluations, you can control what checks, if any, should be performed.

from parlant.client import (
GuidelineContent,
GuidelinePayload,
ParlantClient,
Payload,
)

client = ParlantClient(base_url=SERVER_ADDRESS)

# Start evaluating the guideline's impact
evaluation = client.evaluations.create(
agent_id=AGENT_ID,
payloads=[
Payload(
kind="guideline",
guideline=GuidelinePayload(
content=GuidelineContent(
condition=CONDITION,
action=ACTION,
),
operation="add",
coherence_check=CHECK_COHERENCE,
connection_proposition=PROPOSE_CONNECTIONS,
)
)
],
)

Coherence Check

This is to ensure that agents aren't confused by self-contradicting guidelines. Here, there may be very obvious contradictions, such as:

When X, Then Y and When X, Then not(Y).

But there may also be subtler ones (which Parlant also helps you catch), such as:

When X, Then Y and When A, then not(B)

such that A is a sub-case of X, and B is a sub-case of Y. For example:

When the customer has items in the cart, Then quickly lead them to express checkout

and:

When the customer has an item in the cart that costs more than $300, Then engage them in discussion on its unique qualities

Connection Proposition

To understand connection proposition, we first need to understand how Parlant chooses which guidelines activate for an agent when it's about to say something to the customer.

Basically, Parlant examines the session at its current state, and asks questions about it: "Is this guideline relevant now?", "Is that guideline relevant now?". To do this, it primarily tests the guidelines' conditions.

This would have worked perfectly by itself, until we considered having two guidelines following the form:

When X, Then Y and When Y, Then Z

Now imagine a situation where, by looking at a session, we infer that X does in fact apply, but not Y. With the naive logic above, Parlant would have only fed the agent with the guideline to do Y.

But when we step back and analyze this case, we know that the agent is just about to do Y, which means that, according to the guidelines we have installed, Z should also apply.

To overcome this challenge, Parlant semantically evaluates causal connections between the guideline you're about to add and the agent's existing guidelines, and indexes them behind the scenes in the form of a graph—a graph of causal connections.

Explaining and Troubleshooting Agent Behavior

Message generation in Parlant goes through quite a lot of quality checks.

These checks produce artifacts that can help explain how the agent interpreted elements such as guidelines, glossary terms, and tools.

When you run into issues, you can inspect these artifacts to better understand why the agent responded the way it did, and whether it correctly interpreted your intentions.

$ parlant session inspect \
--session-id SESSION_ID \
--event-id EVENT_ID

Over time, this feedback loop helps you build more precise and effective sets of guidelines.