Moderating messages

Screening what comes in and goes out

Moderation is a safety screen on both sides of a conversation. It checks what a customer sends before the agent reads it, and what the agent is about to say before the customer sees it. A moderation model sorts the text into categories such as harassment. If the text is flagged, Parlant keeps it away from the model, or makes the agent write its reply again.

What you'll learn

• How moderation differs from message review
• The three moderation settings and when each fits
• What happens to a flagged customer message and a flagged agent reply
• How to write a moderation model of your own

Talk this page through with your coding agent

Moderation provides separate checks for incoming and outgoing message text. Each check asks a moderation model (a classifier that sorts text into categories such as harassment) whether the text is acceptable, and Parlant acts on the verdict.

Moderation vs. message review

Moderation is about a moderation model's categories, not about your instructions. Message review is the check that asks whether a reply complies with your agent's instructions.

Two settings control moderation, and they are separate on purpose. Whether an agent is moderated is a property of the agent. What model does the checking is deployment configuration. It lives in the same models mapping that selects your other models.

This split lets your agent declare that it must be moderated without knowing which provider will do the checking where it is deployed. You decide the policy once; the deployment decides the mechanics.

Turning it on

There are two steps. First, configure the moderation models (shown in the next section). Then, on the agent itself, choose whether it requires them.

Note that outgoing moderation needs block output, the mode where Parlant finishes the whole reply before delivering it. That way Parlant has the complete reply to check before delivery:

import parlant.sdk as p

agent = await server.create_agent(
    id="support",
    name="Support Agent",
    instructions="...",
    moderation=p.Moderation.ON,
    output_mode=p.OutputMode.BLOCK,
)

The moderation setting takes one of three values:

Value	Meaning
AUTO (default)	Each checkpoint runs if a model is configured for it, and is skipped silently if not. Configuring only the customer side gives input checking and nothing else.
ON	Both checkpoints must have a model. Validated at startup and whenever the agent is created or updated, so a missing model fails fast instead of silently skipping a layer you declared mandatory.
OFF	No moderation for this agent, whatever is configured.

With ON, creating this agent fails until both moderation routes are configured. If you would rather a missing route leave that checkpoint disabled, use AUTO.

Note that moderation is not configurable per message. Callers of the API cannot opt a message in or out. Once you have decided your agent is moderated, it is moderated.

Choosing the models

Once you have told the agent whether it is moderated, you need to tell Parlant what does the moderating.

A model route is a named kind of work in the models mapping you pass to the server, so that each kind of work can run on its own model. The two moderation checkpoints are ordinary routes, and they resolve independently:

async with p.Server(
    models={
        "*": "openai",
        "moderation.customer": p.ModelConfig("openai:omni-moderation-latest"),   # incoming messages
        "moderation.agent": p.ModelConfig("openai:omni-moderation-latest"),      # outgoing messages
    },
) as server:
    ...

Moderation is strictly opt-in. No provider registers a default moderation model, so a provider-wide fallback such as models={"*": "openai"} never turns it on. Only the explicit routes do.

Note that among the built-in providers, only OpenAI offers a moderation model. If you deploy on another provider, you route the checkpoints to a provider that has one, or to a custom one, or you leave them unconfigured.

Waiving one side

Suppose you want to require incoming moderation with ON, but you have deliberately decided not to moderate your agent's replies. Route the waived checkpoint to the built-in null provider, which never flags. The route is then genuinely configured, and your configuration shows that the waiver is intentional:

models={
    "moderation.customer": p.ModelConfig("openai:omni-moderation-latest"),
    "moderation.agent": p.ModelConfig("null"),
}

What happens to a flagged message

Once both checkpoints are configured, here is what Parlant does when one of them flags a message.

Customer messages

Parlant checks customer messages before they enter the session. When a message is flagged, Parlant stores it with flagged: true and its moderation tags, such as harassment or jailbreak. From there it treats the message as censored everywhere.

What does censored mean in practice? Rule matching, response generation, and the choice of canned responses (approved example messages the agent imitates) all see a censor notice carrying the reasons, instead of the text itself.

Your agent does not have to go silent about it, either. A policy (one of the agent's plain-language rules) can condition on the censoring: "when the customer's last message is censored, ...". So the agent can explain why it did not address the message.

Note that attachments are not covered. Captions and filenames reach the model unmoderated, as Input attachments notes.

Agent messages

Parlant checks agent messages after the response is finished, in block output mode only. A flagged response is regenerated, not shown. Parlant feeds the moderation reason (never the flagged text) back to the agent as guidance, and the agent tries again. These retries share the same retry budget as message review.

If the reply is still flagged after those attempts, the turn gives up. Parlant then writes one final message explaining that it could not respond. Only if that give-up message is also flagged does Parlant emit a fixed fallback string instead.

Whenever a message is blocked, Parlant records the verdict with its tags on the emitted event's internal metadata, for your frontend and for audit.

The last-resort fallback text is a setting on the responder, the component that writes your agent's replies. Your coding agent can change it through initialize_container, the startup callback that receives the server's components:

from parlant.core.engines.compass.responder.responder import Responder


async def initialize_container(container: p.Container) -> None:
    container[Responder].moderation_fallback = (
        "Sorry, I can't respond to that right now. How else can I help?"
    )

Learn more about the retry budget

Handling semantic failures covers how many retries a turn gets and what giving up means.

Why block mode only? In stream mode, the text is already on the wire by the time the message is complete, so the streaming loop cannot gate it. Parlant warns at startup and on agent creation when an agent has agent-side moderation configured but a non-block output mode. If that is intentional, waive the checkpoint with null.

tip

Streams and blocks covers the two output modes.

Note that the agent-side check fails open. If the moderation model errors, Parlant emits the original message and logs a warning. Moderation is a safety layer on top of generation, not a delivery gate.

info

For a check that judges a reply against your own instructions, use Message reviews.

Writing your own moderation model

When no built-in provider fits, or when a classifier of your own should decide, your coding agent can write the moderation model.

A moderation model implements ModerationService, a class with two methods: moderate_customer for incoming messages and moderate_agent for outgoing ones. Each receives a context object and returns a ModerationCheck, the verdict Parlant acts on. Its flagged field says whether to block, and tags names the reasons.

from parlant.core.nlp.moderation import (
    CustomerModerationContext,
    ModerationCheck,
    ModerationService,
)


class MyGuard(ModerationService):
    async def moderate_customer(self, context: CustomerModerationContext) -> ModerationCheck:
        verdict = await my_classifier(context.message)          # your own classifier
        return ModerationCheck(flagged=verdict.bad, tags=verdict.tags)   # flagged: block it; tags: why

Both context objects carry the session. The agent-side context also carries the turn's engine context. So a context-aware model can weigh the whole conversation, the matched policies, and tool results, rather than the message text alone.

Note that customer-side moderation runs at the API boundary, before a turn starts. There, the engine context is None, and the session history is what you have to work with.

moderate_agent defaults to the same check as moderate_customer. Override it when the two sides should behave differently.