Principles Library / Surface agent uncertainty

Surface agent uncertainty

Agents Must Surface Uncertainty Explicitly

Principle intent

Agent systems must be designed to surface uncertainty, ambiguity, and boundary conditions rather than proceeding with false confidence.

Read the full intent

An agent operating outside its reliable range must flag it rather than execute as if the range were sufficient. The human can only intervene at the right moment if the system signals when intervention is needed.

When it holds: what it does for you

Humans step in early

People intervene when an agent's judgment falls short, before the gap compounds.

Trust fits the situation

Teams know where an agent's output holds up and where it needs a human look.

Pipelines keep their intent

Uncertainty travels with the work, so later stages don't build on false certainty.

When it is ignored: from early signs to a lasting condition

Early warning signs · Visible now, cheap to fix

What happens over time · Accumulates, costly to reverse

The condition it becomes

Oversight Erosion · Primary

The review process still runs. It stopped meaning anything.

How this principle feeds it

oversight still runs but never gets a trigger, so it looks rigorous while missing the cases that matter.

Quality Fragility · Secondary

Customers find defects before the team does.

How this principle feeds it

agent failures with no warning are found after they reach users, when they are expensive to fix.

Implementation Drift · Primary

The agents are building. Nobody is sure it still matches the plan.

How this principle feeds it

locally correct outputs diverging from plan through unacknowledged substitutions is the condition's definition.

Attribution Failure · Secondary

We know what happened. Nobody can explain why.

How this principle feeds it

with no confidence recorded at each stage, nobody can say which stage's guess caused the drift.

Where is your system on this path?

Diagnose

Diagnose

AI signs are on by default. Use the AI switch to exclude them. This principle applies once agents produce or act on work in your delivery.

1. Which of these early warning signs are you seeing?

  • Agent outputs look plausible but keep missing what the situation needed.
  • Agents sound equally confident whether or not the situation fits their design.
  • Agents have no way to pause or flag an ambiguous or novel situation.
  • Nobody can tell from an output whether the agent was in range or extrapolating.
  • Post-incident reviews find the agent hit an out-of-range condition and kept going.
  • Agent outputs get the same review depth in every type of situation.
  • Each pipeline stage hands on its output with no record of how sure it was.
  • Reviewers approve agent uncertainty flags without reading them because there are so many.

2. Over the last few months, have you noticed…

For each consequence: Not seen / Sometimes / Persistently / Not sure.

  • Failures arrive with no warning because the agent never raised a signal.
  • The team leans on agent outputs where the agent had no basis for confidence.
  • Governance hears about out-of-range cases only after they have failed.
  • Trust follows average agent performance because nothing marks the situations that matter most.
  • Pipeline outputs look right at each stage but drift from the original intent.
  • Uncertainty flags turn into noise, so real out-of-range cases pass unchecked.

3. The condition it becomes

Based on your answers, recognize at most two conditions: No / Partly / Yes, that is us.

Oversight Erosion: The review process still runs. It stopped meaning anything.

Quality Fragility: Customers find defects before the team does.

Implementation Drift: The agents are building. Nobody is sure it still matches the plan.

Attribution Failure: We know what happened. Nobody can explain why.

Take your Diagnosis to the next level by invoking AI Diagnostic Coaching tool.

What else have you observed? Optional. Shared only if you choose to start coaching.

Start a Diagnosis session: review and edit a draft containing your note, conditions you recognized in Step 3 (including whether they match partly), warning signs and consequence ratings before sending it. No scores or principle files are sent.

See what practices will help you

Act

Act

Showing all 5 practices. Answer the Diagnose questions and the most relevant come first.

Define each agent's reliable range first

Define each agent's reliable operating range before deployment. Build explicit detection for inputs or situations that fall outside it.

Start by asking your team: What situations would give this agent real uncertainty, and what happens in them today?

Avoid · You’ll know it’s working when

Avoid

  • Assuming an agent that does well on familiar inputs will hold back on unfamiliar ones

You’ll know it’s working when

The team sees an agent's limits in its signals before anything fails.

Route uncertain work to human review

Set uncertainty thresholds that send work to human review instead of on to the next agent.

Start by asking your team: How does work get escalated to human review when an agent hits a situation it can't handle reliably?

Avoid · You’ll know it’s working when

Avoid

  • Treating uncertainty signaling as error handling. Errors are detectable failures, uncertainty is when the system can't judge its own output.

You’ll know it’s working when

Agents pause on ambiguous or novel situations and a person picks them up.

Carry uncertainty through every pipeline stage

In multi-agent pipelines, make uncertainty signals travel through every stage. Don't let any stage absorb them and pass on a clean-looking output.

Start by asking your team: Where in your pipeline does an agent's uncertainty get passed on as if it were certainty?

Avoid · You’ll know it’s working when

Avoid

    You’ll know it’s working when

    Each stage can see how sure the previous agent was about its input.

    Review how often agents flag uncertainty

    Review uncertainty signal frequency regularly. A falling rate may mean detection is failing, so check that before you credit the agent with getting better. A rate too high for reviewers to read means the threshold needs resetting.

    Start by asking your team: How does this agent signal when it's working outside its reliable range?

    Avoid · You’ll know it’s working when

    Avoid

    • Setting uncertainty thresholds once at deployment and never revisiting them as context changes
    • Believing more capable agents need less uncertainty management

    You’ll know it’s working when

    The team can say which situations an agent handles well and which it doesn't.

    Surface novel situations instead of guessing

    When an agent meets a novel situation, make surfacing it the default. Don't let the agent substitute its own answer and carry on.

    Start by asking your team: Can someone looking at this agent's output tell whether it was confident or extrapolating?

    Avoid · You’ll know it’s working when

    Avoid

    • Treating confidence scores as the same thing as uncertainty signaling

    You’ll know it’s working when

    Outputs show whether the agent was in range, so people know when to look closer.