Oversight Erosion · Primary
The review process still runs. It stopped meaning anything.
How this principle feeds it
oversight still runs but never gets a trigger, so it looks rigorous while missing the cases that matter.
Principles Library / Surface agent uncertainty
Agents Must Surface Uncertainty Explicitly
AI signs are on by default. Use the AI switch to exclude them. This principle applies once agents produce or act on work in your delivery.
For each consequence: Not seen / Sometimes / Persistently / Not sure.
Based on your answers, recognize at most two conditions: No / Partly / Yes, that is us.
Oversight Erosion: The review process still runs. It stopped meaning anything.
Quality Fragility: Customers find defects before the team does.
Implementation Drift: The agents are building. Nobody is sure it still matches the plan.
Attribution Failure: We know what happened. Nobody can explain why.
What else have you observed? Optional. Shared only if you choose to start coaching.
Start a Diagnosis session: review and edit a draft containing your note, conditions you recognized in Step 3 (including whether they match partly), warning signs and consequence ratings before sending it. No scores or principle files are sent.
Agent systems must be designed to surface uncertainty, ambiguity, and boundary conditions rather than proceeding with false confidence.
An agent operating outside its reliable range must flag it rather than execute as if the range were sufficient. The human can only intervene at the right moment if the system signals when intervention is needed.
People intervene when an agent's judgment falls short, before the gap compounds.
Teams know where an agent's output holds up and where it needs a human look.
Uncertainty travels with the work, so later stages don't build on false certainty.
The review process still runs. It stopped meaning anything.
oversight still runs but never gets a trigger, so it looks rigorous while missing the cases that matter.
Customers find defects before the team does.
agent failures with no warning are found after they reach users, when they are expensive to fix.
The agents are building. Nobody is sure it still matches the plan.
locally correct outputs diverging from plan through unacknowledged substitutions is the condition's definition.
We know what happened. Nobody can explain why.
with no confidence recorded at each stage, nobody can say which stage's guess caused the drift.
Showing all 5 practices. Answer the Diagnose questions and the most relevant come first.
Define each agent's reliable operating range before deployment. Build explicit detection for inputs or situations that fall outside it.
Start by asking your team: What situations would give this agent real uncertainty, and what happens in them today?
The team sees an agent's limits in its signals before anything fails.
Set uncertainty thresholds that send work to human review instead of on to the next agent.
Start by asking your team: How does work get escalated to human review when an agent hits a situation it can't handle reliably?
Agents pause on ambiguous or novel situations and a person picks them up.
In multi-agent pipelines, make uncertainty signals travel through every stage. Don't let any stage absorb them and pass on a clean-looking output.
Start by asking your team: Where in your pipeline does an agent's uncertainty get passed on as if it were certainty?
Each stage can see how sure the previous agent was about its input.
Review uncertainty signal frequency regularly. A falling rate may mean detection is failing, so check that before you credit the agent with getting better. A rate too high for reviewers to read means the threshold needs resetting.
Start by asking your team: How does this agent signal when it's working outside its reliable range?
The team can say which situations an agent handles well and which it doesn't.
When an agent meets a novel situation, make surfacing it the default. Don't let the agent substitute its own answer and carry on.
Start by asking your team: Can someone looking at this agent's output tell whether it was confident or extrapolating?
Outputs show whether the agent was in range, so people know when to look closer.