AI Guides › Prompting
The Confidence Check: Making Every Answer State Its Own Uncertainty
By Nigel Guy · 2 min read
Answers tend to arrive in the same even, assured tone whether they're
well-supported or a plausible-sounding guess. Nothing in the delivery
distinguishes "this is well-documented" from "this is my best reconstruction
of something I'm not fully sure about" — both come out sounding equally
settled. If you only ever ask for the answer, you never get told which kind
you received.
The rule: ask the answer to grade its own confidence and name what would
change it — a flat, undifferentiated answer hides exactly the information
you need to decide how much to trust it.
The mechanism
- Ask directly: "how confident are you in this, and why?" appended to
the original question. This is a separate ask, not an assumption that
confidence will show up unprompted.
- Ask for the confidence to be specific, not a single label. "High
confidence on the general approach, lower confidence on the exact figure"
is more useful than a single word like "confident" — different parts of
an answer usually deserve different levels of trust.
- Ask what would change the answer. "What would make you revise this?"
surfaces the load-bearing assumption. If nothing would change it, that's
either genuine certainty or a sign the question wasn't examined closely —
worth a second look either way.
- Treat "I'm not certain" as a useful answer, not a failure. A flagged
uncertainty is more useful than a confident guess dressed up the same way
as a verified fact — it tells you where to check before you rely on it.
What to skip
Skip running a full confidence check on low-stakes questions where being
wrong costs nothing — it's built for anything you're about to act on. And
skip treating a stated "high confidence" as verification; it's a
self-assessment, not an audit, and self-assessments can be wrong in either
direction.
Guardrails
- A stated confidence level is not a probability in any calibrated sense.
Treat it as a flag for where to look closer, not a number to trust
literally.
- This surfaces uncertainty the answer is aware of. It won't surface a wrong
premise the question itself was built on, or a gap the answer never
considered.
- For anything time-sensitive — current prices, current rules, current
versions of anything — a stated high confidence still needs checking
against a current source; confidence about a fast-moving fact ages badly
regardless of how it was expressed.
All 751 AI guides · JulieMango plans from £17/mo