Lab NotebookEntry 02AI that decides

The double threshold: answer, review or hand off to a human

When can an AI answer without anyone checking? The same decision theory as Entry 01, now with a third way out: "I'm not sure, get someone".

With the same AI and the same customers, four ways of organizing customer service lead to costs ranging from €4.6k to €78k per thousand inquiries. The difference isn't in the model. It's in two numbers almost nobody calculates.

The AI can answer on its own where being wrong is cheap. Viewing times, yes. Rent amounts and contract clauses, only with someone looking.

Origin
An AI rental assistant that qualified prospects against real inventory (client not identified).
Data
10,000 synthetic inquiries of 4 types, fixed seed 20261021. No data from the real project.
Two voices
In the notebook, for the people who build the model. On a napkin, for the people who hold the budget
Benches
1 · Who answers? (strategy, confidence, costs, reviewer and capacity)

Same model, four bills

Take an AI that answers customer inquiries for a rental company. It does well on viewing-time questions, somewhat worse on availability, and gets price and contract questions wrong often. Now organize customer service in four ways, without changing the model:

The last one is 29% cheaper than reviewing everything and uses 38% less staff time (1,992 versus 3,201 minutes per thousand inquiries). And it still lets errors through: 55 per thousand inquiries. That isn't a defect. It's the point: 95% of those errors are about viewing times, where each one costs a repeat contact.

And almost all of the gain over reviewing everything, 92%, comes from the simplest part: splitting by type of question. Looking at each answer's confidence adds the rest.

In Entry 01 the threshold decided between calling and not calling. Here the threshold becomes two, because there's a third way out: the AI can say "I'm not sure, get someone".

Where this came from

In 2023 I built a rental assistant for a real estate company in North America. The table of available units became a knowledge base; the assistant talked to prospects, qualified them by occupancy, price range and number of bedrooms, recommended matching units and passed only the leads that made sense to the agent. An operator marked units as rented, so the knowledge base followed real inventory.

The design that worked was "the AI qualifies, the person closes". This entry is the question it left behind: can the split between what the AI does alone and what stays with a person become a calculation?

Three paths, three costs

For each inquiry, the AI produces an answer and a confidence p, the probability that the answer is right. There are three paths:

The model is simple on purpose, and its assumptions are in plain sight: the person doesn't make mistakes; the review catches every error (the "Where this breaks" section shows what happens when it doesn't) and never rejects a correct draft; the AI has zero marginal cost; and E is the same for every question of a given type. With that, the expected costs are:

CAI = (1 − p)·E   ·   Crev = R + (1 − p)·H   ·   Chum = H
(1)

Comparing them in pairs gives the two thresholds. The AI answers on its own when CAI < Crev, and the inquiry goes straight to a person when Chum < Crev:

on its own if p > 1 − R / (E − H)   ·   person if p < R / H
(2)

In between, a draft with review. With a review at €4 (2 minutes), a human answer at €10 (5 minutes) and a price error at €200 (a lost lead), the upper threshold sits at 97.9% and the lower one at 40%. For a contract error at €1,500, the upper threshold moves to 99.7%: in practice, the AI never answers contract questions on its own.

That middle band only exists when the upper threshold sits above the lower one, which requires E > H² / (H − R), with R < H; with these values of R and H, E above €16.67. Otherwise reviewing never pays off, and there's a single threshold between the AI and the person: the AI answers if p > 1 − H / E. When in doubt, the rule that always holds is equation (1): pick the path with the lowest expected cost.

The extreme case is surprising: when an error costs less than redoing the answer (E ≤ H), reviewing never pays off. Getting a viewing time wrong costs €8, one extra contact; redoing it costs €10. The math tells the AI to answer every viewing-time question on its own, even the low-confidence ones.

Automation isn't an on/off switch. It's a threshold per type of question, and what sets the threshold is how much it costs to be wrong there.

The idea has a pedigree. In 1957, C. K. Chow designed an optimal character-recognition system with a reject option for ambiguous cases, and in 1970 he described the trade-off between error and rejection: reject when the probability of the best class falls below a threshold. The double threshold extends that, with three actions and a person who has a cost of their own. In machine learning the topic came back as selective classification (trading coverage for risk) and as learning to defer, where the model learns when to pass the case to a human expert. What's new with AI agents is the size of the bill.

A physicist's detour

The blip on the radar screen

The ROC curve has its roots in Second World War radar: someone had to decide whether a blip on the screen was an aircraft or noise, and each kind of error had a very different price. The name stuck: receiver operating characteristic. The theory was formalized in the early 1950s as the theory of signal detectability (Peterson and Birdsall, 1953), and it's the same theory that decides, today, whether an AI answer is good enough to go out with nobody looking.

The confidence that counts is the measured one

Everything above depends on p being a real probability: among answers with confidence 0.9, nine in ten are right. That's called calibration, and it doesn't come for free.

Modern neural classifiers tend to be poorly calibrated, usually overconfident (Guo et al., 2017, in computer vision). Large language models can estimate reasonably well whether their own answer is right when the question comes in the right format, but the authors themselves report that RLHF fine-tuning hurts calibration (Kadavath et al., 2022). And when models state their confidence in words, they tend to overstate it and cluster at high values, which separates right from wrong answers poorly (Xiong et al., 2024). In practice: the "confidence" the agent reports is on the wrong scale and often in the wrong order too.

The simulation shows the damage in a generous scenario. The "measured" confidence is perfect calibration, a ceiling no company reaches. The "reported" one keeps the order and only shifts the scale upward, the best case for reported confidence. With the same thresholds, the double threshold now automates 61.5% of inquiries instead of 44.6%, costs €5,524 instead of €4,576 per thousand (21% more) and lets 82 errors per thousand through instead of 55. Almost all of the extra loss comes from availability and price: inquiries that should have been reviewed go out on their own, and price inquiries that should have gone straight to a person land in review. With a confidence that also ranks badly, the gap tends to be larger.

The confidence that counts is the one you measure: real inquiries already answered, with the right answer known, grouped by question type and by confidence band. A few hundred per type are enough for the lower threshold, near 40%. The upper threshold is another story: to claim that a band errs less than 0.3% of the time (the contract threshold), you need about a thousand answers in a row with no errors, by the rule of three (zero errors in n cases gives an upper bound of about 3/n, at 95% confidence). With little data, give the upper threshold a margin: use the lower bound of accuracy, not the average. It's the same lesson as Entry 01, on a new machine: the cost rule works on probabilities, not on scores.

The bench below runs the simulation's 10,000 inquiries. The numbers quoted in the text are those of the default settings; move the controls to see where each strategy breaks.

Bench · Who answers?

10,000 synthetic inquiries · seed 20261021
Strategy
Confidence used for the threshold
€4
€10
× 1
100%
Team available (minutes per 1,000 inquiries; €2 per minute)
—
AI on its own—
With review—
Straight to a person—
Errors to customers / 1,000—
Team minutes / 1,000—
Cost / 1,000 inquiries—
TypeError costsThresholds (upper · lower)AIReviewPersonErrors / 1,000

—

This bench uses made-up costs and accuracy. Want the same math for your customer service? Vettore Data runs the diagnostic: it measures your AI's real accuracy by question type and sets the thresholds with your costs. See the diagnostic →

What matters to whoever pays

Where the money is

Three levers, in order of size:

How to use this in practice

The double threshold isn't specific to customer service. It applies to any workflow where a model produces an output, someone can check it, and errors have a price. A recipe that works:

1. Log everything from day one

For each inquiry: the type (from a classifier or from rules), the AI's answer, the confidence (the reported one, if that's all there is), the path taken and, when there is one, the person's correction. Without this log, none of the following steps is possible.

2. Run in shadow mode before letting go

For the first few weeks, every answer goes through review, and the reviewer marks it "right" or "wrong". That produces the labeled set for free, as a by-product of work that would be done anyway.

3. Calibrate per type

With the labeled set, replace the reported confidence with the observed accuracy in each band (isotonic regression or Platt scaling, or simply a table per band when volume is small). One calibration per question type, not a global one. In the simulation, the distortion of the reported confidence is a simple shift that Platt scaling fixes completely; in real life, recalibration fixes the scale but not a bad ranking.

4. Put the thresholds in configuration, not in code

Each type's two thresholds come from equation (2), with the R, H and E the business wrote down. They live in a configuration file with an owner and a date, and they change when the costs change.

5. Audit both ends

A small random sample of automatic answers reviewed after sending, to check that accuracy is still where it was. And a sample of reviews checked by a second person, to measure how much the reviewer actually catches (the next section shows why that matters).

Some places where the same calculation applies unchanged: document reading (fields read with high confidence go straight in, intermediate confidence goes to a check, illegible documents go to manual entry), content moderation, technical-support ticket triage, expense-reimbursement approval and product classification in a catalog. In all of them E differs by category, and that's what makes the threshold worth money.

Mandatory section

Where this breaks

To reproduce

The core of the decision fits in a few lines. The full script, with the synthetic population, every strategy and the sensitivity analyses, runs with Python's standard library alone.

def route(p, E, R=4.0, H=10.0):
    """Lowest-expected-cost path for an answer with calibrated confidence p."""
    cost = {
        "ai": (1 - p) * E,          # the error reaches the customer
        "review": R + (1 - p) * H,  # always reviewed; rewritten if the draft was wrong
        "human": H,                 # answered from scratch
    }
    return min(cost, key=cost.get)

print(route(0.95, E=200))   # review
print(route(0.99, E=200))   # ai
print(route(0.30, E=8))     # ai: being wrong costs less than redoing

References

Decision theory with rejection

  1. Chow, C. K. (1957). An Optimum Character Recognition System Using Decision Functions. IRE Transactions on Electronic Computers, 247–254.the reject option for ambiguous cases
  2. Chow, C. K. (1970). On Optimum Recognition Error and Reject Tradeoff. IEEE Transactions on Information Theory, 16(1), 41–46.the error–reject trade-off
  3. Geifman, Y. & El-Yaniv, R. (2017). Selective Classification for Deep Neural Networks. NIPS 2017.risk versus coverage with a confidence threshold
  4. Mozannar, H. & Sontag, D. (2020). Consistent Estimators for Learning to Defer to an Expert. ICML, PMLR 119, 7076–7087.the model that learns when to hand the case to a person
  5. Elkan, C. (2001). The Foundations of Cost-Sensitive Learning. IJCAI.the cost-based threshold, from Entry 01

Calibration

  1. Guo, C., Pleiss, G., Sun, Y. & Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML, PMLR 70, 1321–1330.modern networks are poorly calibrated; temperature scaling
  2. Kadavath, S. et al. (2022). Language Models (Mostly) Know What They Know. arXiv:2207.05221.self-evaluation by language models
  3. Xiong, M. et al. (2024). Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. ICLR 2024.confidence stated in words tends to be overstated

History

  1. Peterson, W. W. & Birdsall, T. G. (1953). The Theory of Signal Detectability. University of Michigan, technical report.the formalization of signal detection
  2. Receiver operating characteristic. Wikipedia.the roots in Second World War radar

Want the double threshold with your own customer-service numbers?

Vettore Data runs the diagnostic: it splits questions by type, measures the AI's real accuracy and sets the thresholds with your costs. The executive version of this entry, with a calculator, is there.

See the diagnostic at Vettoreor get in touch