Should Your AI Be 90% Accurate—or 85% Accurate and 5% Human?
Imagine you have built a customer-facing AI agent. It answers questions, recommends actions and perhaps even makes a few decisions. Behind it sits a sophisticated decision model. And its 80% accurate
Not bad. But it is not ready for production. So your team has two possible paths.
Option A: Push the AI until it reaches 90% accuracy.
Option B: Push it to perhaps 85%, but introduce a lightweight human touch for the uncertain or sensitive interactions ( catering to further 5%).
At first glance, this sounds like an engineering question. But is it?
When a Human Makes a Mistake, It Is an Accident
Suppose a human customer-service agent misunderstands your question.
You may be irritated.
“Perhaps she misunderstood me.”
“Maybe I didn’t explain it properly.”
“Everyone makes mistakes.”
Now imagine an AI agent makes exactly the same mistake. Our reaction can be surprisingly different.
“This thing doesn’t work.”
One mistake has suddenly become evidence against an entire technology.
There is actually a name for something close to this phenomenon: algorithm aversion.
Experiments have found that people can lose confidence in algorithms after seeing them make mistakes—even when those algorithms perform better overall than humans. In other words, we can be surprisingly forgiving of human imperfection and surprisingly unforgiving of machine imperfection. (DOI)
Research involving autonomous vehicles provides an interesting parallel. Some research suggest that a self driving car, mile to mile, is probably safer than a human-driven car. But psychologically, many humans seem to have made the choice that they would still rather be killed on the road by another human than by a self driving car.
That distinction matters enormously for AI adoption.
Psychologically, a human error can feel like an accident.
A machine error can feel like a failure.
And failures make us question the system.
Perhaps the Goal Is Not to Eliminate Failure
This leads to a slightly uncomfortable idea. What if the fastest route to AI adoption isn’t making AI perfect? What if it is designing the system so that inevitable AI imperfections don’t become catastrophic moments of lost trust?
Consider aviation. Commercial aviation has achieved extraordinary levels of safety. Yet nobody claims that aircraft are literally incapable of accidents. Society accepts aviation not because airplanes have crossed some magical boundary called 100% safe. We accept them because their level of safety, combined with procedures, pilots, redundancy, regulation and recovery mechanisms, has made the residual risk acceptable.
That suggests a much more interesting question for AI:
How safe is safe enough?
And immediately behind it comes another:
What should happen when the AI reaches the boundary of what it knows?
That second question may matter just as much as the first.
Enter the Human—But Only Lightly
Suppose pushing your AI from 80% to 85% accuracy requires reasonable effort.
Going from 85% to 90% requires considerably more.
Going from 90% to 95% requires heroic effort, a larger model, more data, more compute, three committees and possibly the sacrifice of a data scientist.
Eventually, improvements encounter diminishing returns.
So instead of thinking:
AI OR Human
perhaps the better architecture is:
AI → uncertainty → Human
Let the AI handle what it handles well.
But when confidence drops, ambiguity rises, consequences become significant or the customer simply seems uncomfortable, introduce a lightweight human intervention. Not necessarily an entire call-centre operation. Sometimes the human only needs to review, confirm, reassure or take over. That tiny human presence changes something important. The system is no longer pretending to be infallible.
It has been designed to recognise its own boundaries.
And strangely enough, admitting imperfection may make an intelligent system more trustworthy, not less.
“Fine. But Why Not Just Build 100% Accurate AI?”
Because 100% is a dangerous number.
Engineers love it.
Mathematics is suspicious of it.
Real life usually laughs at it.
AI systems optimise multiple objectives simultaneously: accuracy, latency, cost, privacy, fairness, robustness, explainability and many others. Improving one dimension can sometimes degrade another.
Differential privacy gives us a beautiful mathematical example. To protect individuals, privacy-preserving systems can deliberately introduce uncertainty or noise. Stronger privacy can therefore come at the cost of some statistical utility. In differential privacy, the parameter epsilon helps control this trade-off: smaller epsilon generally means stronger privacy, but potentially less utility. (Debabrata Pruseth Blog)
So asking for maximum accuracy, maximum privacy, minimum latency, zero cost and perfect explainability is a little like walking into a restaurant and asking for:
“The tastiest meal you have. Zero calories. Free.”
Optimization has trade-offs.
AI does too.
And Mathematics Has Something Even More Humbling to Tell Us
There is a deeper philosophical layer here.
Gödel’s incompleteness theorems showed, roughly speaking, that sufficiently expressive formal mathematical systems have fundamental limits: there can be mathematical statements that are true but cannot be proved within that formal system.
Alan Turing’s work on computability gave us another boundary. His famous Halting Problem showed that there cannot be a universal algorithm capable of determining, for every possible program and input, whether that program will eventually stop.
These theorems do not mean that AI can never reach 100% accuracy on a particular well-defined task.
But philosophically, they teach us something much more valuable:
Formal systems have boundaries.
There are limits to what can be proved, computed or known from within particular systems.
Perhaps intelligence should therefore not be defined as always having the answer.
Perhaps part of intelligence is knowing when you don’t.
The Most Intelligent AI May Be the One That Knows When to Call a Human
This changes how we think about production AI. Instead of asking only:
“How do we make this model more accurate?”
perhaps we should also ask:
“How should this system behave when it is uncertain?”
That leads to a very different architecture. The AI handles routine and high-confidence situations. Uncertainty is measured rather than hidden. Sensitive or consequential cases receive additional checks. Humans enter selectively rather than constantly. And failures become recoverable interactions rather than reasons to abandon the technology.
The human is no longer there because the AI is bad.
The human is there because uncertainty is part of the architecture.
85% AI + Human May Sometimes Beat 90% AI
And this brings us back to our original question. Would I rather deploy:
90% accurate AI with no human intervention
or
85% accurate AI with intelligent human intervention around its uncertainty?
There is no universal answer. It depends on the consequences of errors, their distribution, cost, latency, regulatory requirements and many other factors.
And importantly, 85% and 90% are illustrations—not universal thresholds.
But there is a larger principle hiding behind those numbers.
We shouldn’t optimise only for model accuracy.
We should optimise for system trustworthiness and adoption.
A slightly less accurate model surrounded by good uncertainty detection, escalation, human judgment and graceful recovery may create a better real-world system than a more accurate model operating alone. Because the final mile of AI adoption may not be an AI problem at all. It may be a human problem.
For decades we have asked:
How close can machines get to humans?
The more interesting question now might be:
How should humans and machines divide uncertainty between them?
Perhaps the future of AI isn’t a machine that never fails. Perhaps it is a machine sophisticated enough to know when it might fail—and humble enough to ask a human for help.
There is something wonderfully ironic about that. After spending billions trying to make machines intelligent, one of the most intelligent things we may eventually teach them to say is:
“I’m not completely sure. Let me get someone.”
Discover more from Debabrata Pruseth
Subscribe to get the latest posts sent to your email.
