Does Every AI Control-Plane Decision Need an LLM? Evaluating Specialized Decision Models for Policy-Aware AI Routing

Author: Debabrata Pruseth
Publication Date: 2026/09/30
Document Type: Technical Note / Research Article
Language: English

Abstract

Large language models (LLMs) are increasingly used not only to generate content but also to make control-plane decisions such as selecting models, tools, agents, and workflows.

Existing routing research has demonstrated that heterogeneous model portfolios can improve cost–quality trade-offs, but a preceding systems question remains: what computational mechanism should perform the routing decision itself? We compare deterministic rules, weighted heuristics, supervised machine learning, a general-purpose LLM, and a specialized fast decision model in a synthetic routing environment designed to represent regulated-banking constraints. Deterministic governance first constructs a policy-eligible candidate set; routing intelligence then selects only among permitted candidates. The primary experiment contains 400 requests, of which 304 require live model-selection decisions and 96 are deterministically resolved as NO ROUTE. Preferred Synthetic Reference-Route Agreement (SRRA) is 75.5%/71.5% for the general-purpose LLM and 74.5%/70.0% for the specialized decision model on IID/template-held-out benchmarks.

Exact paired McNemar tests with Holm correction identify no statistically detectable agreement difference; this does not establish equivalence. Across actual live routing decisions, mean latency is 1423.75/1406.17 ms for the LLM execution path and 543.80/562.02 ms for the specializeddecision execution path, approximately 2.50–2.62 times lower for the latter. Total observed primary routing API cost is USD 0.223006 versus USD 0.029983, approximately 7.44 times lower under the evaluated providers and pricing. A separate 500-decision repeated-execution study shows greater observed selection consistency for the specialized decision path, descriptively.

The results do not establish a universal winner; they indicate that general-purpose generative inference need not be the default mechanism for bounded, already-structured AI control-plane decisions.

Keywords
System 1 AI, System 1 decision models, System 1 vs LLM, specialized decision models, LLM routing, AI model routing, policy-aware AI routing, AI control plane, model router, AI governance, enterprise AI architecture, banking AI, regulated AI, agentic AI, LLM cost optimization, LLM latency, Jev 1.13

Download Research PDF

Does Every AI Control-Plane Decision Need an LLM? Evaluating Specialized Decision Models for Policy-Aware AI Routing

Suggested Citation
Pruseth, D. (2026). Does Every AI Control-Plane Decision Need an LLM? Evaluating Specialized Decision Models for Policy-Aware AI Routing. Debabrata Pruseth AI Blog.

Companion Note
This page provides the abstract and full-text PDF for the research version of the article. A companion blog post explains the same work in a more narrative and implementation-focused style.

Read the companion blog:
https://debabratapruseth.com/system-1-vs-llm-does-every-ai-decision-really-need-a-large-language-model/