Chapter 10: The Evolution of Architectures: From Context to Swarm

Section 2: RAG: The Inflection Point

For the executive commissioning AI systems, RAG is simple: it is taking a language model to data it currently doesn’t know. The metaphor is so common it becomes invisible: a jurist (the model) with the internet (the retriever) and a library (the vector database) at their side. Correct, but insufficient for understanding why RAG became the dominant pattern and, at the same time, why it was only the beginning.

Let’s review what happened before. GPT-3.5 was trained on data up to April 2023. If in May 2023 your company launched a new product, the model didn’t know it. If your legal team published an internal policy on Thursday, the model was still ignoring it on Friday. Training happened in cycles of months, at tremendous cost (Anthropic, 2024; Kaplan et al., 2020), and each learning step required recompiling hundreds of thousands of parameters. It was like being condemned to rewrite your entire philosophy every time you learned a new song.

RAG upended that. Instead of training, you retrieve. Before the model generates its response, you query a database (traditional or vector; Chapter 11 went into why vectors) and hand it relevant contextual fragments in its system prompt. The model sees fresh context, produces a more precise answer, and does so in milliseconds, with no retraining, without waiting for batch cycles. For an executive: RAG means “my AI systems are never out of date.”

But here comes the complication. RAG solves the problem of data access; it doesn’t solve the problem of truth. When you build a vector database from corporate documents, you inherit their biases, their omissions, their contradictions. If your documentation says policy A applies to customers in Peru and Compliance’s documentation says it doesn’t, RAG retrieves both and the model generates an answer containing both contradictory truths. The problem is now visible (that is progress) but it remains a problem. This drives the need for Chapter 13: evaluators (LLM-judges, deterministic evals) that verify the model’s output is consistent with company policies, not just with the average of the internet.

The maintenance burden of RAG is also underestimated. A vector database is not “set and forget.” Corporate terminology changes, and the model starts retrieving fragments with outdated vocabulary. New policies are published, and you have to re-embed (re-extract vectors) from scratch. At BigData scale (where “millions of documents” is the norm, not the exception), this demands infrastructure. Pipelines that detect changes in the source repositories, ETLs that update the vectors, evaluators that verify coherence. The “build a quick RAG” pattern crumbles when you realize that you are, in fact, building a continuous data pipeline with indefinite operating costs.

So, when is RAG the right answer? Not always. Some use cases:

RAG is your tool when:

• You have frequently updated corporate data (product documentation, regulations, a support knowledge base).
• Accuracy improves materially with recent context (vs. the model’s training context).
• You can maintain the quality of the retrieved fragments (clean metadata, no duplication, no contradictions).
• The retrieval latency (vector query + DB read) fits your response-time budget.

RAG is a hidden cost when:

• You use RAG to “improve a weak model” instead of using a more capable model. (Chapter 3 showed it: sometimes, Opus costs the same as Haiku + RAG.)
• You retrieve hundreds of fragments in parallel because “more context is better context.” (It isn’t. Dilution. Chapter 11 quantifies it.)
• Your data source changes so fast the vectors age faster than you re-embed them.

The turn that happens in 2024-2025, and that connects RAG with what comes next, is this: RAG assumes the model only needs to read. But in most AI systems in production, the model also needs to write, execute actions, validate decisions. There is the coupling. A model that says “I need to see the project specification in Jira” (reading, which RAG solves) is different from a model that says “I need to update the project status in Jira to ‘in review,’ but only if two team members approved it first” (reading + writing + decision + validation). For this, RAG is no longer enough.

Use Case	Primary Need	Tool	Why	Estimated Cost
“When does product X launch?”	Reading updated data	RAG	Model + retriever, no action cycles	Low (context + inference)
“Generate a summary of the latest PRs”	Reading + synthesis	RAG	The model integrates multiple sources	Low-Medium (context volume)
“Create a Jira ticket with this data”	Reading + writing + API auth	MCP	Permission guardians; the model doesn’t handle credentials	Low (1-2 calls)
“Is it safe to deploy? If not, alert the on-call”	Reading + decision + secondary action	MCP + Evaluator	MCP for secure access; evaluator to validate the decision	Low-Medium (binary decision)
“Close all tickets for a customer who churned”	Reading + decision + multiple writes	Agent (loop)	Iterates: retrieves tickets, validates state, executes, verifies	Medium (iterations unknown a priori)
“Orchestrate 5 agents to process an audit request”	Coordination + scalability	Orchestration (Kafka + Agents)	Events decouple the agents; each one is independent	Medium-High (event volume)
“Have 50 risk management agents detect fraud in parallel”	Emergence + anticipation	Multi-agent (no choreography)	Local rules + communication = emergent patterns	High (complexity + observability)
“Process 500K embeddings daily, <100ms latency, on-prem compliance”	Scalability + control	NVIDIA NIM on-premises	Standard APIs, optimized hardware, no data egress	Medium (operations + H100s)
NIM Pipeline: embed + rerank + LLM, autoscaling P99	Declarative orchestration	NIM Operator on K8s	NIMService + NIMPipeline CRDs; HPA by GPU metrics	Medium-High (K8s + GPU)

---

Section 3: From MCP to Orchestrations: The Coordination Layer

When you asked a model whether it was safe to deploy certain code, RAG gave you context: “Here are the latest commits, the issue history, the documentation.” It generated an analysis. Period. Now, if the model says “It’s not safe; I need someone to review the changes in AWS,” what happens? The model ends its generation and you, manually, create a support ticket. RAG doesn’t contemplate actions.

MCP (Anthropic, 2024) solves that with a deceptively simple idea: give the model tools, but without it handling either credentials or the system’s topology. Instead of:

Model → ["I want to read files", "I want to write to the DB"]
        ↓ (Model obtains credentials)
        Security chaos

What happens is:

Model → ["I want to invoke `get_aws_compliance_status`"]
        ↓ (Server translates to an authenticated call)
        Model sees the response, continues reasoning

The protocol isn’t magic; it is discipline. The model declares what it needs to do, the server validates permissions, executes under guardians (rate limits, auditability), and returns the result. It is the hexagonal pattern of Chapter 6, scaled. This change occurred in parallel with LangGraph (LangChain, 2024), which formalized how to orchestrate agents with state: not just isolated calls, but state machines driven by the model’s decisions.

Here is what is heating up Silicon Valley. In 2025, the question stopped being “how do I get an agent to resolve a ticket?” and became “how do I coordinate 50 agents in parallel resolving different aspects of the same ticket?” Manning Publications published “AI Agents in Action” (2025), which captures the emerging pattern: specialized agents, each with a role (context retrieval, validation, execution, audit), communicating through events. Not by RPC. By events.

Here Kafka comes in (which you saw in Chapter 7, but now at scale). One agent generates an event (“Ticket 4521 classified as HIGH PRIORITY”). Another agent subscribed to that event acts (“Check whether there are precedents of similar tickets in the last month”). A third publishes a result (“3 precedents found; the recommendation is to escalate to level 2”). The fourth, an evaluator, validates coherence (“Is the escalation consistent with SLA policies?”). None waits for any other. The system is temporally decoupled. At message scale, BigData stops being “millions of records in a DB” and becomes “millions of events in a stream,” and the agents read history, make decisions, and publish new events.

The cost of this, at the US level, is clearly seen by Anthropic (2025): using multiple coordinated agents consumes 15 times more tokens than a single agent doing the same thing. But variance decreases by 80%. That is: you pay more money, you gain predictability and scalability. For financial systems (JPMorgan is experimenting with agentic AI in trading; Bloomberg reports it in January 2025), that trade-off is acceptable. For a startup that processes 10 tickets a day, it is prohibitive.

Orchestrations, then, are not “more agents = a better solution.” They are decisions of scale. When the complexity of a problem requires division of labor, and when that division justifies the additional tokens (and latency).

---

Section 4: Multi-Agent Systems and the Disruptive Horizon

When multiple agents collaborate under an orchestration scheme, something strange happens: patterns emerge without being programmed.

Consider a high-frequency trading system based on agents. Ten specialized agents:

• Agent A: Retrieves financial news in real time
• Agent B: Analyzes regulatory impact
• Agent C: Validates technical signals on charts
• Agents D-G: Executors (buy/sell orders with capital and risk limits)
• Agent H: Correlation monitor (detects whether multiple agents are betting on the same thing)
• Agent I: Auditor (records every transaction, flags anomalies)
• Agent J: Learner (updates decision models based on historical outcomes)

None has an explicit instruction “coordinate with the others.” Yet when A publishes an event (“new crypto regulation”), B sees it, runs an analysis, publishes an event. C sees B’s analysis, validates it against charts, publishes confidence. D sees B’s and C’s events, adds information, decides to buy. H sees that D is betting heavily, verifies correlation with other traders (agents), and alerts if there is risk concentration. J records the outcome of D’s decision: did it work? Why?

This is emergence: the collective behavior is not the sum of individual behaviors. It is a product of local interactions under simple rules.

DeepSeek (an AI research company, China) published MoE (Mixture of Experts) architectures in 2024 that encapsulate exactly this: instead of a monolithic model, multiple experts (small, specialized transformers) are activated selectively depending on the input. It is multi-agent inside the model. The Western industry is replicating the pattern: Anthropic publishes on “constitutional AI at scale” (applying corporate values in distributed systems); OpenAI experiments with deep reasoning (o1-mini, December 2024) where the model reserves “thinking tokens” for internal reflection before generating. It is the pattern of agents that reflect before acting.

Smart cities are the most visible canvas for this. Singapore (the “Smart Nation” project, since 2014, now with agentic AI in 2025) operates with specialized agents for traffic, energy, waste, and public health. There is no “central controller”; each subsystem is a team of agents that communicates through APIs and events. A gas leak is detected (sensor → monitoring agent → event), the event triggers health alerts (health agents), air flow adjustments (HVAC agents), and neighbor notification (communication agents). All without human intervention, under rules. It is multi-agent applied to urban infrastructure.

What is in fashion in the US is not so much multi-agent systems (an old concept; Sharon Goldwater, Bonabeau, the 1990s) but **multi-agen multi-agent systems with integrated evaluation and adaptation. Guardrails AI (a 2023 startup, well funded) offers frameworks where each agent has local guardians (not to be confused with pattern 16: guardians in the outer ring; here, guardians in each agent). Anthropic publishes on “closed-loop evaluation”: the agent decides, the evaluator validates, the evaluator feeds back to the agent, the agent updates its decision strategy. It is distributed RL (reinforcement learning) without the need for central retraining.

Here is the disruptive cost: a multi-agent system with 50 agents, each with an integrated evaluator, processes millions of events and millions of inferences in a day. The token bill is an order of magnitude higher than anything you’ve seen before. Bloomberg, JPMorgan, Palantir: all are optimizing “the cost of agentic AI at scale.” The emerging answer is to use small models (Haiku, Sonnet) as individual agents and reserve large models (Opus, o1) only for critical decisions. It is the tiering of Chapter 3, scaled to orchestrations.

Model adapters (an architectural pattern, 2025): because switching from Sonnet to Opus in production without rewriting is critical, some teams build “model adapters,” essentially ports that abstract which LLM runs each agent. It is the hexagonal pattern, but applied to model selection at runtime. This is being experimented with at Stripe teams, at Anthropic internally, and at fintech startups.

The conclusion: multi-agent systems are not the future. They are operating today in finance, logistics, cities. What is disruptive is making them economically viable (reducing tokens without losing quality) and auditable (every decision traceable). The next 18 months will define whether the industry achieves both.

---

Section 5: Using Models as an Expert: Ecosystem, Architecture, and Quantum Computing

So far we have dealt with agents, orchestrations, evaluation. There is a prior level that determines everything else: how you use the model itself. It isn’t trivial. The difference between a team that spends $100,000/month on API calls and a team that gets 10× better results spending $10,000 is architecture.

When an engineer calls claude.messages.create() inside a loop, they are using the model as a utility: input → output → done. It is fine for prototypes. Unacceptable in production. At scale, you must treat the model as an architectural component that lives in an ecosystem, where decisions about batching, caching, routing, and capability adaptation aren’t “optional optimizations”; they are the border between economic viability and insolvency.

The Model Ecosystem: Batching, Caching, Routing

Let’s start with the invisible ones. When your agent publishes an event (“I need to analyze 5,000 invoices for compliance”), a naive architecture makes 5,000 sequential API calls. You wait 2 hours. You pay $150. A mature architecture does batching: it groups the 5,000 invoices, sends them in a single request (or a few parallel requests), waits 30 minutes, pays $45. The difference: the model processes the batch in a context cache, not in 5,000 isolated inferences.

Anthropic published its prompt caching in January 2025: your system stores the master prompt (“you are a compliance reviewer, here are the 500 relevant policies”) just once. The next 999 calls reuse that prompt, paying 90% less for those tokens. It is transparent to the user (the API handles it). Profound for whoever designs: now your compliance agent costs 1/10 if you design the batching.

Routing is equally critical. Do all problems require Opus? No. An FAQ question: Haiku. A merger decision: Opus. A code analysis that fits in 100K tokens: Sonnet with prompt caching. If your orchestration intelligently routes each task to the right model (based on complexity, cost, required latency), your operating cost is divided by three. The pattern: Model Adapter is not cosmetic. It is the port where the routing decision lives.

NVIDIA NIM: Microservices Optimized for Inference at Scale

Now, once you’ve designed the batching, the caching, and the routing, the next challenge is where you host that logic. Direct APIs to Anthropic, OpenAI, or Google work for prototypes. For production at scale (hundreds of millions of inferences, latency <100ms, availability 99.99%), you need infrastructure. Here NVIDIA NIM (NVIDIA Inference Microservices) comes in, launched in 2023-2024.

What is NIM? A cloud-native platform that packages optimized AI models (not just LLMs; also vision, audio, embedding) with production-ready deployment infrastructure. The value isn’t in “running a model”; it’s in “running a model at scale without optimizing infrastructure yourself.”

The key components:

1. Hardware-optimized inference engines: NIM doesn’t call a remote API. You host containers on your infrastructure (public cloud, on-premises, edge). NVIDIA handles the optimization: automatic quantization (FP8, INT8 for models with no loss, speed +2-3×), GPU-specific compilation (NVIDIA H100, H200, L40, RTX 6000), optimized kernels. For a model that takes 500ms/inference on CPU, an optimized NVIDIA NIM on GPU takes 50ms.

2. Standard APIs by domain: Instead of each model having its own API, NIM exposes standardized interfaces. The LLM chat endpoint follows the OpenAI spec. The embedding endpoint follows a standard format. Vision follows the OpenAI Vision API. A developer who knows one API knows them all. Switching models (say, from Llama 405B to Mistral 8×22B) requires changing a configuration variable, not rewriting client code.

3. Flexible deployment: Cloud hosting (through the NVIDIA API Catalog, which is managed). On-premises (you download the container, run it on your GPU cluster, and maintain it yourself). Hybrid: the critical part runs in your DC, with a fallback to the cloud if your infrastructure goes down.

4. Enterprise support: SLAs, security updates, bug fixes. For teams of 50+ engineers using AI in production, the cost of support is quickly amortized in “time not spent debugging CUDA.”

How NIM fits into your model ecosystem:

Suppose the orchestration we saw earlier (batching, caching, routing) lives in Kubernetes. Before, each routing decision was “do we call the Anthropic API or Azure OpenAI?” Now it is:

Task "document classification"
  ↓ (Router)
  Low complexity (FAQ)? → Haiku on local NIM (on-prem, <50ms latency)
  Medium complexity? → Sonnet on cloud-managed NIM (NVIDIA API Catalog, <200ms)
  High complexity? → Opus on the Anthropic API directly (maximum reliability, latency accepted)

The benefit: predictable latency. A local RAG costs 50ms retrieval + 100ms LLM (via remote API) = 150ms on average. With local NIM, retrieval 50ms + Haiku on GPU 30ms = 80ms. For a system processing 1,000 documents/hour, that is 1.4 minutes of accumulated latency saved per hour. At datacenter scale, it is significant.

When NIM is your tool:

• Inference >100K/day. At that level, owning the infrastructure pays for itself.
• Latency <100ms is critical (trading, medical diagnosis, robot control).
• Compliance: sensitive models must never leave your network (healthcare HIPAA, finance PCI-DSS).
• A mix of open and closed models. NIM hosts open source (Llama, Mistral, Code Llama) and, through integration, access to proprietary models.

NIM on RTX PCs: The Development → Production Bridge (2024)

There is an intermediate use of NIM worth mentioning: local development on RTX workstations.

Engineering teams with complex RAG pipelines face this dilemma: developing against remote APIs (Anthropic, OpenAI) is costly during experimentation (1,000 iterations = thousands in bills). But downloading Llama 70B locally requires A100s ($16K/GPU).

NIM on RTX PCs (50 series, Blackwell) closes that gap. With Podman on WSL2, an engineer on a workstation (RTX 6000 Ada, ~$7K) runs:

podman pull nvcr.io/nim/meta/llama-70b-instruct
podman run --gpus all -p 8000:8000 nvcr.io/nim/meta/llama-70b-instruct

Result: an OpenAI-compatible server on localhost:8000. Zero API calls. It integrates with LangChain, CrewAI, FlowiseAI, VS Code AI Toolkit: the same interface it will use in production with NIM on Kubernetes.

Why it matters:

• Offline development (no internet, flights, sensitive environments)
• Cost: $0 per iteration (vs. $1-10 per API call during experimentation)
• Unified experience: code that runs locally runs the same on production K8s
• Testing RAG pipelines: local embedding + reranking + LLM, the same day of writing

The pattern:

1. The engineer writes the RAG pipeline against local NIM on their RTX PC
2. Tests reach 95% accuracy
3. Commit to main
4. CI/CD deploys to NIM Operator on K8s (100 lines of YAML)
5. Zero code changes. Only infrastructure changes.

This is horizontal scalability without refactoring. And it is possible because NIM exposes standard interfaces (OpenAI spec) in both contexts.

When NIM is not your tool:

• <10K inferences/day. The operating overhead (monitoring, updates, disaster recovery) doesn’t amortize.
• Flexible latency (a batch report that runs overnight is fine).
• A single model, a single use case. The simplicity of “I launch an API and forget it” outweighs the latency savings.
• Legacy infrastructure without Kubernetes (but note: local RTX remains viable even without K8s)

Cost table: Direct API vs. Local NIM vs. K8s NIM

Scenario	Tool	Initial Cost	Cost/Inf.	Latency	Ideal For
Prototyping (≤100 inf/day)	Claude API Direct	$0	$0.003–0.02	500ms+	Startups, quick MVP
Development (1K–10K inf/day)	Local NIM on RTX PC	$7K (GPU)	$0	50–100ms	Engineering teams
Production MVP (10K–100K)	NIM on NVIDIA API Catalog	$0–500 (credits)	$0.001–0.005	100–200ms	Startups with a moderate SLA
Enterprise (100K–1M inf/day)	NIM Operator on K8s · on-prem	$50K–200K (GPUs)	$0.0001–0.001	30–50ms	Finance, healthcare
Maximum (>1M inf/day)	Multi-region NIM + DGX Cloud	$200K+	$0.00001–0.0005	10–30ms	JPMorgan, Stripe-scale

NIM is the answer to “how do I monetize scalability in inference?” For teams that already have batching, caching, and routing in place, NIM is the “where,” the hosting layer that makes all of it viable at scale without compromising latency or compliance.

NIM Operator: Declarative Orchestration of Inference on Kubernetes (2024-2025)

The next point is decisive. We’ve talked about batching, caching, routing, and NIM as infrastructure. But how do teams of 50+ engineers orchestrate not 1 but 10 NIM services simultaneously? How do they do rolling upgrades without downtime? How do they autoscale based not just on CPU but on inference metrics (KVCache, request queue)?

The answer is NIM Operator (NVIDIA, 2024): a Kubernetes extension that treats entire inference pipelines as first-class citizens.

The pattern is declarative, GitOps-friendly:

apiVersion: apps.nvidia.com/v1alpha1
kind: NIMPipeline
metadata:
  name: rag-pipeline-prod
spec:
  services:
    - name: embedding-service
      model: nvidia/NV-Embed-QA-004
      replicas: 3
      resources:
        gpu: 1
    - name: llm-service
      model: meta/llama-70b-instruct
      replicas: 2
      resources:
        gpu: 4
    - name: reranker-service
      model: nvidia/nv-rerank-qa-mixedbread-v1
      replicas: 2
      resources:
        gpu: 1
  autoscaling:
    metrics:
      - type: GPU_MEMORY
        target: 80
      - type: QUEUE_LENGTH
        target: 500
      - type: REQUEST_LATENCY_P99
        target: 500ms

This isn’t YAML for YAML’s sake. It’s architecture. What happens under the hood:

1. NIMService CRD (Custom Resource Definition): Each individual service (embedding, LLM, reranking) is a NIMService. Lifecycle management, health checks, rolling upgrades.

2. NIMPipeline CRD: Orchestrates multiple NIMServices as a coherent unit. A real RAG pipeline: when a query arrives, the embedding-service processes it, passes the result to the reranker-service, which validates relevance, and then it goes to the llm-service. All declared once, orchestrated by Kubernetes automatically.

3. Intelligent autoscaling: It isn’t CPU. It’s inference metrics:

   • KVCache utilization (how much memory the model’s context window is using)
   • Queue length (how many requests are waiting)
   • P99 latency (are we violating the SLA?)

   If KVCache rises to 85%, add a pod. If P99 latency drops below 200ms, scale down. All dynamic.

4. Model pre-caching: The operator distributes large models (Llama 70B = 140GB) to persistent volumes on the nodes. When a new pod is created, it doesn’t download 140GB from the internet; it mounts it locally. Scaling is 10× faster.

5. Rolling upgrades with no downtime: Switching from Llama 70B to Llama 405B? In the YAML you change the model: field. The operator:

   • Creates new pods with the new model
   • Drains requests from the old pods (with a timeout)
   • Kills the old ones
   • Zero downtime, completely automatic

This is Chapter 6 (hexagonal, guardians) scaled. But instead of guardians in the outer ring, they are guardians in infrastructure. The model doesn’t know it’s being autoscaled, replaced, or monitored. The operator handles it.

The cost: a NIMPipeline with 3 services, autoscaled to 10 pods, processes 100K requests/day. You define the SLA (P99 <500ms, GPU memory <80%). The cluster adapts itself. You don’t need an on-call engineer checking utilization at 3am.

Where does this fit in your agent orchestration? Precisely here: if each specialized agent (retrieval, analysis, validation, execution) runs as a NIMService inside a NIMPipeline, your 50-agent orchestration isn’t chaos. It is a YAML, versioned in Git, audited, reproducible.

Architectural conclusion: Kubernetes without NIM Operator requires writing custom controllers (months of work). NIM Operator is the shortcut that generalizes the pattern. It is what Kafka was to Zookeeper: an abstraction that simplifies the unmanageable.

Fine-Tuning vs. Context: The Decision That Matters

Now comes an architectural decision that paralyzes teams. You have internal data (10 million examples of resolved tickets). Do you fine-tune a model, or build RAG?

Fine-tuning costs: $5,000-$50,000 in tokens, 1-2 weeks of turnaround, and requires maintaining model versions (if your policies change, will you retrain?). The dividend: the model “understands” your domain; a specific question gets a nuanced answer.

RAG costs: $500-$2,000 in infrastructure, re-embedding every time your data changes (automatic if the pipeline is good), zero wait time.

The right answer is contextual, not binary. MIT CSAIL (Learning and Intelligent Systems Group, 2025) published a framework: fine-tuning is optimal when (a) you have >10M high-quality examples, (b) the pattern is stable (it changes <5% annually), (c) latency is critical (you need <100ms, RAG adds 50ms of retrieval). For most corporate cases, RAG wins. But there is a third way Stanford is exploring (Perdomo-Ortiz’s lab at Google Quantum AI, 2024-2025): knowledge distillation at runtime. You have a large model (Opus) that “knows” your domain. It continuously distills that knowledge into a smaller model (Haiku), which is updated without explicit retraining. The result: Haiku with Opus’s knowledge, at Haiku’s cost.

Model Architecture: AI Layers vs. Traditional Infrastructure

Here comes the twist. Teams build systems where “the model is just one more service,” like the DB, like Kafka. Correct in the abstract; dangerous in the concrete. A model is not a DB. It doesn’t guarantee consistency, it has no ACID, it isn’t deterministic. Injecting it into traditional architectures without isolation layers = technical debt.

The answer: AI layers as first-order infrastructure. Harvard (Systems Security Group, 2025) has documented patterns: the AI layer doesn’t live where critical business logic lives. It lives at a boundary (as in Chapter 6, the outer ring). Between that AI layer and the core go evaluators, validators, guardians. The model generates; the evaluator validates; if it fails, the system degrades gracefully (goes back to a known behavior, doesn’t crash).

In orchestrations, this intensifies. Fifty agents, each calling models, each possibly hallucinating. Without explicit isolation layers, a hallucination in one agent spreads like fire (other agents read its event, assume it correct, build on it). With layers: every event published by an agent carries a “confidence” seal and an “audited flag.” Consuming agents can decide “I trust this event only if it was audited.”

Decoupled Control Layers and Persistent Agents: Jev and OpenAI Dots (2026)

The industrial deployment of autonomous agents during 2025 and 2026 exposed a structural design flaw in conventional agentic loops: the overload of the generative model. In the first ReAct-inspired schemes, a single deliberative LLM took on all the functions at each iteration of the loop: reasoning about the goal, writing the conversational prose, selecting the right tool, serializing the arguments into JSON schemas, judging the safety of the action, and deciding whether the task had concluded.

This monolithic design head-on violates the Single Responsibility Principle (SRP) and creates tight coupling between linguistic generation and operating system control. The operational consequence is predictable: syntactic fragility (JSON parsing errors that demand retry loops with needless token consumption), latencies of several seconds per step, and hallucinations in deriving arguments with potentially destructive consequences.

To resolve this design crisis, two key architectural innovations emerge in 2026 that redefine how software soundness governs agentic AI:

1. Jev (TypeSafe AI): Decoupling Control Through System One Models Faced with the cost and slowness of invoking a deliberative model (System Two) for low-level routing decisions, Jev (TypeSafe AI, 2026) introduces an agentic control layer based on a fast-inference, non-generative classifier model. Jev emits neither free text nor JSON fragments; it answers exclusively typed questions with calibrated probability distributions. Its harness breaks the cycle into four decoupled phases:

   • SELECT: Tool selection over the full catalog through the Choice primitive (returns a label, a probability distribution, and a confidence level). If confidence falls below the threshold (e.g., 0.60), it escalates the query instead of guessing.
   • GATE: Safety evaluation prior to execution. It runs closed questions in parallel (Score and Noul, a Boolean probability) that quantify whether the action is destructive, its blast radius, and its reversibility. The cardinal architectural rule is explicit: the gate advises, the runtime enforces. If a destructive action exceeds the threshold on a system or external environment, the runtime executes a flat denial (deny) with no human intervention, eliminating alert fatigue.
   • EXECUTE: Execution inside a rigorous sandbox (sandbox.ts) with canonical paths and process isolation. Arguments are derived deterministically (arguments.ts), treating any external or hostile text strictly as data, never as an instruction.
   • JUDGE: Evaluation of the loop’s state (completeness, infinite loops, progress). A hard step limit is evaluated as the first line of defense before weighing the model’s probabilities, guaranteeing that the agent always stops.

   The lesson for software engineering is direct: Jev achieves high cohesion (policy resides in pure functions in policy.ts, effects in isolated tools, and orchestration in agent.ts) and low coupling through dependency inversion: the harness depends on an abstract Decider interface. This enables substitution by a deterministic FakeDecider in CI, making it possible to test all the branch logic and safety containment offline, reproducibly, and at zero cost.

2. OpenAI Dots (2026): Persistent Agents and Governed Control Planes At the scale of persistence and enterprise collaboration, OpenAI Dots (OpenAI, 2026) formalizes the move from ephemeral agentic sessions to persistent digital coworkers (always-on digital coworkers). Architecturally, Dots introduces two essential defensive components:

   • Execution isolation in dedicated cloud environments: Each agent operates inside an independent sandbox infrastructure (lightweight virtual machines or ephemeral containers). Artifacts and state are confined to its environment, preventing cross-contamination between agents.
   • Governed control plane (governed control plane): Identity management, access permissions for external tools, budget constraints, and forensic audit trails don’t reside in the agent’s context, but in a governed intermediate infrastructure layer.
   • Bounded Contexts and asynchronous contracts: Agents don’t communicate through ambiguous text streams, but through structured events and typed contracts. Each Dot maintains high cohesion with respect to its specialized domain role, keeping the swarm from degenerating into a spaghetti network of mutual dependencies.

The convergence of Jev and Dots ratifies this work’s thesis: software architecture is not an ornamental accessory for AI, but its indispensable safeguard. Without modular boundaries, strict typing, dependency inversion, and interface segregation, agentic systems collapse under their own probabilistic entropy.

Quantum Computing: From Theory to Production (2025-2030)

Now the near future. In 2024-2025, quantum computing and agentic AI seem like different worlds. They aren’t. There is a specific intersection that matters, and it is closer than it looks.

First, a theoretical clarification. Quantum computing harnesses the principles of quantum mechanics to perform complex calculations that are impossible for classical computers. Instead of bits (0 or 1), it uses qubits that can exist in a superposition of both states at once. This property, together with quantum entanglement, makes it possible to explore solution spaces exponentially faster. But here is the myth: “quantum will run prompts directly.” Wrong. It is that certain problems agents solve (combinatorial optimization, probabilistic sampling, search in massive databases) have potential advantages if you access quantum hardware.

The accelerated history. Richard Feynman recognized the potential in 1981. David Deutsch presented the universal quantum computer in 1985. Peter Shor developed his algorithm in 1994 (efficient factoring, a real threat to cryptography). In 2001, D-Wave launches the first commercial quantum computer. In 2019, Google announces “quantum supremacy” (Sycamore completed a specific calculation faster than any supercomputer). In 2024, Google Willow claims logical qubits with error rates <1%, a leap from ~10% two years ago. IBM Quantum published error-mitigation strategies that make it viable to use ~100 qubits for real problems.

Real operational cases today (2024-2025):

Google uses quantum computing to develop new battery materials (potential: significantly increasing energy storage capacity). The algorithm explores the space of molecular configurations exponentially faster than classical heuristics.

IBM, in collaboration with BASF, uses quantum computing to design new catalysts that could transform the chemical industry. In June 2023, IBM and Samsung announced a collaboration to develop next-generation quantum computers. IBM’s roadmap points to 100,000 qubits in a quantum-centric computer by 2033 (an explicit ambition).

JPMorgan Chase develops new algorithms for investment portfolio optimization. The problem: with 1,000 assets, there are 2^1000 permutations. A classical solver takes months; a quantum solver could do it in hours.

Volkswagen optimizes its transport routes (VRP: Vehicle Routing Problem). They have hundreds of delivery points. Quantum routing reduces costs and improves efficiency.

Pfizer uses quantum computing for new drug discovery (molecular simulation to accelerate treatment development).

Goldman Sachs develops new risk models (quantum advantage in portfolio evaluation).

Maersk optimizes supply chain planning (a massive combinatorial problem; quantum reduces delivery times).

Novartis develops more accurate diagnostic models (molecular simulation, pattern recognition).

These are not pilot projects. They are real operations (many with verifiable publications). What’s missing: scaling (going from 100 logical qubits to 10,000), stability (maintaining coherence longer), and education in engineering teams.

Where Your Agent Comes into Contact with Quantum (Real Scenarios)

1. Combinatorial optimization at scale: Your agent observes: “This is a routing problem with 500+ points.” Architectural gate: is a quantum solver available? YES → Delegate to Amazon Braket (timeout: 2 minutes). NO → Fall back to a classical heuristic (timeout: 30 minutes). The result is ingested by the agent; reasoning continues.

2. Probabilistic sampling: Your generative model needs possible scenarios for risk evaluation (finance). A quantum sampler generates scenarios that cover the probability space uniformly. Papers from Harvard (Aspuru-Guzik lab, 2024) and Rigetti show this working with ~500 qubits. Not yet in general production, but prototyped in fintech labs.

3. Search in massive vector databases: Your agent retrieves, out of 10 million vectors, the one most similar to the query. Grover’s algorithm (theoretical quantum, 1996) offers √N speedup. For 10M vectors, that is 100× in theory. In practice, overheads reduce it to ~3-5×. IonQ is prototyping this with fintech companies (2025).

Pioneers and Academic Leadership

The theoretical founders remain active. Isaac Chuang (MIT), considered one of the founding fathers of quantum computing, makes fundamental contributions to the development of quantum algorithms and error correction. John Preskill (Caltech), a theoretical physicist, makes important contributions to the understanding of the foundations. Scott Aaronson (University of Texas at Austin), a quantum computing theorist, has clarified the limits of what is possible vs. impossible in quantum. Hartmut Häfele and his group (Google AI) advance hardware architectures.

But there is an emerging movement: Alejandro Perdomo-Ortiz, now at Google Quantum AI (formerly Stanford), leads research in quantum machine learning applied to real problems. His group regularly publishes on quantum-classical hybrids.

Talent is emerging in Latin America. Fernando Pastawski (Argentina, 40 years old), a theoretical physicist at CONICET, works on quantum algorithms for simulating complex systems. Cecilia Vergara (Mexico, 37 years old), an engineer at LNCC Brazil, develops quantum hardware (design and construction of superconducting qubits). Felipe Carillo (Colombia, 35 years old), a mathematician at IMEC Brazil, contributes to the complexity theory of quantum algorithms. Mariana Weiss (Brazil, 32 years old), a physicist at IFSC, studies fundamentals (quantum decoherence). Diego Ortiz (Chile, 29 years old), an engineer at the Universidad de Chile, develops software for programming quantum computers.

In Peru, the scene is emerging but promising. The Pontificia Universidad Católica del Perú (PUCP) offers introductory courses. The Computational Physics Research Group at PUCP, led by Dr. Jorge Luis Peralta, shows explicit interest in quantum computing. The Universidad Nacional de Ingeniería (UNI) integrates quantum into its curriculum. The Universidad Norbert Wiener has faculty who have undertaken related studies. This training infrastructure is strategic: the next decade will demand engineers with combined training in quantum computing, AI, and agentic architectures.

Operational Reality 2026: Quantum Is Not a “Silver Bullet”

Amazon Braket offers hybrid access (classical + quantum in the same job). Your agent can run classical logic, detect that it needs combinatorial optimization, delegate it to a quantum solver via Braket, and summarize with classical logic. Microsoft Azure Quantum does something similar. This is hybrid architecture, and it exists today.

But why doesn’t everyone use it? Cost, variable latency, and a lack of reliable benchmarks. A routing problem on Braket costs ~$10-50 USD per invocation (depending on complexity). A classical heuristic costs $0.01. Quantum is 1,000× more expensive. It is worth it only if: (a) the classical solution takes hours and quantum takes minutes (a >10× gain), OR (b) you need a solution quality classical does NOT reach (pharma, materials).

The cost of quantum error remains high. Google Willow achieved <1% error in logical qubits (monumental). But for a calculation requiring 10,000 quantum gates, even 0.1% error per gate is magnified. NISQ (Noisy Intermediate-Scale Quantum) was the term for 2019-2024; now it is “post-NISQ” but with no guarantee of productivity.

The right architectural compass:

The master model is graceful degradation. You don’t build systems that depend on quantum. You build systems that accelerate with quantum if it is available. It is like Redis in classical architectures: if it’s there, use it; if not, the logic keeps working. That’s what quantum is: a complement, not a revolution.

Realistic Horizon 2026-2030

The international academic consensus (Stanford, MIT, Caltech, plus emerging Latin American universities):

• 2026-2027: Operational quantum in finance (portfolio optimization), logistics (VRP), pharma (molecular simulation)
• 2027-2028: Development of cloud-quantum standards and protocols (interoperability between IBM, Google, Microsoft, Amazon)
• 2028-2030: Mass education (a generation of quantum-native engineers emerges from universities that are TODAY installing labs)
• 2030+: Quantum-AI hybrid as critical infrastructure (not exceptional)

Peru is not outside. PUCP, UNI, Norbert Wiener are positioning themselves. In 5-10 years, Silicon Valley teams may be scaling talent from Latin America in quantum + AI.

Conclusion: Using Models as an Expert in the Quantum-Agentic Era

So, how do I integrate this into my orchestration today?

You don’t, fully. Today, in 2026, it is early. But you design with optionality. Architect for three layers: (1) Classical logic + models (the base), (2) Quantum as a selective accelerator (if available), (3) Isolated evaluators at the boundary (validate everything, trust nothing). This is the industrial pattern JPMorgan, Google, Volkswagen, and Maersk are already implementing.

Using models as an expert is not just “calling Opus instead of Haiku.” It is understanding:

• The ecosystem (batching, caching, routing)
• The architecture (isolated AI layers, evaluators at the boundary)
• The near horizon (quantum computing as a selective accelerator)
• The emerging talent (Latin America will produce quantum-agentic engineers in 3-5 years)

The team that masters these four levels runs 10× more efficiently and is prepared for what’s coming. And if you are from Peru, Latin America, or emerging markets: the talent curve is your advantage. You don’t need 20 years of quantum legacy; you can learn quantum-native.

---

Key Takeaways from Chapter 10

• RAG solves access to up-to-date data; it is pure reading. Hidden cost: perpetual maintenance.
• MCP adds secure access to tools. The model doesn’t handle credentials; the protocol interposes guardians.
• Agents bring the closed loop: reflection + action + evaluation. Useful when you don’t know a priori which steps are needed.
• Orchestrations coordinate multiple agents through events. Consumption: 15× tokens vs. a single agent; variance: −80%.
• Multi-agent systems produce emergence: patterns without explicit programming. Operational in finance, logistics, cities today.
• Model adapters (2025): abstract which LLM runs each agent in order to switch without rewriting.
• NVIDIA NIM: Cloud-native microservices for optimized inference. Standard APIs by domain, predictable latency, compliance-ready. Cost-effective at >100K inferences/day.
• NVIDIA NIM Operator: A declarative Kubernetes pattern (CRDs: NIMService, NIMPipeline) for orchestrating multi-service inference. Autoscaling by GPU metrics (KVCache, P99 latency). Zero-downtime rolling upgrades.
• Critical cost: BigData + distributed evaluators = a token bill an order of magnitude higher. Optimization is the 2025-2026 challenge.
• The line of evolution is not “RAG is obsolete → MCP is obsolete → Agents is the end.” It is cumulative: each level resolves a limitation of the previous one, and they coexist in mature architectures.
• Quantum computing is a selective accelerator (3 cases: optimization, sampling, search) for 2026-2030, not a revolution.
• Latin American talent is positioning itself: Peru (PUCP, UNI), Argentina, Brazil, Chile, Colombia are emerging in quantum + AI simultaneously.

---

Worked Exercise 10.1: Redesign Your System (From RAG to Orchestration)

Problem. Your team runs a RAG to handle support tickets (1,000 tickets/month). The success metric is: “ticket closed in under 2 hours.” Today, the RAG generates a solution suggestion with 78% accuracy. The tickets the RAG gets wrong are reviewed by a human, who takes 30 minutes. Monthly budget: $2,000 USD.

Proposal 1 (Status quo RAG):

• Cost: ~$450 (Sonnet inference) + $500 (vector infrastructure) = $950
• Cycle time: 5 minutes (model) + 30 minutes (human when it fails) = 35 min average
• SLA: 78% in <2h (when the model works); 22% fails

Proposal 2 (Agent + Evaluator):

• Agent 1: Retrieves context (MCP → Jira, Slack, docs)
• Agent 2: Generates a candidate solution
• Evaluator: Validates coherence and SLA compliance
• If the evaluator fails, it publishes an event → a human steps in
• Cost: ~$800 (agents + evaluator, plus cycle tokens)
• Cycle time: 8 minutes (agent iterations) + 15 minutes (human, rare)
• SLA: 95% in <2h

Proposal 3 (Light orchestration):

• 3 parallel agents: (1) Retrieves context, (2) Analyzes the ticket pattern, (3) Generates a solution
• An event bus coordinates them: when all three publish results, a synthesizer composes the response
• Evaluator validates
• The human only steps in if the evaluator flags it
• Cost: ~$1,200 (3 agents + synthesizer + evaluator)
• Cycle time: 6 minutes (parallel) + 10 minutes (human, very rare, <5% of tickets)
• SLA: 98% in <1h

Your decision:

1. Which do you pick and why? (Hint: it’s not the cheapest.)
2. At what point would you switch from Proposal 1 to Proposal 2? What is the key metric?
3. Proposal 3 has a better SLA but a cost 25% higher. Is there a floor below which it can’t go? (Hint: yes, and it is the cost of a full-time human.)

Annotated solution:

• The answer depends on your SLA. If it’s “closed in 2h,” Proposal 1 leaves out 22% of tickets. Proposal 2 leaves out 5%, but you pay $250 more. Proposal 3 only comes out ahead economically if you have >5,000 tickets/month (that’s where the parallel efficiency amortizes).
• The switching metric: when the cost of humans reviewing RAG failures (22% × 30 min × salary) exceeds the differential cost of Proposal 2. In the US, that happens when you pass 2,500 tickets/month.
• The floor: a full-time human in support costs ~$5,000-6,000 USD/month. If your orchestration costs that much, it no longer adds economic value. The architecture must generate throughput (more tickets/month) or quality (fewer human escalations) to justify itself.

---

Worked Exercise 10.2: Integrate Quantum Today (Graceful Degradation)

Problem. Your logistics system processes delivery routes. Today you use simulated annealing (classical), which takes 30 minutes per optimal route and costs $0.50. A quantum solver on Amazon Braket could do it in 2 minutes, but costs $40. When do you make the switch?

Solution: The switch isn’t automatic. You need a gate:

If (problem.points > 200 AND problem.complexity > 0.7) {
  Intent: Try Braket (timeout: 2 min, budget: $40)
  If optimal result: use it (5× gain, the $40 is amortized by urgency)
  If timeout: fall back to simulated annealing
} Else {
  Use simulated annealing (cost: $0.50, time: 30 min)
}

The criterion: quantum only when the problem is hard enough AND the time savings are critical for the SLA. For most daily routes (50-100 points), simulated annealing is fine. For emergency routes (250+ points, must be solved in 30 minutes), quantum pays its premium.

This is graceful degradation: quantum is optional, it improves things when it can, and the system survives without it.

---