Jump to another pillar
Pillar 02 of five
What you ship
What runs underneath
VAO · pillar 02 · what runs underneath
Ten layers an orchestration vendor has to choose deliberately, because a wrong choice at one constrains every layer above it.
Named products go stale within a quarter. The layers do not, and neither does the reason each one exists — which is why this is organised by job rather than by logo. Components are graded by how far each has been taken, because a stack diagram where every name carries equal weight tells the reader nothing.
What decides
Holds the graph of steps, the state between them, and the pause-and-resume path when a human has to approve something.
Chosen for explicit state and durable checkpointing: an approval pause is graph structure, not a prompt instruction, and execution resumes in the same frame days later.
One path to every model, so no agent names a provider. Routing by task class and cost, and one model family checking another's work before anything binding.
Two different jobs, deliberately separated: vLLM serves the weights, LiteLLM decides which endpoint a call reaches. Collapsing them is what makes a stack impossible to move off a provider.
Six classes, from frontier reasoning down to small and fast, plus the open-weight class that makes sovereign deployment possible.
Named as classes, never as favourites. At enterprise scale customers mandate the model, so the architectural requirement is that swapping one costs nothing above the gateway. The four open-weight players are set out below.
The arithmetic and the forecasting that no language model is allowed to touch: coverage calculations, how late a supplier runs, whether a confirmation means anything.
Putting a language model on delivery-slip prediction is how a demo becomes an expensive, badly calibrated regression. All of these are permissively licensed, CPU-capable and air-gap-friendly — which the layer above them frequently is not.
Where taught judgement is compiled into a model rather than held in prompts. A platform question and a method question, which are not the same question.
The distinction that matters commercially: offline preference alignment and online environment reinforcement are not one feedback loop. Different data collection, different infrastructure, different cost model — and the platform choice decides neither.
What it knows
Finding the right prior case in years of correspondence, in several languages, where half the meaning sits in part numbers and reference codes.
Never dense retrieval alone. Two reference codes differing by one digit embed almost identically and mean different things, so the lexical half carries more weight here than most reference architectures assume. And changing the embedding model is a project, not a release — it invalidates every stored vector.
Not just the protocol. Who may reach which system, whether this specific call is allowed right now, whether it was logged, and what re-qualifies a tool when its description changes.
A server that follows every design rule at rest is not yet a governed connector layer. The registry answers what exists and who may reach it; the gateway answers whether this call is allowed and whether it was logged. Both are required and neither substitutes for the other. The scanner is what makes a tool-description change trigger re-qualification mechanically, rather than relying on someone noticing in review.
The domain graph, the checkpointed run state, the retrieval indexes, and the versioned artefacts a taught routine ships as.
Never vectorise the graph. Structured relationships are queried by traversal; embedding them returns fuzzy answers to questions that have exact ones. The artefact registry has to be mirrorable, or the air-gapped topology cannot be updated at all.
What proves it
Traces of what each agent saw, chose and cost. Then four classes of test — kernel arithmetic, taught-routine suites, whole-loop runs in a synthetic environment, and calibration.
Instrumented first, not last. Without per-step traces there is no cost-per-resolved-case figure, and the economics become assertions. Calibration is the class that governs the promotion gate — in both directions.
A third party given the trajectories, the methodology and access to the team — which then verifies the score and reviews the hardest cases for leakage and reward hacking.
Not a product to install. Two tiers, never conflated: the vendor builds and runs the harness, the auditor verifies it. The precedent already exists in agentic coding, where a published benchmark plus an independent audit turned an engineering claim into a procurement asset — and the credibility came from being audited on the hardest solves, not from the headline number.
Ask where the data physically runs. Then ask, separately, who could be legally compelled to hand it over. Two questions, two different answers — and conflating them is the most common error in a sovereignty pitch.
| Player | Deployment sovereignty | Legal sovereignty | What it is the right answer to |
|---|---|---|---|
| MistralFR · Apache 2.0 | clears | clears | The default for a French or EU-jurisdiction mandate. On-premise, air-gapped or EU trusted-cloud, and the only one of the four with live defence-sector deployment. |
| Cohere / Aleph AlphaCA · DE | clears | clears | Regulated-sector accounts wanting a production track record rather than a model release. Check which licence variant a proposal names — the earlier one is non-commercial, and the terms are not interchangeable. |
| NVIDIA NemotronUS · open weights | clears | does not | Long-horizon agentic work — the class this architecture actually stresses. Best-documented training data of the four, which strengthens a never-trained-on-your-data commitment when a compliance team asks what the base model itself saw. |
| IBM GraniteUS · Apache 2.0 | clears | does not | Audit-trail and indemnification-first accounts where vendor nationality is not the binding constraint. IP indemnification is a standard term, which none of the other three offer by default. |
Why the distinction has teeth. In June 2026 a frontier model’s access was suspended for roughly three weeks under US export controls, while nothing about where any individual customer ran it had changed. Deployment sovereignty does not protect against that. Only legal sovereignty does — which is why a no-US, no-China mandate rules out two of the four above, regardless of how strong their on-premise story is.
One family is excluded outright. The Llama 4 community licence withholds rights to the multimodal models from any company with its principal place of business in the EU. For an EU rollout that is a disqualification, not a lower ranking. China-origin open weights remain the comparison customers raise, and are not carried here as candidates.
Sovereignty is only real if the deployment topology supports it. Four of them — and the last is the one to design for first even though it sells last, because everything that makes it possible relaxes into the other three. The reverse is a rewrite.
The honest risk in publishing your own benchmark. A vendor-authored benchmark the vendor tops is self-serving by construction, and a sophisticated buyer will say so first. Three mitigations, all of which have to be in place before publication rather than after: open-source the tasks and the verifier so anyone can reproduce the run, commission the audit before publishing, and publish the cases the system loses.
Layers 01 to 03 are replaceable by design — that is the requirement, not an accident of choice. Layers 04 to 10 are where a vendor’s own accumulated work lives, which is why they are the ones worth building carefully and the ones nobody demos.
The stack is not the moat. It is the cost of entry to having one.