Owning the Means of Prediction

The Machine Worth Owning

The most valuable thing in AI is not a model. It is the loop that makes the next model. Use becomes telemetry (the record of what happened), telemetry becomes training, and a better model draws more use. Models depreciate; the loop appreciates. Whoever runs the loop owns the means of prediction.

The labs built that loop in private. Every layer now has excellent open-source versions, so you can build it in the open, on hardware you choose and under keys you hold:

              THE SOVEREIGN LOOP

  a human delegates a task
       │
  [ identity ]     who authorized this?    L8
       │
  [ harness ]      the loop doing work     L4
       │    ├── [ memory + knowledge ]     L5
       │    └── [ tools, via MCP ]
       │
  [ gateway ] ──► [ model ]                L1+L3
       │           weights unlocked only
       │           by your keys, inside
       │           hardware you trust      L2
       │
  [ sandbox + oracle ]                     L6
       │           did it do what it
       │           claims? verify, gate
       │
  [ audit log + telemetry lake ]           L7+L8
       │
  [ post-training, on your traces ]        L7
       │
       └──► a model slightly better at
            your work than yesterday's,
            owned by you  ──► (loop)

  L9 orchestration: the whole machine
     rebuilds from a repo and your keys

Why bother? Because your usage is an asset. Every hour with an AI assistant leaves prompts, corrections, tool traces, and judgments, exactly what post-training feeds on. Picture a restaurant that starts noting which plates come back half eaten. Same cooks, same stove, and next month's menu is better.

Buyers now say this out loud. Banks and defense ministries ask for control of the compute, models, data, and edge, and even cloud sellers advise keeping vendors swappable. When the landlord recommends you buy a house, the market has spoken.

Renting frontier intelligence is a spectacular deal and often the right call. Owning pays most for the defense contractor whose code cannot leave the building, the hospital holding patient records, the law firm mid-acquisition, the startup whose moat is its workflow, and any team that wants its AI spend to leave an asset instead of a receipt. Roll Your Own made the build-versus-buy case for security tooling; this essay aims it at the loop itself.

What Sovereign Actually Means

Sovereignty is not doing everything yourself. It is exit rights, key custody, and owning your telemetry. In a sovereign stack:

• You control the compute. It runs on hardware you own, or rent where confidential computing keeps the provider out of your memory.
• You control the keys. Weights, memories, logs, and telemetry are locked with your keys. Nobody can be forced to hand over what they never held.
• You control the weights. The model is a file you can inspect, fine-tune, roll back, and run with no network at all (air-gapped).
• You control the telemetry. Your usage trains your models. The flywheel spins in your garage.

Notice what is missing: purity. Sovereignty is a dial, not a switch. A laptop running a compressed (quantized) model is fully sovereign and modestly capable. A colocated GPU server is sovereign and far more capable. A confidential cloud VM with keys on premises is mostly sovereign. A frontier API behind a gateway that logs to your lake is barely sovereign, but you kept the telemetry. Set the dial where threat model and budget meet.

In 2026 "sovereign AI" mostly means national programs: NVIDIA's twenty-plus "AI factories" across Europe, Gulf capital in the hundreds of billions, and India's GPU mission with tens of thousands of accelerators. This essay means the same architecture for one organization. If countries call it strategic infrastructure, yours might be too. One prerequisite: if you cannot run Postgres without a managed service, build that muscle first.

The Nine-Layer Map

The stack is nine layers. Each answers one question. Hold the questions in your head and the tool names slot in behind them.

Layer 1  Compute substrate      Where does the model run?
Layer 2  Cryptographic authority Who controls the keys?
Layer 3  Models                  Who owns the weights?
Layer 4  Harness                 What turns a model into a worker?
Layer 5  Memory and knowledge    What does it remember?
Layer 6  Execution oracle        How do you trust what it did?
Layer 7  Telemetry, post-training Who learns from the usage?
Layer 8  Identity and access     Who authorized this?
Layer 9  Orchestration, deploy   How does it reproduce?

Do not build them in this order. The order that works is in the build section: three layers on one box, prove the loop, then harden.

Licenses are listed because here they matter, and my own repositories are labeled with their maturity. Anyone can rent GPUs, but almost nobody has wired the whole loop. That gap is the opportunity, for you as much as for me.

1. The Compute Substrate

Where does the model run? On an inference engine you did not write, on hardware chosen for memory bandwidth. The engine is solved, under Apache-2.0 and MIT, by people better at CUDA kernels than your team needs to be.

From a browser tab to a datacenter

Memory sets what fits. The ladder, as of mid-2026:

WHERE A MODEL CAN RUN          WHAT FITS (4-BIT)

  browser tab (WebGPU)         ~3B to 9B
  phone, edge device           ~2B to 8B
  base laptop, 8 to 16 GB      ~7B to 14B
  desktop, 24 to 32 GB         ~24B to 35B
  workstation, 48 to 128 GB    ~70B to 120B MoE
  big node, 192 GB and up      ~405B to 700B
  ──────────────────────────────────────────────
  on-prem rack                 anything, with headroom
  hosted GPU cloud             anything, by the hour

WebLLM and Transformers.js (both Apache-2.0) run Gemma-class models in a browser tab, so "try it" becomes a link. A phone runs an 8B model offline at 10 to 25 tokens per second under MLC LLM (Apache-2.0) or llama.cpp. A laptop can exercise every layer here.

The key fact: the whole ladder speaks the same OpenAI-compatible API. The layers above cannot tell a desktop running vLLM from an eight-GPU rack or a rented H200, so scaling up is an endpoint change, not a rewrite. The desk is the staging environment for the datacenter.

For production GPU serving, the race is vLLM (Apache-2.0, PyTorch Foundation) versus SGLang (Apache-2.0). vLLM is the default, with 200+ architectures and the widest hardware coverage (AMD, Intel, TPU), the only serious escape hatch from NVIDIA. SGLang reuses shared prompt prefixes and runs 15% to 30% faster on agent workloads, so benchmark both.

For laptops and offline boxes, use llama.cpp (MIT): zero dependencies, every major GPU backend, and a built-in OpenAI-compatible server. Ollama (MIT) wraps it and is the laptop default. On Apple Silicon, MLX (MIT) is 15% to 25% faster and also fine-tunes. Two specialists: ExLlamaV3 (MIT) keeps a 70B model coherent at 1.6 bits per weight in under 16 GB of VRAM, and KTransformers (Apache-2.0) splits mixture-of-experts models across GPU and CPU so a big-RAM workstation can run 671B-class models.

Hardware, honestly

One fact drives every choice: token generation speed is bounded by memory bandwidth, not compute. The model re-reads all its weights for every token. Picture a chef who must walk the whole pantry for each word of a recipe: the width of the pantry door matters more than how fast the chef chops. So a 1 TB/s GPU runs about four times faster than a 256 GB/s machine, whatever the FLOPS charts say. Capacity sets what fits; bandwidth sets how fast it talks.

• Around $2,500: a 128 GB AMD Strix Halo mini PC, an M4-class Mac, or a used RTX 3090/4090 box. Runs 32B models quickly, a quantized 70B at 12 to 15 tokens per second, and 120B-class MoE well.
• Around $5,000: an RTX 5090 (32 GB at 1.8 TB/s, fastest under 30B), NVIDIA's DGX Spark (128 GB, but slow at dense 70B), or a Mac Studio M3 Ultra (96 GB at 819 GB/s, the balanced pick). Apple pulled its 256 GB and 512 GB Mac Studios in spring 2026 amid the RAM squeeze, ending "DeepSeek on a Mac" for new buyers.
• $10,000 to $15,000: two to four RTX 5090s, an RTX PRO 6000 Blackwell (96 GB), or used A100 80GB cards at $4,000 to $9,000 each. Runs 70B at FP8 for a department, though A100s predate confidential computing and cannot attest.
• Around $50,000: an eight-GPU box like the tinybox pro v2, or eight used A100s. Quantized 400B to 671B-class models for hundreds of users.
• $250,000 and up: 8x H100/H200/B200 nodes. Frontier open models at full precision and production concurrency. Used H100s trade at $15,000 to $28,000 and falling.

Every tier has a rented twin, from hyperscalers, cheaper specialist clouds (CoreWeave, Lambda, Nebius, and regional players), and confidential instances with attestation. Rent for spiky use or burst training; buy for steady load. The stack is identical either way, so the choice is reversible.

For multi-node serving on Kubernetes, llm-d (Apache-2.0, CNCF Sandbox, backed by Red Hat, Google, IBM, and CoreWeave) is vendor-neutral and NVIDIA Dynamo (Apache-2.0) is NVIDIA-tuned. The charming wildcard, exo (Apache-2.0), has run 671B models on Mac Studios linked over Thunderbolt: a demo of the thesis, not production.

The leftover dependency is NVIDIA's proprietary CUDA. AMD ROCm is a real escape hatch that costs some speed and a few debugging evenings. And local inference needs no platform team: my enklayve-ai (MIT, early, v0.1.5) is a small Rust desktop app running Qwen and Llama fully offline.

2. Cryptographic Authority

Who controls the keys? You do, in a hierarchy you can draw on a whiteboard, with hardware at the root. This is where "we run it ourselves" becomes "no one else can read it, even with our hardware in hand."

The backbone is OpenBao (MPL-2.0), the Linux Foundation fork of HashiCorp Vault that now out-ships the original. It encrypts on request, so raw keys never leave it. Around it: age (BSD-3) encrypts files, even huge weights; SOPS (MPL-2.0, CNCF) keeps secrets in git; step-ca (Apache-2.0) is a private certificate authority so every service speaks mutual TLS; and libsodium (ISC) supplies the primitives. At the root, a $650 YubiHSM 2 holds your CA root and OpenBao's unseal keys, so all trust traces to an object in a safe. Cloud KMS is fine as one wrapping layer, never the sole authority.

The pattern that ties the stack together

The core design is envelope encryption with attestation-gated release. Think of a locked box inside a safe. The weights are locked with a random data key (AES-256-GCM). That key is locked again by an HSM-sealed OpenBao master key, and the safe opens only for a machine that has cryptographically proven what it is. Plaintext weights exist only inside a verified enclave.

That proof is confidential computing, and it is no longer exotic. AMD SEV-SNP and Intel TDX encrypt a VM's memory so even the host cannot peek, and NVIDIA Confidential Computing on H100, H200, and Blackwell extends that to the GPU at roughly 2% to 5% overhead. Germany's national health record runs on it, and the same vendor's Privatemode offers confidential H100 inference. Moxie Marlinspike's Confer launched consumer attested inference in December 2025; by March 2026 it was going into Meta AI. Apple's Private Cloud Compute published the canonical design. The pattern won.

To run it, bet on Confidential Containers (Apache-2.0, CNCF Incubating). It puts each Kubernetes pod in its own trusted execution environment (TEE) and ships Trustee, a key broker that checks attestation before releasing secrets. Veraison (Apache-2.0) runs your own attestation verifier. Without TEE hardware, Keylime (Apache-2.0, CNCF) gives TPM-based boot attestation, and LUKS disk encryption is baseline hygiene. Graveyard file: Enarx is dormant, and Constellation, the first confidential Kubernetes, is in maintenance mode under the Business Source License (BUSL).

Running attestation yourself, honestly tiered

The quiet part: full attestation-gated release only works on specific silicon, and most layer 1 tiers lack it. It needs an H100, H200, B200, GB200, or RTX PRO 6000 Blackwell on an SEV-SNP or TDX host. On consumer RTX cards, A100s, Apple Silicon, and Strix Halo, weights sit unencrypted in GPU memory, and LUKS, Keylime, and a locked door are your protection. So attestation comes in three tiers:

• Boot attestation only, for consumer and A100 builds: protect the disk, measure the boot, own the room.
• Rented confidential GPUs (Azure's NCC series, GCP's confidential A3, specialist clouds), the easiest path. You verify the enclave and release keys from your own OpenBao, so the provider never sees plaintext.
• On-prem confidential GPUs, still frontier work. Dell, ASUS, and Cisco ship CC-validated systems, NVIDIA's nvtrust (Apache-2.0) verifies GPUs locally, and you mirror reference measurements and revocation lists yourself. Host support only reached stock Ubuntu in 25.04 and 25.10, and NVIDIA's reference design uses a proprietary key broker where this essay puts Trustee. The all-open path exists, but you assemble it.

TEEs shrink what you must trust but do not empty it: you still trust the silicon and microcode, and side-channel research continues. Attestation raises the cost of a breach from "subpoena the cloud provider" to "break the hardware." A very large raise, not a royal flush.

3. The Models

Who owns the weights? You do, the moment you download the file. What you cannot download is knowledge of what went into it.

A delicious irony: Chinese labs lead the leaderboard with the cleanest licenses. DeepSeek V4 (MIT) is the strongest open coder, Qwen (Apache-2.0) the widest family (0.6B to 397B, coder variants at every size), GLM-5.2 (MIT) the best on long coding tasks, and Kimi K2 (modified MIT) the agentic specialist. Meta's Llama 4, meanwhile, has a 700M-user cutoff and bars EU companies from its multimodal models. Read licenses, not flags.

The geopolitical worry has a straight answer: a weight file is static, runs air-gapped, and cannot phone home, unlike a foreign lab's hosted API. What you cannot audit is the training data:

• Weights only: DeepSeek, GLM, Kimi, and most of the leaderboard. Top capability, opaque origins.
• Weights plus recipes and data: NVIDIA's Nemotron 3 and, at the top, AI2's OLMo 3 (Apache-2.0, with its full 5.9T-token corpus, code, checkpoints, and logs), where "audit every layer" is literally true.

Also: OpenAI's gpt-oss-120b (Apache-2.0), a US-made reasoner on one 80 GB GPU. Google's Gemma 4, now Apache-2.0, owns the small tier. Mistral's Devstral Small 2 (Apache-2.0) is the best coder for one 24 GB GPU and the European pick. IBM's Granite 4.1 brings ISO 42001 certification and signed weights.

Treat weights like the supply-chain artifacts they are

Three cheap habits. Refuse pickle, a format that can run any code when it loads (malicious checkpoints turn up on hubs regularly); safetensors (Apache-2.0, security-audited) stores only numbers. Sign what you deploy: OpenSSF Model Signing (Apache-2.0, v1.0 since April 2025) signs a whole model directory and checks it at load; NVIDIA already uses it. Version models like containers: ORAS or KitOps (both Apache-2.0, CNCF) into Harbor (Apache-2.0, CNCF Graduated), signed with cosign. Your registry, your rollback.

For embeddings, the number fingerprints behind search, BGE-M3 (MIT) is the default, Qwen3-Embedding (Apache-2.0) the accuracy leader, and nomic-embed v2 (Apache-2.0, training data published) the auditable choice. All run on a CPU. Beyond text, whisper.cpp (MIT) hears, Kokoro (Apache-2.0) speaks, and Qwen and Gemma vision variants read documents. On the output side, C2PA's late-2025 revision signs generated text, timely as the EU AI Act's transparency rules take full effect in August 2026.

4. The Harness

What turns a model into a worker? A loop. The harness gives a model tools and context, runs what it asks for, and repeats until the job is done. Claude Code defined the category; open tools caught up fast. Your telemetry is born here.

Read mini-swe-agent (MIT), Princeton's benchmark-winning agent in about 100 lines of Python: a prompt, a shell, and a while-loop. A harness is not magic. It is a loop with taste.

The one test: can the harness point at any OpenAI-compatible endpoint? If not, it is a rental. The tools that pass, as of July 2026:

• OpenCode (MIT): terminal-native, 75+ providers, the most-starred coding agent on GitHub. The default pick.
• OpenHands (MIT): autonomous, with the strongest sandboxing.
• Cline (Apache-2.0) for the IDE, goose (Apache-2.0, Linux Foundation) for automation beyond code, Aider (Apache-2.0) for surgical git edits (releases have slowed), and gptme (MIT) for offline purists.

Two connectors finish the layer. The Model Context Protocol (MCP) is a universal plug for AI tools: build a tool once and it survives any swap of model or harness. It now lives at the Linux Foundation's Agentic AI Foundation (since December 2025), backed by Anthropic, OpenAI, Block, Google, and Microsoft. A gateway, usually LiteLLM (MIT), gives one endpoint with keys, budgets, and logging, routing to local vLLM by default and to a frontier API by choice. RouteLLM (Apache-2.0) showed that sending only hard queries upstream kept about 95% of quality at a fraction of the cost. The gateway is also your policy chokepoint.

My own entry, codelicious (MIT, early but well-tested), is a headless CLI that turns markdown specs into pull requests, with builders in isolated git worktrees, parallel reviewers, and one fixer. It proves the pattern; it does not compete with OpenCode.

5. Memory and Knowledge

What does it remember? Whatever you save outside the context window, in stores you own. Models are stateless: every session starts from zero, like a brilliant contractor with amnesia. An assistant's memory of your architecture and decisions is a database someone built, and if the vendor built it, it is theirs. Memory compounds, so this is prime real estate.

Memory comes in two shapes. The first is recall: facts and episodes fetched by similarity. Mem0 (Apache-2.0) consolidates facts across sessions. Letta (Apache-2.0), the productized MemGPT paper, runs agents that curate their own memory. Graphiti (Apache-2.0) is a knowledge graph that knows what was true then as well as now. Underneath, boring defaults: pgvector in your existing Postgres, or sqlite-vec plus SQLite FTS5 for hybrid search in one file. Reach for Qdrant (Apache-2.0) only when volume demands it.

The second is structure: what is true about this codebase right now. Code embeddings do not know who calls whom. My OpenLore (MIT), the one repository of mine here that has earned the word production (42 releases, more than 5,500 tests, no LLM in the hot path), keeps a deterministic code graph. An agent's orient(task) call gets the relevant functions, callers, specs, and insertion points in about 430 microseconds, replacing tens of thousands of tokens of file reading. In its own benchmarks, losses included, it cuts agent round-trips by 26%. A model that never has to guess your architecture spends its brainpower on the change itself. Scaffolding is how a 32B model punches above its weight.

Getting knowledge in

Skip careful ingestion and retrieval (RAG) fetches beautifully from garbage. The 2026 default parser is Docling (MIT, LF AI & Data Foundation), which reads PDF, Office, and audio locally. Around it: unstructured (Apache-2.0 library, open-core company) for breadth, Apache Tika (Apache-2.0), the thousand-format workhorse that will outlive us all, Firecrawl (AGPL, self-hostable) for web-to-markdown, and SearXNG (AGPL), self-hosted search that keeps your interests private. Ingested documents are the main carrier of prompt injection, so treat ingestion as a trust boundary. Encrypt the memory store with layer 2 keys and log access, because "what did the agent know and when" will eventually be asked, possibly by a lawyer.

6. The Execution Oracle

How do you trust what it did? You do not trust. You verify against the one oracle that is free, exact, and incorruptible: the computer itself. A model predicts; a computer executes. When a model says "the tests pass," you do not argue. You run the tests.

The boundary. Isolation is a ladder. Plain Docker or Podman stops accidents, not hostile code sharing your kernel. gVisor (Apache-2.0), which catches system calls before they reach the kernel, is the easiest strong upgrade. At the top, Firecracker (Apache-2.0), the microVM behind AWS Lambda, boots in about 125 milliseconds, and Kata Containers (Apache-2.0) gives each container its own VM with GPU passthrough. Self-hostable platforms add an API: microsandbox (Apache-2.0), whose MCP server lets agents spawn microVMs, Daytona (AGPL-3.0), and E2B (Apache-2.0, a real effort to self-host). On Kubernetes, the SIG-official Agent Sandbox with warm pools is the emerging standard. On laptops, Anthropic and OpenAI separately landed on the same open-source design: Anthropic's sandbox-runtime and Codex's sandbox both use macOS Seatbelt and, on Linux, bubblewrap with seccomp and Landlock.

The eyes. Falco (Apache-2.0, CNCF Graduated) uses eBPF to turn system calls into alerts like "agent spawned a shell, wrote outside its workspace, opened an outbound socket," with 2026 releases aimed at coding agents. Tetragon (Apache-2.0) blocks them in the kernel. Falco is the smoke detector; Tetragon is the sprinkler.

The policy. Open Policy Agent (Apache-2.0, CNCF Graduated) writes "may this agent call this tool with these arguments" as code. Cedar (Apache-2.0), AWS's formally verified authorization language, is rising for rules you can analyze.

The content wall. The top 2026 attack is indirect prompt injection: instructions hidden in a page or file the agent reads. Small local guard models screen for it. Qwen3Guard and IBM's Granite Guardian are Apache-2.0, the sovereign picks over Meta's Llama Guard and its community license. NeMo Guardrails (Apache-2.0) runs input, output, and tool rails, and Meta's LlamaFirewall adds a strong injection detector. These filters are probabilistic, but they raise the attacker's cost at every trust boundary.

The network wall. A hijacked agent wants your data out. The defense is boring and absolute: deny all outbound traffic, allowlist through a proxy you control, and filter DNS yourself, as both labs' sandboxes already do. On Kubernetes, Cilium policies say "this sandbox may reach the package mirror and the git remote, and nothing else," and Tetragon enforces it. An agent that can read secrets but not reach the internet is an embarrassment; one that can do both is a breach.

Why "oracle," not "sandbox"? Because a guardrail you have not tested is a guess. My Verisim (MIT, research-stage, with a working product in verisim-certify) turns the guess into a measurement. It fires harmful commands past your guardrail at a sealed real shell and certifies coverage with confidence intervals. A hand-rolled denylist, the kind every team writes first, stopped 6 of 72: 8.3% coverage. A hardened hook reached 75%. Certify, do not assert. My proxilion-mcp (MIT, early) intercepts agents' tool calls at a gateway, and invariant (MIT, pre-alpha, formally verified in Lean 4) checks AI commands to robots and biosynthesis equipment against safety profiles before they run. Prototypes, plainly labeled.

The trade-off is friction: an agent that can do nothing dangerous can do nothing useful. Use graduated autonomy: a wide-open sandbox, tight policy at the boundary, and human gates only at steps you cannot undo, like deploys, pushes, payments, and deletions.

7. Telemetry and Post-Training

Who learns from the usage? Whoever holds the traces, and there is no reason it should not be you. Captured, every session compounds; uncaptured, it evaporates. Five stations, all commodity now.

Capture. Instrument once with OpenLLMetry (Apache-2.0): vendor-neutral OpenTelemetry spans for every model call and tool use. The center of gravity is Langfuse (MIT core): tracing, scoring, datasets, and export straight to fine-tuning files. ClickHouse acquired it in January 2026, so write down the exit path.

Store. Traces land in S3-compatible storage you run. The old default, MinIO, is now archived, a story told in License Weather. Use SeaweedFS (Apache-2.0) for scale or Garage (AGPL, fine internally) for three small nodes.

Curate. Raw traces are not a dataset; Argilla (Apache-2.0, Hugging Face) is the labeling bench. The cheapest gold: each time a human accepts or rejects an agent's work, that is a preference pair, exactly what direct preference optimization (DPO) eats. You already make these labels. You just do not keep them. My agent-replay (MIT, very early) records, replays, and diffs runs and exports golden regression datasets. My origin (MIT, a sketch) tracks sample fingerprints and licenses through training, so you can prove what went into a model.

Train. The numbers are smaller than most think. LoRA trains small add-on adapters, about 1% of parameters, like clip-on lenses for a camera you already own. QLoRA does it on a 4-bit base with a quarter of the VRAM. GRPO, behind DeepSeek-R1, needs only a checkable reward like passing tests. Behavior shifts from about 1,000 curated traces; 5,000 to 50,000 is the practical band. A QLoRA fine-tune of a 32B model on 20,000 traces runs overnight on two consumer GPUs, or for about $5 an hour rented. Tools: Axolotl (Apache-2.0, YAML configs in git), Unsloth (twice as fast on one GPU; check its dual license), LLaMA-Factory (zero-code), TRL (reference code), and verl (Apache-2.0, GRPO at scale).

Gate. No fine-tune ships without evals, because a model can get better at your niche and quietly worse elsewhere, safety included. My securifine (MIT, early) measures that slide for security tuning. The kit: lm-evaluation-harness (MIT) for general capability, promptfoo (MIT) to A/B against the base and red-team with 500+ attack vectors, and DeepEval (Apache-2.0) as a pytest-style CI gate. For coding agents, build a private benchmark from your own resolved issues: uncontaminated and truer than SWE-bench, whose hardest problems a 2026 audit found riddled with flawed tests.

The hard part is people. Recording engineers' sessions raises privacy and works-council questions. Decide what you capture, tell people, scrub personal data with Presidio (MIT, Microsoft's self-hosted PII redaction) before it lands, and encrypt under layer 2 keys. EU homework: under the AI Act, enforced for general-purpose AI from August 2, 2026, heavy post-training can make you a downstream "provider." LoRA fine-tunes fall far below the line, but know where it is. Sending data to a vendor does not make it more private, only less visible.

8. Identity and Access

Who authorized this? Every agent action should trace back to a human through a chain you can verify later. Agents revive security's oldest problem, the confused deputy: a helper with valid credentials doing what the boss never meant. An agent holding your OAuth token is the most enthusiastic confused deputy ever built. The 2026 consensus has four self-hostable pieces:

• Humans: single sign-on via Keycloak (Apache-2.0, the battleship) or Authentik (MIT core, easier). Passkeys for admins are the cheapest high-leverage control in this essay.
• Workloads: SPIFFE/SPIRE (Apache-2.0, CNCF Graduated) answers "what is this agent?" with attested, auto-rotating identities and free mutual TLS. Caveat: sub-agents spawned on the fly strain its pre-registration model.
• Permissions: OpenFGA (Apache-2.0, CNCF), in the lineage of Google's Zanzibar, models "agent A, acting for user U, may read document D," so retrieval never leaks documents to people who could not open them.
• Delegation: Biscuit (Apache-2.0) tokens can be narrowed offline by any holder, like a hotel key card recut for one room and one night. A parent agent hands a sub-agent one repository for one hour, enforced by cryptography, not hope.

Authority that only shrinks as it flows mattered enough that I spent a year on it. My proxilion (MIT, Rust, pre-release) is a self-hosted OAuth proxy between agents and the SaaS APIs they call, binding each request to a cryptographic chain rooted at a human. It builds on the PIC protocol (provenance, identity, continuity), designed by Nicola Gallo and the PIC Protocol team, whose reference implementation I maintain at provenance (MIT). Its two rules: every action traces to an origin that cannot change across hops, and authority can only narrow. Both are prototypes at the right enforcement point.

Finally, an audit log that an attacker or embarrassed insider can edit is a story, not evidence. Trillian (Apache-2.0), the Merkle-tree log behind Certificate Transparency, is the fully open choice for append-only, verifiable history. Apple publishes Private Cloud Compute measurements to such a log; yours can record every agent action and key release.

9. Orchestration and Deployment

How does it reproduce? A sovereign stack you cannot rebuild is a snowflake with a flag on it. This least glamorous layer is the power to rebuild everything from a repository and a set of keys.

Start smaller than feels professional: one Docker Compose file. harbor (Apache-2.0, the av/harbor CLI, not the registry) spins up a pre-wired local stack with one command. Coolify (Apache-2.0) is a self-hosted Heroku with one-click Ollama and LibreChat (MIT, the no-strings chat frontend). Move to k3s (Apache-2.0) when you need Agent Sandbox, Kyverno admission policies ("every agent pod must run in gVisor" in three lines of YAML), and Cilium's network policy.

Provision with OpenTofu (MPL-2.0, Linux Foundation), the Terraform fork now ahead in places like state encryption. Fidelity moved more than 50,000 state files to it. The fork worked, which is the point of forks.

Observe with Prometheus and Grafana, or VictoriaMetrics (Apache-2.0) at scale, with OpenTelemetry GenAI traces end to end. Back up with restic (BSD-2), encrypted and offsite, and rehearse the restore: a backup you have never restored is a rumor. Base weights can be downloaded again; your telemetry lake and adapters exist nowhere else on earth. My policybind (MIT, proof-of-concept) sketches AI policy as YAML: allow and deny rules across nine providers including local Ollama, with scoped tokens and budgets.

An air-gapped stack is fed, not found. Images and models arrive as cosign-signed bundles via Harbor replication or ORAS on removable media, packages come from your mirrors, and attestation collateral is cached inside, as NVIDIA's docs require. The test: unplug the uplink for a week. Everything that breaks was a dependency you had not admitted to.

The deepest version is Nix: bit-for-bit reproducible builds, the infrastructure twin of attestation. But CUDA-in-Nix is a known source of lost weekends. Adopt it last, if at all.

Making It Work: Three Builds

Do not build nine layers at once. Build three, prove the loop, then harden. Here is one system at three sizes. Like a sourdough starter that moves from a jar to a bakery, each grows into the next with a redeploy and new config, on a rack you own or a cloud you rent.

Build one: the desk ($2,500 to $5,000)

One machine with lots of unified memory: a Strix Halo mini PC, a Mac, or a used-GPU box. Ollama serving a Qwen3 32B-class coder. OpenCode or Cline pointed at localhost. SQLite with sqlite-vec, plus OpenLore for code. Seatbelt or bubblewrap sandboxing. Langfuse capturing every trace from day one, though training is months away. Backups encrypted with age.

This proves the whole protocol: encrypted local inference, memory, sandboxing, and owned telemetry. Everything else is scale-out and hardening, except attestation, which needs Hopper-class silicon. Starting here is a sequencing decision, not a toy one.

Build two: the closet ($15,000 to $50,000)

A GPU server (two to four RTX-class cards or an eight-GPU box), owned, colocated, or rented until usage proves out. vLLM serving one good large model at FP8 behind LiteLLM. Postgres with pgvector, OpenBao, Keycloak, sandboxes on k3s with gVisor, Falco watching, and Trillian recording. Traces flow to Langfuse and SeaweedFS. Each quarter an Axolotl job and the eval gate make the org's model a bit better at the org's work. This is the smallest build where the flywheel visibly spins, for $500 to $2,500 a month.

Build three: the room ($250,000 and up)

8x H100/H200 nodes, or confidential-GPU cloud instances. Frontier open models (DeepSeek V4, GLM-5.2, Kimi K2) served across nodes with llm-d. Confidential Containers with attestation-gated key release. SPIFFE on every workload, OpenFGA on every resource, and verl training against your test suites. For organizations whose regulator, or alpha, demands it all: the dial turned to the end.

The order of operations

1. Inference plus harness. Get a model answering and a loop editing files.
2. Memory plus telemetry capture. Data kept from week one is the asset everything compounds on.
3. Sandbox plus policy. The boundary and the eyes.
4. Keys plus identity. Secrets into OpenBao, SSO in front, stores encrypted at rest.
5. First fine-tune plus eval gate, once curated traces pass about a thousand.
6. Attestation, TEEs, and multi-node, when the threat model or headcount demands it.

License Weather

"Open source today" is not sovereignty. Governance is. A license file is today's weather, not the climate. In about three years:

• Terraform, Vault (in 2023), Nomad, and immudb moved to BUSL. Vault then went to IBM, which bought HashiCorp for $6.4B in 2025.
• Zitadel and Grafana moved to AGPL, and Open WebUI to a custom branding license. Redis and Elasticsearch changed course earlier.
• MinIO went from Apache to AGPL in 2021, lost its admin UI in 2025, stopped shipping binaries, and was archived in April 2026.
• Google replaced its 104,000-star open Gemini CLI with a closed successor in June 2026. Roo Code archived itself in May, Continue.dev went read-only in June, and Hugging Face archived Text Generation Inference, once a default, in March.
• MiniMax shipped M2.7 open-weight, then quietly required written authorization for commercial use.

Most were reasonable business decisions. All stranded people who mistook a license file for a promise. Among coding agents, the survivors are permissive, terminal-first, and model-agnostic; the casualties were tied to one vendor's strategy.

The defenses are cheap. Prefer foundation-owned projects (CNCF, Linux Foundation, Apache), since a foundation cannot pivot to enterprise pricing. This essay's strongest picks (Kubernetes, OpenTofu, OpenBao, SPIFFE, Falco, in-toto, MCP, Prometheus) all are. Archive each license, for code and weights alike, the day you adopt. Keep one page per layer naming your exit: fork, alternative, or export path. And when a rug-pull comes, the community routes around it. OpenBao and OpenTofu are both healthier than their parents, maybe the most optimistic fact in this essay.

What You Give Up

Ownership costs capability, time, and attention. Pretending otherwise would betray the whole argument. Here is the bill.

Raw model quality. A local open model is not a frontier model. On long-horizon agentic work the gap is something like 25% to 30%, and anyone who says otherwise is selling something. Scaffolding, routing the hard 5% of queries to a frontier API, and post-training on your telemetry win much of it back. The rest is the price of ownership, the way a house needs maintenance a hotel does not. If your work lives at the very edge, rent, gateway-and-log, and revisit yearly. The gap has moved in only one direction since 2023.

Integration labor. Nine working layers are not a working system. The vendor's real product was the wiring, and now it is yours, down to the 2 am page when the GPU box wedges. Budget part of an engineer permanently.

The newest shiny thing. Open versions of frontier features arrive six to eighteen months later. If you need the newest thing the month it exists, sovereignty is a tax on that.

The arithmetic, worked once. Forty engineers at $150 per seat per month is $72,000 a year, forever, growing with headcount. The closet build is about $30,000 of hardware over three years, $500 to $1,000 a month in power and colocation, and a fifth of an engineer, perhaps $40,000 a year. Call it $60,000 a year, flat while headcount doubles. The savings are modest; the asset is the point. The owned $60,000 builds a telemetry corpus and a better model; the rented $72,000 leaves nothing behind. Two caveats: an idle GPU loses value while unbought tokens cost nothing, and an eight-GPU H100 node draws about 10 kW, a colocation contract, not a closet.

What you get back: costs that stop scaling with usage, data that never reaches a vendor, compliance stories that survive an auditor, immunity to price hikes and retirements, and a model that keeps learning your work while everyone else's stays generic. For a growing class of organizations that clears easily. For the rest, a credible exit still changes the negotiation: the price of everything rented is set by the cost of owning.

The Flywheel Is the Point

The stack is not nine tools. It is one loop. An agent acts, an oracle checks it, the trace lands in your lake, and the next model learns from it. Thanks to thousands of open-source maintainers, that loop is no longer proprietary. Every component has a name, a license, and a repository, and most fit on a machine that fits on a desk.

Start with the desk. Capture traces from day one. Turn the dial as needs harden. The intelligence is rentable; the learning should be yours.