• 8 min read

Self-Hosting AI Coding Agents: On-Premise Beats the Cloud

tl;dr

Full self-hosting wins only when GPU utilization stays above break-even. For most teams, a split architecture with cloud-managed inference and on-prem execution is the economically rational choice.

Featured image for "Self-Hosting AI Coding Agents: On-Premise Beats the Cloud"

The per-seat AI coding bill has quietly become a per-token bill, and the gap between sticker price and actual spend is where budgets break. That disconnect is why more teams are asking whether they can run the agent loop, the model, and the execution sandbox entirely on infrastructure they control. The answer depends on what “self-hosted” actually buys you, and whether your team can keep GPUs busy enough to justify the fixed cost.

What does “self-hosted” actually mean for a coding agent?

Most products self-host one layer and market it as the whole thing. An agent has three parts that can each live in a different place: the loop that plans and calls tools, the inference that generates code, and the trigger that hands work over. Claude Code lets you route inference through your own AWS or Google account while the trigger stays on GitHub. Cursor runs the agent loop and planning in its cloud while a local worker handles file edits and commands. Only a few projects — OpenHands, Coder Agents, and IBM Bob — put all three layers on your infrastructure, and even they depend on a model provider unless you run local weights.

The distinction matters because compliance reviews often bundle two different requirements into one phrase. One requirement is a network boundary: source code and prompts cannot leave your VPC. The other is vendor independence: the system keeps running if the vendor changes course or shuts down. Coder satisfies both — you operate the control plane, the database, and the provisioner on your own hardware, and the AGPL-3.0 license means your installation survives a vendor pivot. Ona’s Enterprise runners execute code in your VPC but keep the orchestration plane, so a security reviewer accepts the network boundary while the continuity requirement goes unmet. IBM Bob’s self-hosted deployment, which reached general availability on October 1, 2026, supports air-gapped environments and is already used by 80,000+ IBM developers internally, making it one of the few platforms that satisfies both constraints at enterprise scale.

How do the costs compare: self-hosted vs. managed APIs?

The economics flip on GPU utilization, not headcount. Anyscale’s cost model shows self-hosted coding agent serving with Ray and vLLM at about $2,920 per month for always-on serving — roughly $58 per registered developer — or about $840 per month for work-hours-only serving at $17 per developer, assuming 50 developers sharing one RTX PRO 6000 at $4 per GPU-node hour. The equivalent API baseline for an active developer runs about $800 per month in Claude Code API costs. For a 50-developer team, that’s roughly $35,040 per year in infrastructure versus $480,000 per year at the API baseline. BMW found the cost-benefit of moving from API to in-house inference with as few as 8 developers.

But the comparison isn’t symmetric. LiteLLM’s internal Moyai deployment cost about $700 per day — roughly $21,700 over 31 days — against a $101,872 monthly Devin bill, an estimated 79% reduction, though they characterize this as an internal comparison rather than a matched-workload benchmark. On the autocomplete side, Tabby’s analysis shows 50 developers on GitHub Copilot Business costs $11,400 per year at $19 per seat, while a dedicated self-hosted GPU box costs roughly $3,000 once; breakeven occurs at roughly a dozen seats on spare capacity or roughly thirty seats on a dedicated box. Omid Saffari’s model prices a managed Vercel eve deployment at about $182 per month for one developer and 100 tasks, versus $3,448 for 100 hours on an eight-H200 self-hosted GLM-5.3 cluster before operations — self-hosting clears its GPU cost only above 21.28 task equivalents per paid cluster-hour.

ToolPricing modelSelf-hosted scopeTarget audience
OpenHandsFree (MIT), Teams $499/mo up to 10 seatsFull stack: loop, inference (BYOK/local), sandboxTeams wanting complete control with MIT license
TabbyFree (Apache 2.0), self-hosted onlyAutocomplete + chat, single Docker imageTeams replacing Copilot with owned GPUs
IBM BobEnterprise (Bobcoins), customFull lifecycle: planning, coding, testing, modernizationRegulated enterprises needing air-gap + governance
Coder AgentsCommunity free, Premium quote-onlyFull stack + Agent Relay for cloud agentsOrgs already on Coder platform, air-gap mandates
OpenCodeFree (MIT), optional Go $10/moTerminal agent, 75+ providers including localIndividual devs and teams wanting model flexibility
MoyaiFree (open source), runs on your infraCloud agent loop + LiteLLM gatewayTeams already using LiteLLM, Slack-driven workflows

Which tools actually run fully on your infrastructure?

OpenHands is the reference implementation: MIT-licensed, 80,000+ stars, and a real agent platform with web UI, CLI, SDK, and editor integrations over ACP for Zed, JetBrains, and VS Code. Running it yourself is a one-liner — uv tool install openhands --python 3.12 and openhands serve, or a docker run against the published image. It scores 68.4% on SWE-bench Verified using Claude Opus 4.6 with the CodeAct v3 scaffold while running entirely self-hosted. OpenCode takes a different angle: an MIT-licensed terminal agent with 187,000+ GitHub stars and connectivity to 75+ model providers including locally hosted models, making it the most model-agnostic option for developers who live in the terminal.

Coder Agents became generally available on September 9, 2026, with the whole stack running on your infrastructure including air-gapped networks. Their Agent Relay product, launched with SpaceXAI, lets Cursor Cloud Agents execute inside Coder workspaces on your network while Cursor continues to run the agent loop — a hybrid that satisfies the network boundary without requiring you to operate the inference stack. IBM Bob’s self-hosted option decouples the entire inference layer from IBM’s SaaS, letting banks, defense contractors, and healthcare systems run a full-lifecycle agentic platform inside their own firewalls with bring-your-own-license model flexibility. Moyai, open-sourced by LiteLLM in October 2026, gives you a cloud coding agent that runs on your infrastructure and points at your LiteLLM gateway, so you control the model routing and see all usage in your own dashboard.

What’s the operational burden you’re signing up for?

Owning the weights does not supply the pieces you need to run a production agent platform. A self-hosted GLM-5.3 coding agent still needs a harness around the model: an inference team, an agent runtime, a sandbox boundary, observability, and enough concurrent work to keep expensive accelerators busy. The managed Vercel eve path gives you checkpointing, approvals, isolated sandboxes, subagents, and evals out of the box. With self-hosting, you assemble and operate the agent layer yourself. That’s not a small commitment — Coder’s CEO noted that building an agent isn’t the hard part; running agents safely and reliably requires careful management of models, tools, repositories, dependencies, context, and guardrails.

The infrastructure complexity is real. Self-hosting Temporal, Restate, Inngest, DBOS, LangGraph, or Trigger.dev for agent orchestration ranges from one Postgres database (DBOS) to four Temporal services plus a database and Elasticsearch, or ten containers for Trigger.dev v4. For coding agents specifically, you’re adding a model serving stack (vLLM, Ollama, or llama.cpp), a routing layer (LiteLLM or similar), sandbox infrastructure (Docker, microVMs, or Kubernetes), and the agent harness itself. Teams that set expectations based on a benchmark score rather than a measured rollout tend to be disappointed when they measure daily delivery throughput. The gap between lab conditions and shipping code is the first thing to plan around, not the last.

When does self-hosting make economic sense?

The Utilization Gate framework I’ve observed: self-hosted infrastructure has a flat cost curve that only beats per-token and per-seat pricing when concurrent agent work keeps accelerators busy above a break-even point. If your team cannot sustain enough concurrent agent work to keep GPUs utilized above that threshold, a hybrid model with a cloud-managed loop and on-prem execution is the only economically rational choice. For autocomplete workloads, Tabby shows Copilot cheaper than a dedicated self-hosted box at small team sizes without agent overage. For autonomous agent workloads, the crossover moves higher — Omid Saffari’s 21.28 task equivalents per paid cluster-hour is a concrete threshold.

Gartner projects that 80% of enterprise software engineers will need to upskill for generative AI by 2027, which means the demand for agent compute is growing. But demand growth doesn’t automatically mean your utilization crosses the line. The dominant enterprise pattern isn’t full self-hosting but split architectures where the agent loop and inference stay vendor-managed while only code execution moves on-prem — Cursor Self-Hosted Machines, Devin Outposts, and Coder Agent Relay all fill this middle. Full self-hosting makes sense when a hard policy keeps code, prompts, or model traces inside infrastructure you control, and you have enough concurrent work to amortize the GPU investment.

How should you decide?

Start with the compliance question: can file contents needed for inference and model traces cross a managed boundary? If no, full self-hosting may be mandatory regardless of cost — look at Coder Agents, IBM Bob, or OpenHands Enterprise. If yes, but checkout and tool execution must remain inside your network, Cursor Self-Hosted Machines or Coder Agent Relay split the difference. If both can cross, ask whether measured demand can fill more than 21.28 modeled task equivalents in every paid cluster-hour. Below that line, rented self-hosted GPU time loses before storage and engineering enter the budget.

For teams already running platform infrastructure, Coder Agents extends an existing investment. For teams building from scratch, OpenHands or OpenCode on a self-hosted vLLM stack gives you the most control with the lowest license cost — but you’re hiring an inference team whether you call it that or not. The LiteLLM team’s Moyai is worth a look if you’re already using their gateway and want a Slack-driven cloud agent that keeps your keys. And if you’re an indie developer or small team, the fully offline stack with a local model on a Mac Mini or single server keeps monthly cost at electricity only.

The honest answer: most teams don’t need full self-hosting. They need a network boundary for execution and a spending cap for inference. The vendors selling “self-hosted” as a binary are obscuring the real decision, which is how much of the stack you’re willing to operate — and whether your utilization justifies it.