Bastion is an inference runtime that serves open LLMs inside your own datacenter or VPC. Your data never leaves your network — and you stop paying per token to sit behind someone else's rate limits.
We benchmark Bastion on your model and your hardware. No data leaves your side.
Thanks — we'll reach out to set up a benchmark within a day.
| throughput | p50 latency | $ / 1M tok | egress | |
|---|---|---|---|---|
| Hosted API | 2,400 tok/s | 210 ms | $2.80 | all prompts |
| Bastion (on-prem) | 5,900 tok/s | 90 ms | $0.42 | none |
Illustrative. Actual numbers come from a benchmark on your workload.
A single container or Helm chart into your Kubernetes cluster or bare metal. Air-gapped installs are supported. No traffic to us, no model weights leaving your perimeter, no third-party API in the path.
Works with your existing GPUs — H100, A100, L40S, MI300X.
Continuous batching, paged attention, FP8 and INT4 quantization, and speculative decoding — with kernels tuned per GPU. The same weights that crawl on a naive server saturate your hardware here.
OpenAI-compatible endpoint, so your apps point at it without a rewrite.
Per-token usage and cost metrics, autoscaling across your nodes, SSO, and audit logs. The people who need on-prem inference usually need to prove where the data went, too.
Every request stays inside your compliance boundary, by construction.
Teams running real inference volume under data-residency or sovereignty constraints — where sending prompts to a hosted API is either a compliance problem or a line-item that grows with every user. If you've already decided the models run on your side, Bastion is the runtime that makes them fast.
Tell us the model and the hardware you're targeting. We'll run Bastion against your current setup and send back throughput, latency, and cost — measured, not promised.
Thanks — we'll reach out to set up a benchmark within a day.