Prime Intellect launches serving platform for open models
Prime Intellect has introduced Prime Inference, a platform for serving frontier open-source models through serverless endpoints or reserved capacity on the company’s GPUs across multiple datacenters. Before its public release, the system handled nearly a trillion tokens per day in internal workloads including reinforcement-learning rollouts, synthetic-data generation, evaluations and long-running coding agents.
Prime Inference is the serving layer of Prime Intellect’s open training stack, linking deployed model outputs with later training activity. The platform is OpenAI-compatible and supports automatic failover between datacenters. NVIDIA Blackwell is available now, while Vera Rubin is listed as forthcoming. Prime says its GLM-5.3 endpoint has ranked among the fastest on OpenRouter and reports near-zero tool-call errors and 100% uptime since launch.
The serving stack combines NVIDIA Dynamo, vLLM, Mooncake and FlashInfer, with prefill and decoding handled by separate GPU groups. On a GB200 NVL72 system, a 1:4 prefill-to-decode ratio served 66 sessions per prefill group at 101 tokens per second per user. NVFP4 key-value compression increased cached capacity per decoder from 1.09 million to 1.63 million tokens. Batch inference and one-click dedicated deployments are on the roadmap.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.