Sakhanda Wire
NVDA $155.46 -0.44% MSFT $506.24 -2.27% GOOGL $280.20 -1.24% META $618.87 -1.20% AMZN $257.89 +0.05%
← Back to the news

Prime Intellect launches serving platform for open models

Prime Intellect has introduced Prime Inference, a platform for serving frontier open-source models through serverless endpoints or reserved capacity on the company’s GPUs across multiple datacenters. Before its public release, the system handled nearly a trillion tokens per day in internal workloads including reinforcement-learning rollouts, synthetic-data generation, evaluations and long-running coding agents.

Prime Inference is the serving layer of Prime Intellect’s open training stack, linking deployed model outputs with later training activity. The platform is OpenAI-compatible and supports automatic failover between datacenters. NVIDIA Blackwell is available now, while Vera Rubin is listed as forthcoming. Prime says its GLM-5.3 endpoint has ranked among the fastest on OpenRouter and reports near-zero tool-call errors and 100% uptime since launch.

The serving stack combines NVIDIA Dynamo, vLLM, Mooncake and FlashInfer, with prefill and decoding handled by separate GPU groups. On a GB200 NVL72 system, a 1:4 prefill-to-decode ratio served 66 sessions per prefill group at 101 tokens per second per user. NVFP4 key-value compression increased cached capacity per decoder from 1.09 million to 1.63 million tokens. Batch inference and one-click dedicated deployments are on the roadmap.

Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.

Originally published by MarkTechPost on

Read the original on MarkTechPost ↗

The reporting and the image belong to MarkTechPost and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news