Baidu releases 280B-parameter ERNIE 5.5 Flash Base model

Baidu has released ERNIE 5.5 Flash Base, a text-only sparse Mixture-of-Experts checkpoint containing 280B parameters and activating 14B for each token. The weights represent pre-training only, with no mid-training or post-training. Baidu says the model was trained from scratch on 33.5T tokens using the Muon optimizer. Its training context is 128K tokens, while fine-tuning and inference support up to 1M tokens.

The model has 43 layers. Thirty-seven use Heavily Compressed Attention, which caches one entry per 128 tokens, while six combine Multi-head Latent Attention with sparse attention. Each MoE layer contains 512 routed experts, selects 10, and includes one shared expert. ECHO reuses the final backbone layer to predict future tokens and support speculative decoding; its 0.36B parameters are excluded from the headline total.

In Baidu’s unified, self-reported evaluation, ERNIE 5.5 Flash Base achieved the top result on 24 of 30 base-model benchmarks, including 78.4 on MMLU-Pro and 82.3 on HumanEval+. It is available in FP8 and BF16 through Hugging Face and ModelScope. Baidu documents serving with SGLang and vLLM, using the completions endpoint rather than chat. The license is free for organizations below US$50M in revenue but excludes the EU, UK and South Korea.