Linkup Research releases 149M-parameter SPARSEUP sparse encoder
Linkup Research has released SPARSEUP, an open-source learned sparse embedding model built on a 149M-parameter ModernBERT backbone. The model is distributed under the Apache 2.0 license, with weights available on Hugging Face, and can be loaded through Transformers or Sentence Transformers using trust_remote_code=True. Unlike dense retrievers, it assigns weights to vocabulary tokens, allowing its representations to work with inverted indexes and preserve interpretable terms.
The team trained SPARSEUP from the LateOn-unsupervised checkpoint with contrastive learning, seven hard negatives per query and in-batch negatives. It restored ModernBERT’s masked-language-modeling head and introduced a logit shift, per-token top-12 vocabulary expansion and case folding. These changes reduced the output vocabulary dimensions from about 50,000 to roughly 34,000.
Linkup reports an average score of 56.4 nDCG@10 on BEIR-13, calling it the strongest public sparse encoder it knows of below 150M parameters. With the same backbone and data, SPARSEUP scored below LateOn at 58.9 and DenseOn at 57.9. Using a Seismic inverted index, it achieved over 97% recall against exact search in about 380 microseconds per query on MS MARCO, with single-threaded evaluation.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.