Sakhanda Wire
NVDA $152.94 -2.05% MSFT $505.64 -2.38% GOOGL $280.11 -1.27% META $619.47 -1.11% AMZN $256.54 -0.47%
← Back to the news

Linkup Research releases 149M-parameter SPARSEUP sparse encoder

Linkup Research has released SPARSEUP, an open-source learned sparse embedding model built on a 149M-parameter ModernBERT backbone. The model is distributed under the Apache 2.0 license, with weights available on Hugging Face, and can be loaded through Transformers or Sentence Transformers using trust_remote_code=True. Unlike dense retrievers, it assigns weights to vocabulary tokens, allowing its representations to work with inverted indexes and preserve interpretable terms.

The team trained SPARSEUP from the LateOn-unsupervised checkpoint with contrastive learning, seven hard negatives per query and in-batch negatives. It restored ModernBERT’s masked-language-modeling head and introduced a logit shift, per-token top-12 vocabulary expansion and case folding. These changes reduced the output vocabulary dimensions from about 50,000 to roughly 34,000.

Linkup reports an average score of 56.4 nDCG@10 on BEIR-13, calling it the strongest public sparse encoder it knows of below 150M parameters. With the same backbone and data, SPARSEUP scored below LateOn at 58.9 and DenseOn at 57.9. Using a Seismic inverted index, it achieved over 97% recall against exact search in about 380 microseconds per query on MS MARCO, with single-threaded evaluation.

Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.

Originally published by MarkTechPost on

Read the original on MarkTechPost ↗

The reporting and the image belong to MarkTechPost and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news