Zyphra presents tree-based sparse attention for long-context models
Zyphra has introduced Zyphra Hierarchical Attention (ZHA), a sparse attention method that uses a tree-structured index to select relevant entries from a model’s key-value cache. Unlike approaches whose indexer scores every previous token, ZHA makes the search hierarchical. It groups keys into chunks, creates summary keys level by level and uses top-p routing to expand only branches carrying the estimated attention mass. The complete KV cache remains stored, while the tree determines which entries are read.