StateBench measures how models track changing information in long documents
Researchers have introduced StateBench, a benchmark for evaluating how language models follow changing information about entities across long documents. The tasks reflect situations such as variables being reassigned in code, characters moving in a story, or objects changing hands. They range from finding an unchanging or most recent value to following cumulative updates and dependencies between variables.