HarnessDev finds most agent-harness changes fail to generalize
Researchers from ByteDance Seed, Singapore University of Technology and Design, Georgia Institute of Technology, M-A-P, and TokenWave.AI introduced HarnessDev, a benchmark that evaluates the runnable software surrounding a language model rather than only its answers. The harness includes execution, tools, context, state, recovery, and verification. Six models started from the same minimal seed, which scored 0 without a loop, planner, verifier, retry, or stopping rule, and built harnesses for five benchmarks covering four domains.