Study finds agent memory works best when calibrated to model capability
An evaluation of agentic memory across eight models found that adding more stored guidance does not consistently improve performance. The ALTK-Evolve method distills lessons, strategies and mistakes from an agent’s previous task trajectories, then supplies them during inference without changing model weights or requiring human annotation. The appropriate amount of guidance varied by model capability.
Stronger models with remaining performance headroom generally benefited from receiving the complete guideline set. DeepSeek-V3.2 improved task completion by 9.5 percentage points, while Claude Opus 4.6 gained 4.1 points. For gpt-oss-120b, a compact core supplemented with task-specific retrieval was more effective: task completion rose 16.1 points with only 5% more tokens, compared with a 51% increase for the full set. GLM-5 showed no measurable improvement, a pattern the authors describe as saturated rather than a proven causal explanation.
The tests used 585 multi-step AppWorld tasks across nine simulated applications. The authors say curated retrieval can improve accuracy and control cost, while prompt caching can make repeatedly supplied static guidance more affordable. They also identify learned guideline selection, weaker models and experiments separating context-window size from model capability as areas for further work.
Written by Sakhanda Wire AI from reporting by Hugging Face. Facts, quotes and full context are in the original article below.