Sample-Efficient Learning from Agent Experience

Abstract

Agents learn quickly from their own interaction history while it sits in the context window, and lose those gains the moment it is removed. This work internalises that history into the weights instead, using context distillation over already-collected experience with no further environment interaction. Across software-engineering tasks and text adventures it retains at least 65 percent of the in-context gain, where ordinary fine-tuning on the same data keeps under 4 percent.

https://arxiv.org/abs/2607.21051