Preprint2026
CAST: Game Solvers as Turn-Level Teachers for LLM Agents
Yu Wang, Yi-Kai Zhang, Wentao Shi, Ziang Ye et al. · arXiv · 2026
Abstract
Reinforcement learning on long-horizon games hands the model one reward at the end, which says nothing about which of fifty moves mattered. This work uses a classical game solver's value function as a per-turn teacher: the change in solver value after an action becomes that turn's credit. The authors show this is equivalent to on-policy distillation from the solver while needing only scalar values, never teacher logits.
Why it matters
A clean example of borrowing a signal from classical AI to fix a modern training problem. Wherever a solver already exists, dense per-step supervision turns out to be nearly free.
reinforcement learningagentscredit assignmentdistillation
Read the source
https://arxiv.org/abs/2607.25308