Lengthy-context dealing with stays a core problem for language fashions: even with prolonged context home windows, fashions typically fail to reliably extract, cause over, and use the knowledge throughout lengthy contexts. Current works like Recursive Language Fashions (RLMs) have approached this problem by agentic manner of decomposing lengthy contexts into recursive sub-queries by programmatic interplay at inference. Whereas promising, the success of RLMs critically depends upon how these trajectories of context-interaction applications are chosen, which has remained unexplored. On this paper, we research this drawback and introduce Self-Reflective Program Seek for Lengthy Context (SRLM), a framework that augments programming-based context interplay with uncertainty-aware self-reflection. SRLM leverages three intrinsic alerts: self-consistency, reasoning hint size, and verbalized confidence. These function complementary indicators of a mannequin’s inside uncertainty, and the mannequin makes use of them to guage and evaluate candidate context-interaction applications. In depth experiments throughout various benchmark datasets, context lengths, and spine fashions, present that SRLM persistently outperforms state-of-the-art baselines, yielding as much as 22% enchancment over RLMs underneath the identical time funds. Our findings present that recursion itself shouldn’t be the first driver of efficiency in RLMs, and a easy self-reflective program search can match or surpass RLM with out requiring self-query or express recursion mechanisms. We discover that for context lengths throughout the mannequin’s context window, RLMs with recursion typically degrade efficiency relative to the bottom mannequin, whereas SRLM yields constant and sturdy beneficial properties throughout each quick and lengthy contexts. We additionally discover that RLM is much less efficient in duties with semantically intensive nature, the place heuristic program search is inadequate and broader contextual understanding is required, whereas self-reflection in SRLM gives a semantic sign that higher steers reasoning in these difficult long-context eventualities.
