Chain-of-thought (CoT) prompting is a de-facto normal approach to elicit reasoning-like responses from massive language fashions (LLMs), permitting them to spell out particular person steps earlier than giving a remaining reply. Whereas the resemblance to human-like reasoning is plain, the driving forces underpinning the success of CoT reasoning nonetheless stay largely unclear. On this work, we carry out an in-depth evaluation of CoT traces originating from competition-level arithmetic questions, with the goal of higher understanding how, and which components of CoT truly contribute to the ultimate reply. To this finish, we introduce the notion of a possible, quantifying how a lot a given a part of CoT will increase the probability of an accurate completion. Upon examination of reasoning traces by means of the lens of the potential, we determine stunning patterns together with (1) its typically robust non-monotonicity (attributable to reasoning tangents), (2) very sharp however generally powerful to interpret spikes (reasoning insights and jumps) in addition to (3) at occasions fortunate guesses, the place the mannequin arrives on the appropriate reply with out offering any related justifications earlier than. Whereas a few of the behaviours of the potential are readily interpretable and align with human instinct (comparable to insights and tangents), others stay obscure from a human perspective. To additional quantify the reliance of LLMs on reasoning insights, we examine the notion of CoT transferability, the place we measure the potential of a weaker mannequin below the partial CoT from one other, stronger mannequin. Certainly aligning with our earlier outcomes, we discover that as little as 20% of partial CoT can “unlock” the efficiency of the weaker mannequin on issues that had been beforehand unsolvable for it, highlighting that a big a part of the mechanics underpinning CoT are transferable.
