Tiny Recursive Reasoning with Mamba-2 Attention Hybrid

ABSTRACT

Recent work demonstrates that tiny networks (7M parameters) can achieve strong performance on abstract reasoning through latent recursion. We investigate whether Mamba-2's state space recurrence, itself a form of iterative refinement, preserves reasoning capability when replacing Transformer blocks in a recursive scaffold. Maintaining parameter parity (6.8M), we find that the Mamba-2 hybrid improves pass@2 on ARC-AGI-1 by +2.0% and consistently outperforms at higher K values. Our results validate that SSM-based operators are viable candidates in recursive design, establishing a first step toward understanding the best mixing strategies for 'more thinking time' over 'bigger models'.