SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

ABSTRACT

SkillRL is a framework that enables LLM agents to learn high-level, reusable behavioral patterns from past experiences. While traditional memory-based methods store redundant and noisy raw trajectories, SKILLRL abstracts these into a hierarchical skill library. Key features include Experience-based Skill Distillation, a Hierarchical SKILLBANK for universal and task-specific heuristics, and Recursive Skill Evolution where the library co-evolves with the agent's policy. SkillRL achieves 10-20% token compression compared to raw trajectory storage while significantly enhancing reasoning utility in complex environments.