AgentBench-2025: Evaluating Autonomous Swarms in Adversarial and Dynamic Environments

ABSTRACT

As multi-agent swarms are deployed in open-ended web environments, standard static benchmarks fail to capture their vulnerability to dynamic threats and deceptive actors. We introduce AgentBench-2025, a comprehensive evaluation suite that places agentic swarms in simulated, adversarial corporate networks. The benchmark evaluates agents on task completion, API rate-limit management, and resistance to active 'Intent Redirection' attacks by rogue agents. Our results reveal that while frontier models excel in cooperative tasks, they fail catastrophically when forced to verify the authority of conflicting instructions in dynamic environments.