STAR: Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction

ABSTRACT

The paper introduces STAR (STatistical and Agentic Reasoning), a novel framework designed to predict the performance of large models across diverse benchmarks from limited observations. Existing statistical methods often struggle with pattern shifts and lack of interpretability, while pure LLM approaches suffer from hallucination. STAR bridges these gaps by combining data-driven statistical expectations with knowledge-driven agentic reasoning, grounded in Expectation Violation Theory (EVT). STAR consistently outperformed all baselines under extreme sparsity (95% masking), achieving a 14.46% gain over standard PMF and demonstrating strong generalization to unseen task categories like Math and OCR.