RT-1: Robotics Transformer for Real-World Control at Scale

ABSTRACT

By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets. We investigate whether this paradigm can be applied to robotics. We present RT-1, a Robotics Transformer model trained on a massive real-world dataset of 130k episodes, collected over 17 months by a fleet of 13 robots. RT-1 can execute over 700 linguistic instructions with a 97% success rate. The model demonstrates unprecedented generalization capabilities, executing novel instructions, operating in unseen backgrounds, and manipulating unobserved objects. We show that simply scaling data and model size systematically improves the robustness of end-to-end robotic control.