Language models can explain neurons in language models

ABSTRACT

Understanding the internal mechanisms of massive language models is a critical bottleneck for AI safety and alignment. Given the billions of parameters in modern models, manual human inspection of individual neurons is impossible. We introduce an automated interpretability pipeline that uses a highly capable language model (GPT-4) to generate natural language explanations for the behavior of individual neurons in a smaller, target model (GPT-2). We then use the explainer model to simulate the neuron's activations and score the explanation based on its accuracy. Using this automated pipeline, we successfully generated explanations for all 307,200 neurons in GPT-2, marking a significant step toward scalable mechanistic interpretability.