Zoom In: An Introduction to Circuits

ABSTRACT

Neural networks are generally regarded as opaque black boxes. However, if we zoom in and carefully examine the weights and activations of convolutional neural networks, we find highly interpretable, human-understandable features. We propose that neural networks are composed of 'circuits'—computational sub-graphs consisting of linked, meaningful features. We demonstrate that early vision layers detect curves and high-low frequencies, which later combine into complex object detectors like dog heads or car wheels. By mapping these circuits, we provide a framework for reverse-engineering artificial intelligence, moving from empirical observation to mechanistic understanding.