Identifying Intervenable and Interpretable Features via Orthogonality Regularization

ABSTRACT

This paper addresses the fundamental challenge of 'feature disentanglement' in modern deep learning. We propose an Orthogonality Regularization technique to identify features that are both interpretable to humans and physically intervenable by agents. By enforcing a structured geometric prior on the representation space, we demonstrate that models can learn to separate causal factors from spurious correlations. This is a critical step toward 'verifiable reasoning' in autonomous systems, allowing human overseers to audit why an agent chose a specific trajectory. We validate our approach across several benchmarks, showing significant improvements in both model transparency and downstream task robustness.