MEG-XL: Data-Efficient Brain-to-Text via Long-Context Pre-Training

ABSTRACT

Decoding natural language from non-invasive brain recordings like Magnetoencephalography (MEG) remains a significant challenge due to the low signal-to-noise ratio and the scarcity of paired brain-speech data. We propose MEG-XL, a framework for data-efficient brain-to-text decoding that leverages long-context pre-training on large-scale unsupervised MEG datasets. By treating MEG signals as a continuous temporal sequence and using a transformer-based architecture with extended context windows, we demonstrate that pre-training on diverse neural activity significantly improves performance on downstream decoding tasks. MEG-XL achieves state-of-the-art results in decoding continuous speech, showing that long-range temporal dependencies are crucial for capturing the linguistic structures embedded in neural signals.