In a previous report we showed that a decoder can recover open-vocabulary word-level information from 19-channel EEG recorded during silent reading, by aligning short EEG windows with hidden-state embeddings of the presented word taken from a large language model (LLM).
That analysis contrasted two of the model's 33 layers: the context-free input embedding and one mid-depth layer.
Current Study
Here we ask how EEG-decodable structure is distributed over the full depth of the language model.
We trained one decoder per layer of Llama-3.1-8B, with and without a causal transformer over the sequence of EEG windows.
We distinguish word-level decodability from a context-tracking component.
Word-level decodability peaks in early layers and declines steadily towards the last layer.
In contrast, the context-tracking component of the retrieval gain rises with depth, both with and without a causal transformer.
Unexpectedly, a decoder that sees a single 0.5-s window at a time tracks the text passage too.
New Controls
Two new controls dissect this component.
A target-geometry control, a synthetic predictor made of the true target plus noise calibrated to each decoder's word-level accuracy, shows that context tracking is not a by-product of word decoding and target geometry.
A control that permutes predictions across runs from either the same or from different sessions shows that most of the observed context-tracking is not caused by an EEG recording session signature.
Instead, context tracking mostly follows the text.
Conclusions
A single EEG window therefore carries passage-level information that a contextual target can absorb.
The results locate word identity in the early layers of the language model, discourse state in the deeper layers, and make the choice of target layer a design dial for EEG language decoders.