Evaluating spatio-temporal fusion of EEG and fMRI with iEEG
Poster Presentation 33.338: Sunday, May 17, 2026, 8:30 am – 12:30 pm, Banyan Breezeway
Session: Scene Perception: Spatiotemporal factors
Schedule of Events | Search Abstracts | Symposia | Talk Sessions | Poster Sessions
Peter Brotherwood1 (peter.brotherwood@umontreal.ca), Dora Hermes2, Jacob S. Prince3, Kendrick Kay4, Frederic Gosselin1, Ian Charest1; 1Université de Montréal, 2Mayo Clinic, 3Harvard University, 4University of Minnesota
Understanding how visual information unfolds across space and time in the human brain is limited by the fact that no single neuroimaging method provides precise spatial and temporal resolution. To overcome this, we recently introduced a data-driven method that learns how moment-to-moment EEG activity maps onto fMRI response patterns (Charest et al., 2025). Using the Natural Scenes Dataset (NSD; Allen et al., 2022) and the NSD-EEG (Brotherwood et al., 2024), we trained fractional-ridge regression (Rokem & Kay, 2020) models that, at each EEG time point, learn a set of linear weights predicting vertex-wise fMRI beta patterns from electrode amplitudes in response to the same images. The resulting time series of prediction performance recovers the known spatio-temporal hierarchy of the visual cortex: prediction performance in early visual regions (V1/V2) peaks first at ~60–80 ms, while in high-level visual regions performance peaks around 300–400 ms. To assess the accuracy of fusion-derived cortical activity estimates we compared it with broadband intracranial EEG (iEEG) responses recorded in participants viewing the same NSD images (Huang et al., 2024). We averaged both datasets across stimuli to focus on visual responsiveness, and correlated each electrode’s broadband time course with the predicted time course of every cortical vertex. Correlation values decreased systematically with geodesic distance: electrodes nearest a vertex showed the strongest similarity to that vertex’s predicted time course (peak vertex in LOC: R≈0.8), and similarity declined with distance (slope ≈ –0.54). This spatial fall-off indicates that the fusion successfully identifies the local cortical generators most aligned with the iEEG measurements. These findings suggest that our novel EEG-fMRI fusion offers anatomically precise, millisecond-resolved cortical activity estimates whose spatial organisation is now validated directly against intracranial electrophysiology. This opens a new avenue for studying the dynamics of visual processing across the human cortex, overcoming longstanding spatial–temporal trade-offs.