<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual stimuli from cerebral signals could help us better understand the processes by which the human brain generates visual imagery. However, the inherent complexity and significant noise in brain signals limit current efforts to reconstruct visual stimuli, resulting in low-granularity images that miss details. To address these challenges, this paper proposes EEGVision, a comprehensive framework for generating high-quality images directly from brain signals. Leveraging the recent strides in multi-modal models within the realm of deep learning, it is now feasible to bridge the gap between EEG data and visual representation. This process starts with a time-frequency fusion encoder in EEGVision, which quickly pulls out cross-domain and robust features from EEG signals. We then design two parallel pipelines to align EEG embeddings with image features at both perceptual and semantic levels. The process uses a stable diffusion-trained image-to-image pipeline that combines coarse and fine-grained data to get high-quality images back from EEG data. Both quantitative and qualitative assessments affirm that EEGVision surpasses contemporary benchmarks. This network architecture holds promise for further applications in the domain of neuroscience, aiming to unravel the genesis of human visual perception mechanisms. All code is accessible via <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://github.com/AvancierGuo/EEGVision">https://github.com/AvancierGuo/EEGVision</ext-link>.</p>
</abstract>

The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual stimuli from cerebral signals could help us better understand the processes by which the human brain generates visual imagery. However, the inherent complexity and significant noise in brain signals limit current efforts to reconstruct visual stimuli, resulting in low-granularity images that miss details. To address these challenges, this paper proposes EEGVision, a comprehensive framework for generating high-quality images directly from brain signals. Leveraging the recent strides in multi-modal models within the realm of deep learning, it is now feasible to bridge the gap between EEG data and visual representation. This process starts with a time-frequency fusion encoder in EEGVision, which quickly pulls out cross-domain and robust features from EEG signals. We then design two parallel pipelines to align EEG embeddings with image features at both perceptual and semantic levels. The process uses a stable diffusion-trained image-to-image pipeline that combines coarse and fine-grained data to get high-quality images back from EEG data. Both quantitative and qualitative assessments affirm that EEGVision surpasses contemporary benchmarks. This network architecture holds promise for further applications in the domain of neuroscience, aiming to unravel the genesis of human visual perception mechanisms. All code is accessible via https://github.com/AvancierGuo/EEGVision.

EEGVision: Reconstructing vision from human brain signals

China University of Petroleum (East China), Qingdao

Applied Mathematics and Nonlinear Sciences

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"EEGVision: Reconstructing vision from human brain signals"}

The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual...

EEGVision: Reconstructing vision from human brain signals

Data publikacji: 05 sie 2024

Zakres stron: -

Otrzymano: 11 kwi 2024

Przyjęty: 28 cze 2024

DOI: https://doi.org/10.2478/amns-2024-1856

Słowa kluczowe
Visual Reconstruction, Human visual perceptions, EEGVision, Multi-modal models, Neuroscience

© 2024 Huangtao Guo., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

EEGVision: Reconstructing vision from human brain signals

Data publikacji: 05 sie 2024

Zakres stron: -

Otrzymano: 11 kwi 2024

Przyjęty: 28 cze 2024

DOI: https://doi.org/10.2478/amns-2024-1856

Słowa kluczoweVisual Reconstruction, Human visual perceptions, EEGVision, Multi-modal models, Neuroscience

© 2024 Huangtao Guo., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Słowa kluczowe
Visual Reconstruction, Human visual perceptions, EEGVision, Multi-modal models, Neuroscience