Article

Predicting brain activity using Transformers

2023-08-05

Abstract excerpt

The Algonauts challenge [Gifford et al., 2023] called on the community to provide novel solutions for predicting brain activity of humans viewing natural scenes. This report provides an overview and technical details of our submitted solution. We use a general transformer encoder-decoder model to map images to fMRI responses. The encoder model is a vision transformer trained using self-supervised methods (DINOv2)....

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: e13a9b76-ce29-5cc4-a24c-c5dcf6c8f6f7
DOI: 10.1101/2023.08.02.551743

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Predicting brain activity using TransformersDOI 10.1101/2023.08.02.551743Read

NEuRT: A Transformer-Based Model for Explainable Neuronal Activity AnalysisSemantic proximity Read

BrainLM: A foundation model for brain activity recordingsSemantic proximity Read

Variational Autoencoder: An Unsupervised Model for Modeling and Decoding fMRI Activity in Visual CortexSemantic proximity Read

Cortical response to naturalistic stimuli is largely predictable with deep neural networksSemantic proximity Read

A large and rich EEG dataset for modeling human visual object recognitionSemantic proximity Read

st-DenseViT: A Weakly Supervised Spatiotemporal Vision Transformer for Dense Prediction of Dynamic Brain NetworksSemantic proximity Read

THINGSvision: a Python toolbox for streamlining the extraction of activations from deep neural networksSemantic proximity Read

Large-scale hyperparameter search for predicting human brain responses in the Algonauts challengeSemantic proximity Read

Functional Annotation of Human Cognitive States using Deep Graph ConvolutionSemantic proximity Read

Shared Hierarchical Representations Explain Temporal Correspondence Between Brain Activity and Deep Neural NetworksSemantic proximity Read

Interpretable Transformer Models for rs-fMRI Epilepsy Classification and Biomarker DiscoverySemantic proximity Read

Gray Matters: An Efficient Vision Transformer GAN Framework for Predicting Functional Network Connectivity Biomarkers from Brain StructureSemantic proximity Read