2.3k citations · 2.5k across the 9 of their papers we have counts for
11 papers
HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
Eric Nguyen, Michael Poli, Marjan Faizi +10
Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation…
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
Daniel Y. Fu, Elliot L. Epstein, Eric Nguyen +5
State space models (SSMs) have high performance on long sequence modeling but require sophisticated initialization techniques and specialized implementations for high quality and r…
Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding
William Chen, Siyi Hu, Rajat Talak +1
Abstract semantic 3D scene understanding is a problem of critical importance in robotics. As robots still lack the common-sense knowledge about household objects and locations of a…
Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data
Armin W. Thomas, Christopher Ré, Russell A. Poldrack
Self-supervised learning techniques are celebrating immense success in natural language processing (NLP) by enabling models to learn from broad language data at unprecedented scale…
Differentiable programming for functional connectomics
Rastko Ciric, Armin W. Thomas, Oscar Esteban +1
Mapping the functional connectome has the potential to uncover key insights into brain organisation. However, existing workflows for functional connectomics are limited in their ad…
Comparing interpretation methods in mental state decoding analyses with deep learning models
Armin W. Thomas, Christopher Ré, Russell A. Poldrack
Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e.g., perceiving fear or joy…
Evaluating deep transfer learning for whole-brain cognitive decoding
Armin W. Thomas, Ulman Lindenberger, Wojciech Samek +1
Research in many fields has shown that transfer learning (TL) is well-suited to improve the performance of deep learning (DL) models in datasets with small numbers of samples. This…
Challenges for cognitive decoding using deep learning methods
Armin W. Thomas, Christopher Ré, Russell A. Poldrack
In cognitive decoding, researchers aim to characterize a brain region's representations by identifying the cognitive states (e.g., accepting/rejecting a gamble) that can be identif…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
Deep Transfer Learning For Whole-Brain fMRI Analyses
Armin W. Thomas, Klaus-Robert Müller, Wojciech Samek
The application of deep learning (DL) models to the decoding of cognitive states from whole-brain functional Magnetic Resonance Imaging (fMRI) data is often hindered by the small s…
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models
Armin W. Thomas, Hauke R. Heekeren, Klaus-Robert Müller +1
The application of deep learning (DL) models to neuroimaging data poses several challenges, due to the high dimensionality, low sample size and complex temporo-spatial dependency s…