Publications (17)
Personalized Benchmarking with the Ludwig Benchmarking Toolkit
Avanika Narayan, Piero Molino, Karan Goel +2
The rapid proliferation of machine learning models across domains and deployment settings has given rise to various communities (e.g. industry practitioners) which seek to benchmar…
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
Albert Gu, Isys Johnson, Karan Goel +4
Recurrent neural networks (RNNs), temporal convolutions, and neural differential equations (NDEs) are popular families of deep learning models for time-series data, each with uniqu…
PLOTS: Procedure Learning from Observations using Subtask Structure
Tong Mu, Karan Goel, Emma Brunskill
In many cases an intelligent agent may want to learn how to mimic a single observed demonstrated trajectory. In this work we consider how to perform such procedural learning from o…
Mandoline: Model Evaluation under Distribution Shift
Mayee Chen, Karan Goel, Nimit S. Sohoni +3
Machine learning models are often deployed in different settings than they were trained and validated on, posing a challenge to practitioners who wish to predict how well the deplo…
Effectively Modeling Time Series with Simple Discrete State Spaces
Michael Zhang, Khaled K. Saab, Michael Poli +3
Time series modeling is a well-established problem, which often requires that methods (1) expressively represent complicated dependencies, (2) forecast long horizons, and (3) effic…
It's just a matter of perspective(s): Crowd-Powered Consensus Organization of Corpora
Ayush Jain, Joon Young Seo, Karan Goel +3
We study the problem of organizing a collection of objects - images, videos - into clusters, using crowdsourcing. This problem is notoriously hard for computers to do automatically…
Efficiently Modeling Long Sequences with Structured State Spaces
Albert Gu, Karan Goel, Christopher Ré
A central goal of sequence modeling is designing a single principled model that can address sequence data across a range of modalities and tasks, particularly on long-range depende…
It's Raw! Audio Generation with State-Space Models
Karan Goel, Albert Gu, Chris Donahue +1
Developing architectures suitable for modeling raw audio is a challenging problem due to the high sampling rates of audio waveforms. Standard sequence modeling approaches like RNNs…
On the Parameterization and Initialization of Diagonal State Space Models
Albert Gu, Ankit Gupta, Karan Goel +1
State space models (SSM) have recently been shown to be very effective as a deep learning layer as a promising alternative to sequence models such as RNNs, CNNs, or Transformers. T…
Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
Karan Goel, Albert Gu, Yixuan Li +1
Classifiers in machine learning are often brittle when deployed. Particularly concerning are models with inconsistent performance on specific subgroups of a class, e.g., exhibiting…
Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems
Laurel Orr, Atindriyo Sanyal, Xiao Ling +2
The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed…
Robustness Gym: Unifying the NLP Evaluation Landscape
Karan Goel, Nazneen Rajani, Jesse Vig +6
Despite impressive performance on standard benchmarks, deep neural networks are often brittle when deployed in real-world systems. Consequently, recent research has focused on test…
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.…
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces
Eric Nguyen, Karan Goel, Albert Gu +5
Visual data such as images and videos are typically modeled as discretizations of inherently continuous, multidimensional signals. Existing continuous-signal models attempt to expl…
SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization
Jesse Vig, Wojciech KryÅciÅski, Karan Goel +1
Novel neural architectures, training strategies, and the availability of large-scale corpora haven been the driving force behind recent progress in abstractive text summarization.…
Sample Efficient Policy Search for Optimal Stopping Domains
Karan Goel, Christoph Dann, Emma Brunskill
Optimal stopping problems consider the question of deciding when to stop an observation-generating process in order to maximize a return. We examine the problem of simultaneously l…
Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing
Karan Goel, Shreya Rajpal, Mausam
We present Octopus, an AI agent to jointly balance three conflicting task objectives on a micro-crowdsourcing marketplace - the quality of work, total cost incurred, and time to co…