7 citations · 9 across the 9 of their papers we have counts for
11 papers
Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability
Jeffrey Willette, Krishna C. Puvvada, Boris Ginsburg
Long-horizon agentic workflows require models to sustain repeated state-dependent actions all while the context grows, sub-task complexity changes, and new data arrives. Each situa…
Robust Molecular Property Prediction via Densifying Scarce Labeled Data
Jina Kim, Jeffrey Willette, Bruno Andreis +1
A widely recognized limitation of molecular prediction models is their reliance on structures observed in the training data, resulting in poor generalization to out-of-distribution…
Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction
Jeffrey Willette, Heejun Lee, Sung Ju Hwang
The attention mechanism of a transformer has a quadratic complexity, leading to high inference costs and latency for long sequences. However, attention matrices are mostly sparse,…
Training-Free Exponential Context Extension via Cascading KV Cache
Jeffrey Willette, Heejun Lee, Youngwan Lee +2
The transformer's context window is vital for tasks such as few-shot learning and conditional generation as it preserves previous tokens for active memory. However, as the context…
Visualizing the loss landscape of Self-supervised Vision Transformer
Youngwan Lee, Jeffrey Ryan Willette, Jonghee Kim +1
The Masked autoencoder (MAE) has drawn attention as a representative self-supervised approach for masked image modeling with vision transformers. However, even though MAE shows bet…
SEA: Sparse Linear Attention with Estimated Attention Mask
Heejun Lee, Jina Kim, Jeffrey Willette +1
The transformer architecture has driven breakthroughs in recent years on tasks which require modeling pairwise relationships between sequential elements, as is the case in natural…
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Youngwan Lee, Jeffrey Willette, Jonghee Kim +2
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers. A representative MIM model, the mas…
Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
Jeffrey Willette, Seanie Lee, Bruno Andreis +3
Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaran…
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
Jeffrey Willette, Hae Beom Lee, Juho Lee +1
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer. This d…
Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding
Bruno Andreis, Jeffrey Willette, Juho Lee +1
Most existing set encoding algorithms operate under the implicit assumption that all the set elements are accessible, and that there are ample computational and memory resources to…
Improving Uncertainty Calibration via Prior Augmented Data
Jeffrey Willette, Juho Lee, Sung Ju Hwang
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. However, they are often overconfident in their pre…