223 citations · 654 across the 16 of their papers we have counts for
7 papers · 1 filter
SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
Dara Bahri, Heinrich Jiang, Yi Tay +1
Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders o…
Churn Reduction via Distillation
Heinrich Jiang, Harikrishna Narasimhan, Dara Bahri +2
In real-world systems, models are frequently updated as more data becomes available, and in addition to achieving high accuracy, the goal is to also maintain a low difference in pr…
Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection
Dara Bahri, Heinrich Jiang, Yi Tay +1
Detecting out-of-distribution (OOD) examples is critical in many applications. We propose an unsupervised method to detect OOD samples using a -NN density estimate with respect…
Locally Adaptive Label Smoothing for Predictive Churn
Dara Bahri, Heinrich Jiang
Training modern neural networks is an inherently noisy process that can lead to high \emph{prediction churn} -- disagreements between re-trainings of the same model due to factors…
Efficient Transformers: A Survey
Yi Tay, Mostafa Dehghani, Dara Bahri +1
Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision and reinforcement learning. In the…
Deep k-NN for Noisy Labels
Dara Bahri, Heinrich Jiang, Maya Gupta
Modern machine learning models are often trained on examples with noisy labels that hurt performance and are hard to identify. In this paper, we provide an empirical study showing…
Sparse Sinkhorn Attention
Yi Tay, Dara Bahri, Liu Yang +2
We propose Sparse Sinkhorn Attention, a new efficient and sparse method for learning to attend. Our method is based on differentiable sorting of internal representations. Concretel…