42 citations · 70 across the 13 of their papers we have counts for
8 papers · 1 filter
BIG-Bench Extra Hard
Mehran Kazemi, Bahare Fatemi, Hritik Bansal +17
Large language models (LLMs) are increasingly deployed in everyday applications, demanding robust general reasoning capabilities and diverse reasoning skillset. However, current LL…
DSI++: Updating Transformer Memory with New Documents
Sanket Vaibhav Mehta, Jai Gupta, Yi Tay +6
Differentiable Search Indices (DSIs) encode a corpus of documents in model parameters and use the same model to answer user queries directly. Despite the strong performance of DSI…
Train Flat, Then Compress: Sharpness-Aware Minimization Learns More Compressible Models
Clara Na, Sanket Vaibhav Mehta, Emma Strubell
Model compression by way of parameter pruning, quantization, or distillation has recently gained popularity as an approach for reducing the computational requirements of modern dee…
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Vamsi Aribandi, Yi Tay, Tal Schuster +11
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the nu…
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay +3
Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). Such representations are compositional and it is cost…
Efficient Meta Lifelong-Learning with Limited Memory
Zirui Wang, Sanket Vaibhav Mehta, Barnabás Póczos +1
Current natural language processing models work well on a single task, yet they often fail to continuously learn new tasks without forgetting previous ones as they are re-trained t…
Learning Rhyming Constraints using Structured Adversaries
Harsh Jhamtani, Sanket Vaibhav Mehta, Jaime Carbonell +1
Existing recurrent neural language models often fail to capture higher-level structure present in text: for example, rhyming patterns present in poetry. Much prior work on poetry g…
Towards Semi-Supervised Learning for Deep Semantic Role Labeling
Sanket Vaibhav Mehta, Jay Yoon Lee, Jaime Carbonell
Neural models have shown several state-of-the-art performances on Semantic Role Labeling (SRL). However, the neural models require an immense amount of semantic-role corpora and ar…